Python for AI research

00:00 We all know that Python is a major player in the application of machine learning and AI.

00:04 This often involves grabbing Keras or TensorFlow and applying it to a problem.

00:08 But what about AI research? When you're actually trying to create something that has yet to be

00:13 created, how do researchers use Python here? Today you'll meet Alex Levin, a Python developer

00:19 and research scientist at Vicarious, where they're trying to develop artificial general

00:24 intelligence for robots. This is Talk Python to Me, episode 124, recorded May 31st, 2017.

00:31 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the

01:01 ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter,

01:05 where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm

01:10 and follow the show on Twitter via at Talk Python. This episode is brought to you by Linode and

01:16 Talk Python Training. Be sure to check out what the offers are for both of these segments.

01:20 It really helps support the show. Alex, welcome to Talk Python.

01:24 Hi, thanks for having me.

01:25 Yeah, I'm pretty excited to talk about AI, researching AI, and all the stuff that you guys

01:31 are up to. It's something we haven't really touched that much on the show, and it's

01:35 both interesting at a low-level, technical level, but also at a philosophical level, almost. So

01:42 looking forward to talking to you about it.

01:43 Yeah, but before we get into the details, let's start with your story. How did you get into programming

01:48 in Python?

01:48 Well, my background is a bit untraditional for what I do. I actually studied mechanical

01:53 and aerospace engineering at school, and I was determined to make a career as a spacecraft

02:00 engineer. And even some of the Mechie assignments where we used MATLAB, I hated those. But that

02:09 eventually led me to Carnegie Mellon to study space robotics.

02:13 Oh, nice.

02:13 And the atmosphere there is incredible. It's hard not to get caught up in software and AI. And

02:19 that was really my first exposure to AI.

02:21 That's cool. When I think of universities that are cool and amazing in programming and robotics

02:28 and things like that, Carnegie Mellon is right up there. Maybe Stanford, MIT, Carnegie Mellon. Those

02:35 are the top three, at least in my mind.

02:37 Yeah, absolutely. And I was fortunate enough to work with this incredible roboticist, Red

02:42 Whitaker, who was one of the, I guess, founders of the DARPA Grand Challenge of autonomous vehicles.

02:48 Oh, I was going to ask you about the DARPA Grand Challenge. Tell people about what that is. That

02:52 is such an amazing thing.

02:54 Yeah, so that started back in 2003 or 2004. And it's just a challenge to create an autonomous vehicle that drives across the desert down in Mexico. And the first year was, you ask most people, a complete failure, but it really kicked off all this research. And then the next year, it was won by, I believe, Sebastian Thrun and his team from Stanford.

03:17 Yeah, I think so.

03:18 Now it's just almost a piece of cake for these autonomous vehicles to complete the course.

03:22 Didn't the Stanford team eventually more or less move over to Google for their Google car stuff? Do I have that put together right?

03:30 Yeah, I believe a lot of them went over there. And Sebastian Thrun, I think he is very involved in the Udacity effort. They have a self-driving cohort.

03:40 Okay. Yeah, yeah. That's really cool. So when you talk about the car driving through the desert, it's not like it just drove a little ways. There's basically a dirt path through the open plains as well up into the mountains for like 100 miles, right?

03:58 Oh, yeah. You can check out some of the videos from these races. And these are like very intense off-road vehicles. It's through the desert in Mexico, I think on the Baja Strip. And I want to say it's like hundreds of miles.

04:13 Yeah, yeah, that's really cool. So yeah, there's a great Nova on this called the Great Robot Race. I'll be sure to include it in the show notes.

04:21 Oh, cool.

04:21 Yeah, it's really, really neat. But it's super inspiring. And this is not the first year, but the second year that they did the documentary. So it's not where they all failed. And they had different devices, different machines. People had robotic motorcycles. So people had huge trucks. And it seems like the SUVs were the real winners, though.

04:42 Yeah. And it's also worth checking out the DARPA challenge for autonomous human robots. There are some pretty funny fail videos up there on YouTube.

04:53 Okay, so I derailed your story, but I definitely wanted to just talk about that because that's like that was the world that you were in at Carnegie Mellon, right? And you're like, well, okay, this requires programming, right?

05:04 Yeah, yeah. So I was studying mechanical engineering and doing things like CAD and FEA analysis and all that. And my last semester there, I ended up taking one software, well, it was a software slash AI course. And it was all in MATLAB, of course.

05:21 And that really just got me hooked. I loved it. And I figured after graduation, I would just figure out a way to just pivot my career. And I wanted to do AI and software and AI research specifically.

05:37 And then, well, Python seemed to be a natural fit. And I started teaching myself Python. And then, at the same time, dove into any AI textbook and paper I could get my hands on. And then, as soon as I could, started implementing some of these algorithms and models in Python.

05:56 And some of the motivation for that came from things like Kaggle competitions and such. Eventually, I felt comfortable enough as a programmer and AI researcher to seek a position with a few different companies. And that led me to Numenta. And I worked there for a few years.

06:16 Cool. What kind of stuff did you do there?

06:17 So, Numenta is interesting. They run the full gambit from theoretical neuroscience all the way into machine learning models that are implemented for production code.

06:29 So, it was an incredible learning environment, both, I mean, from software engineering perspective and about machine learning algorithms.

06:38 But I've since moved over to Vicarious, and I've been here for about eight months now.

06:44 Okay, nice. So, what kind of work do you do now?

06:46 At Vicarious, we're working to build general artificial intelligence. So, that's AGI. Any problem a human can do, we want our AI to do. And in the past, we've pointed to Rosie, the robot from the Jetsons. But I can say that we're not building exactly Rosie, per se.

07:07 More like R2-D2?

07:09 Oh, sure.

07:11 No, okay. That sounds really interesting. And, you know, when you talk about AI, there's usually some kind of split around specialized AI or general AI, right?

07:23 Like, a specialized AI might be Siri or a self-driving car or something, like, or the thing that solves Go. It's, like, incredibly good at driving the car, but there's no way that it could tell you, you know, what's the weather.

07:37 Or, like, something that's unrelated to what has been taught, right?

07:40 Exactly.

07:41 Then you've got the more general one that you're working on.

07:43 Yeah, yeah.

07:44 There's been a lot of press lately about deep learning models and frameworks, and they've been incredibly successful and useful for some of these narrow AI problems.

07:53 A lot of times in research development of these models and algorithms, we'll use video game environments, specifically the Atari environments.

08:03 Just kind of good test beds for the approaches.

08:07 And you take, for example, a breakout or a space invaders environment, and deep learning can blow away any human player.

08:19 It's solved.

08:20 But you add some minor tweaks to the game, like change the pixel values ever so slightly or just move some blocks around.

08:31 And things that are just imperceptible to a human, they pick that up easily.

08:36 The same deep learning model fails.

08:39 So that is kind of exemplifying the narrow component of those models.

08:45 Yeah, that's pretty interesting.

08:47 So one of the challenges with neural networks and deep learning, is that basically like taking multiple neural networks and sort of treating them as a group of things that then answer questions?

09:03 Or is there more to it than that?

09:05 In a way.

09:06 Let me dial back real quick.

09:07 Yeah, okay.

09:08 Artificial neural networks are a model that dates back to...

09:13 Yeah, at least the 90s for sure.

09:15 Yeah.

09:15 And even earlier than that, I feel like I'm offending some people out there by saying...

09:19 I did that in small talk in 1974.

09:22 What are you talking about?

09:22 Exactly.

09:23 But nowadays, in the past five years and so, we've seen that we can stack layers and layers and layers of these neural networks and make them very deep.

09:32 That's where the name deep comes from because we have the computation power to do so.

09:37 So these deep neural networks are essentially function approximators.

09:42 And you can feed them a ton of data.

09:44 It could be videos from your self-driving car.

09:48 It could be text in English and French if you want to do machine translation.

09:54 And these deep function approximators are able to learn from all that data how to take your input as, for example, English and give you the output in French.

10:04 Right.

10:05 Yeah, that's really amazing.

10:07 I feel like AI and machine learning was one of those things like nuclear fusion.

10:14 It was always 30 years out no matter when you ask somebody.

10:17 It's like in the 90s, it was like, oh, yeah, we're going to have AI.

10:21 And here, we're working on this in Lisp.

10:23 And it doesn't really work.

10:24 And then 10 years later, it's like we're still doing the same basic thing.

10:27 It's not really making any progress.

10:29 But I feel like kind of like you said, like the last five years, especially the last three, four years, it's just, wait a minute.

10:35 This is actually a thing now.

10:36 This is like really today there's reasonable uses of AI in tons of ways.

10:42 I mean, we have like Star Trek coming to life with Microsoft Skype and PowerPoint, right, where it like in real time translates to other languages as you speak, right?

10:54 And just things like that.

10:55 I mean, amazing.

10:56 Yeah, there's some really powerful uses already deployed in applications.

11:00 But it's kind of funny.

11:02 The field of research for AI and specifically for general AI is almost always like the mountain climbing dilemma where you think, oh, the peak is just over the next pass.

11:14 And then you get over the next pass.

11:16 And it's like, oh, it's the next pass.

11:17 And then the next pass.

11:18 And the next pass.

11:19 Everybody told me about this mountain range on the other side of this one, right?

11:23 Yeah, exactly.

11:24 So we do have some pretty powerful AI nowadays.

11:28 But we keep pushing the limits and pushing the limits and defining this path to AGI as we go.

11:33 Yeah, yeah.

11:34 So one sequence maybe is you talked about we had the neural networks.

11:39 They could kind of do a thing.

11:40 We had the deep learning neural networks that are really good at stuff.

11:44 But you make minor changes and they're kind of entirely lost, right?

11:49 So it's on one hand, we've built these things that you can train and they do a thing.

11:53 But maybe they can't adjust so much.

11:56 So is one of the new areas of like, how do you evolve like a deep learning model or something like this?

12:02 Yeah, this is a big area of deep learning research is the ability to generalize across tasks.

12:08 So you would be able to take your example, AlphaGo, the deep learning model that has been defeating the Go champions across the world,

12:18 and try to generalize that to maybe just a slightly different game board.

12:23 You can use a larger game board, for example, or even generalize that to a different game entirely, like chess or something.

12:31 But here at Vicarious, we're taking a bit of a different approach.

12:37 We aren't using deep learning as much as other AI research shops around.

12:42 We specifically are taking a more structured model-based approach using probabilistic graphical models.

12:50 And we're finding that our models and algorithms are much more attuned to things like generalization and unsupervised learning,

12:58 which are some problems in the deep learning community.

13:01 Yeah, yeah.

13:01 That sounds so interesting to work on.

13:05 So maybe tell us, like, most of us listening know what it's like to write software, create some utility, write a web app, or whatever.

13:12 How does, like, working on AI research projects, how much is that the same or different than writing a web app?

13:19 Well, the workflow is pretty similar to a lot of software engineering companies out there.

13:28 At a high level, it's really just agile practices.

13:31 We iterate and move very quickly.

13:34 But the difference is these are through experiments, experimenting on our models and algorithms, not really through product releases.

13:42 Right.

13:42 You don't just plan it out and say, these are the features we want.

13:44 I'm just going to build them.

13:45 Because you don't necessarily know what the answer is, right?

13:48 Yeah, exactly.

13:49 So we don't have a Kanban board of tasks and picking up and everything that you discuss at Scrum or something.

13:56 It's a lot more loose than that.

13:59 And researchers here, a lot like developers in agile workflows, have a lot of freedom.

14:06 And we need that to come up with these short-term ideas and long-term ideas and flesh them out and experiment out them.

14:15 So when we're building our code bases and our experiments, well, I like to think of developing and software engineering code into kind of three classes.

14:25 There's research, prototype, and product code.

14:29 And we mostly dabble in the research and prototype code, where research code is you're quick and dirty.

14:37 You're moving fast through experiments.

14:38 Hackey code is okay.

14:40 Test coverage is usually discouraged because it slows you down.

14:44 And then prototype is a step up where we want well-designed, well-architected for data flow and interfaces.

14:52 We want unit and integration test coverage, style standards like pet bait, and documentation.

14:59 But still, the mentality is this is code that will someday be scrapped.

15:04 So you're not married to it.

15:06 It doesn't have to be beautiful 99% test coverage.

15:09 Yeah, that makes a lot of sense.

15:10 It seems like you just want to play with the code in the beginning.

15:14 And then once you have an idea, like, all right, now this looks like it's going to work.

15:17 Let's go try to, like, really build this out and see what this does.

15:20 Well, almost.

15:21 The workflow is a little bit different because it really starts with theoretical and mathematical foundations.

15:27 So when we get some sort of research idea, if it's defined as just a problem that you want to try to solve or some new constraints on a different experiment, we first go through, okay, what are the mathematical foundations of this?

15:43 And we derive things like message passing algorithms over factor graphs.

15:47 And then we take that math and abstract it into software.

15:52 And that's how we build our experiments.

15:54 And then we do simple little toy problems of these experiments to really inspect the models and algorithms.

16:03 And then we get into things like building well-designed software and everything.

16:08 Sure.

16:08 And do you guys use Python for a lot of this?

16:11 Yeah, absolutely.

16:12 Python is the main language that we use here.

16:15 And it's really becoming ubiquitous in machine learning and AI.

16:19 It's elegant with a really simple syntax.

16:23 We can iterate quickly.

16:25 And it's relatively easy for people to learn.

16:27 A lot of AI researchers come straight out of academia and just toiling with MATLAB for their PhD for five years.

16:36 So they need to learn how to actually program.

16:39 And Python is great for that.

16:41 And then on top of that, Python is really powerful.

16:44 I love the phrase batteries included.

16:47 There's so much that comes with the standard library and then tons of PyPy packages that you can throw on top of that.

16:55 And Python sometimes gets some flack for being a little slow.

17:01 But it's really easy to control a C++ backend and integrate with some C or C++ code.

17:08 Right.

17:08 So maybe you prototype it out and this part is not quite fast enough.

17:11 So you could say, well, let's write this in Cython or write in C++ and then bring it into our Python app or something like that.

17:18 Yeah, absolutely.

17:19 Of course, we profile and optimize algorithms first.

17:23 So if an algorithm is something like ON squared because we have nested for loops but can be refactored into something sublinear, we do that.

17:32 And then we do more profiling and see, okay, this is really a bottleneck.

17:36 Let's port it to Cython.

17:37 Yeah, sure.

17:38 So that's a really good point.

17:40 Even though maybe you could make it run 10 times faster or something in Cython, if you have an algorithm that has a super high complexity, it doesn't really matter, right?

17:51 If you're going to have to take more and more data, it's really the algorithm that's the problem.

17:55 And a lot of people have pointed out that working in Python, it's easier to conceptualize and evolve the algorithm.

18:03 And then if you have to, to optimize it further, right?

18:06 Oh, yeah.

18:06 Yeah.

18:07 It's pretty straightforward to have an idea and then say, okay, I'm going to implement this in some nested loops, but God, this is awful and ugly.

18:16 And I'm just going to put it to do and come back and vectorize this.

18:20 And then you come back and fix it.

18:22 And it's just magnitudes faster.

18:24 Nice.

18:25 So maybe tell us a little bit of some of the tools that you use, like Python is pretty obvious, but things like TensorFlow, Carreras, NumPy, SciPy, those types of things?

18:36 Yeah, yeah.

18:37 We use TensorFlow for some of our projects.

18:39 It's really becoming very popular in the AI and specifically deep learning community where distributed computing over tensor graphs is really valuable.

18:50 Yeah.

18:50 Maybe for people who are not familiar with TensorFlow, just give them a quick elevator pitch.

18:53 What is it?

18:54 Oh, sure.

18:54 So TensorFlow, it's offered in more than Python at this point, but the main API for it is Python.

19:02 And it abstracts away a lot of the C and C++ computations for parallelizing tensors, which are basically just symbols that are distributed over these computation graphs.

19:15 And they offer some really cool tools for visualizing these computation graphs as you're doing things like training your model distributed across clusters or running inference on these models too.

19:30 Nice.

19:30 Okay.

19:30 Yeah, perfect.

19:31 And what are the other ones that you use?

19:33 Well, just like any software company, we try to leverage external tools.

19:38 We're not trying to reinvent the wheel here.

19:39 So a lot of our vision utility functions, for example, are handled with toolkits like OpenCV, which is C++ with Python bindings.

19:49 So a lot of that is pretty fast.

19:51 And SciPy, like you mentioned, for operating on NumPy arrays like Gaussian filters and convolutions.

19:58 It sounds like there's a lot of math involved and some pretty interesting programming.

20:04 Do you have like cognitive science people doing almost like brain modeling work?

20:09 What other disciplines are trying to like work together to solve this problem there?

20:14 Yeah.

20:14 So there's definitely a lot of math involved.

20:16 And actually, when people come to me for advice and say, hey, I want to get into AI research.

20:22 And one piece of advice I give them is you need to have your first language be math and then whatever programming language come after.

20:30 Sure.

20:30 And so give us a sense of like how much math.

20:32 Like if I was, say, super good at differential calculus, but that's all I knew.

20:37 Is that enough?

20:38 Do I need to know like statistics and probability?

20:40 Do I need to know linear algebra, graph theory?

20:42 Like what, when you say math, is it a PhD or is it a minor?

20:46 Well, we do have a lot of PhDs in things that are just obscenely complicated and impressive.

20:56 And those are across computational neuroscience to theoretical math.

21:01 But I would say graph theory, like you mentioned, specifically modeling and running computations over probabilistic graphical models is very important.

21:11 And the foundations of which are in probability and linear algebra.

21:16 Okay.

21:17 So not super out of touch math, but you definitely need to be able to speak those things.

21:23 Oh, no, yeah.

21:24 Yeah, like a pretty good, not first language necessarily, but second language.

21:28 Sure.

21:30 Okay, sure.

21:31 This portion of Talk Pythonemy is brought to you by Linode.

21:35 Are you looking for bulletproof hosting that is fast, simple, and incredibly affordable?

21:39 Look past that bookstore and check out Linode at talkpython.fm/Linode.

21:44 That's L-I-N-O-D-E.

21:46 Plans start at just $5 a month for a dedicated server with a gig of RAM.

21:50 They have 10 data centers across the globe.

21:53 So no matter where you are, there's a data center near you.

21:55 Whether you want to run your Python web app, host a private Git server, or even a file server, you'll get native SSDs on all the machines, a 40 gigabit network, 24-7 friendly support, even on holidays, and a seven-day money-back guarantee.

22:10 Want a dedicated server for free for the next four months?

22:13 Use the coupon code python17 at talkpython.fm/Linode.

22:19 It seems like a really fun thing to do, right?

22:22 Like every day you come in and people have new ideas and you try them and you're kind of exploring new space, right?

22:29 Yeah, absolutely.

22:30 There's not really a defined path to AGI.

22:35 And we try to build it from like, what do you think an artificial agent is in the world?

22:42 It is sensory perception.

22:44 It's goal-directed behavior.

22:46 It's building a model from sensory motor interactions.

22:49 And from that, we try to derive the fundamental properties and constraints of intelligence.

22:57 And that leads to our research ideas and eventually experiments.

23:02 Yeah.

23:02 So you talk about like open CV and computer vision and sensory stuff.

23:08 How much of what you're working on do you hope exists in the real world?

23:13 And how much are you thinking like if we could make a digital thing that could just live, say, on devices, on the internet, and only interact digitally, right?

23:23 Not actually see the world, but, you know, interact with people, say, with voice or text or things like that.

23:27 Hmm.

23:27 Interesting.

23:28 So at Vicarious, we're specifically building AI towards robotics.

23:34 Any robot, any task, we build the brain for it.

23:39 That's our initiative here.

23:41 So a lot of it is in the physical world.

23:44 And me specifically, I'm focused on our main vision team here.

23:48 Nice.

23:48 And how much of this do you see as like a consumer sort of thing versus, say, like factories?

23:54 Are you focused on doing this for people or for companies or I guess that's the question?

24:00 Yeah.

24:00 So there's not so much I can really share about the specific robots and applications that we're building.

24:06 Don't share your secrets.

24:07 I don't want those.

24:08 But the idea is like we want to build general AI.

24:13 So all of the things.

24:15 Yeah.

24:15 That sounds really cool.

24:19 What are some of the other companies that are doing this type of work?

24:23 Like I know what maybe companies are using it, but like AI research is a pretty small group of people I would expect.

24:32 You're right.

24:33 There's a lot of companies out there using AI, practicing AI.

24:38 But for doing the nitty gritty fundamental research, the shops are namely OpenAI, DeepMind, which is out in the UK.

24:49 And they were acquired by Google some years back.

24:51 And us, Vicarious.

24:53 Okay.

24:54 Now that's a pretty small group.

24:55 There's also Google Brain.

24:57 Yeah.

24:58 Do you know what the relationship between like Google Brain versus DeepMind is?

25:02 They're not even the same continent, right?

25:03 No.

25:04 And I've heard some conflicting information from people on the Brain team and people on the DeepMind team.

25:10 But as far as I can tell, DeepMind is kind of a separate entity.

25:14 They don't interact too much on projects.

25:17 DeepMind is much more building towards general AI.

25:22 And Google Brain does AI research, but a lot of it is directed more towards the applications for Google.

25:30 Sure.

25:30 Makes sense.

25:31 Do you consider companies like Uber or, you know, Google Alphabets, Waymo, sort of self-driving car type things?

25:41 Would you put them into this bunch?

25:43 Or do you think of them more as like using AI?

25:46 I would say using AI.

25:48 So my understanding of a lot of these companies, specifically the self-driving car companies, is they're looking at what sort of deep learning models can I use for my vision problems, my LIDAR sensor problems, and my motion planning problems.

26:05 And they'll take existing models that are out there from toolkits such as TensorFlow or PyTorch or Keras even.

26:15 And then they'll tweak those for their specific problem.

26:19 So that's more of a hyperparameter tuning type work, not so much the real fundamental research.

26:26 Right.

26:26 Okay.

26:27 And a lot of these guys are open sourcing their tools like TensorFlow and whatnot.

26:32 And I think there was even like a pact about this.

26:36 Like Google and Microsoft and some other people, didn't they team up on like sharing their research and their work?

26:43 It's been like a year since I've heard about this.

26:47 So my details are flaky.

26:48 Sure.

26:49 Well, there's been some collaboration over sharing these research environments.

26:54 So like I mentioned, video games are often a testbed that we use a lot.

26:59 So Microsoft has developed Minecraft to be a testbed for AI development.

27:05 And OpenAI has released their gym for different AI agents.

27:12 And then also Universe, which is just a big collection of all these different games.

27:17 And I believe there was some collaboration or teamwork between the two of them.

27:21 Okay.

27:21 Cool.

27:21 Yeah.

27:22 Universe sounds, I heard about that a while ago.

27:24 That sounds really pretty wild.

27:26 And as an AI research company, do you guys find stuff like that useful?

27:32 Like, hey, I could take this AI and drop it into Counter-Strike or I could drop it into Pitfall and all these different places where it can interact.

27:41 Is this helpful?

27:41 Yeah.

27:42 Yeah.

27:42 Definitely helpful.

27:43 We haven't explored Universe so much here at Vicarious, but we also build some internal environments that are very similar.

27:52 Mainly, we use internal tools because then we can define an API that might give us some different information than some of those APIs exposed.

28:03 But it's definitely helpful to have a lot of testbeds and use cases out there.

28:08 Yeah.

28:09 Yeah.

28:09 It seems like if you were, say, a grad student writing a master's thesis, being able to just plug into different video games is like a super big boost.

28:17 But if you're a full-on mini-person AI research company, you need something that you can, like, exactly tweak the environment just so.

28:27 Like, I want to understand how the robot interacts with, like, the squishiness of an object, right?

28:32 Yeah.

28:32 Which you probably can't get in, like, Unreal Game or something, right?

28:35 Yeah.

28:36 Exactly.

28:36 Exactly.

28:37 And a lot of this open sourcing is specifically targeting those grad students in universities.

28:44 I see open sourcing these toolkits and also for the same reason why all these companies are publishing a lot of their papers as a recruiting tool.

28:53 Yeah.

28:54 Because a lot of AI researchers are very academic at heart.

28:58 We want to share our information and help each other.

29:02 So the more that these big companies are sharing and being open about what they're building, the more they can attract top down.

29:11 Yeah.

29:11 So companies like OpenAI and Google might have a better chance of getting somebody who's mostly interested in the research side of things than, say, Apple.

29:19 Yeah.

29:20 Exactly.

29:20 This has been a problem for Apple as of late.

29:23 Yeah.

29:23 I guess there's a couple questions around universities, like, you kind of touched on.

29:27 One is, do you know if there are, like, machine learning AI bachelors these days?

29:33 Hmm.

29:34 Or is there just computer science and you can focus on that?

29:36 You know, I don't know if you can major exactly in AI or machine learning, but there are definitely a lot of undergraduate courses that you can take.

29:47 And from what I've heard, these courses are becoming ridiculously popular.

29:52 I try to mentor a couple of students from, like, Stanford and CMU.

29:58 And I always tell them, like, oh, there's a probabilistic graphical models course.

30:03 You have to take that.

30:06 Yeah.

30:06 Yeah.

30:06 That's cool.

30:07 It seems like there'd be a lot more money in working at Google on TensorFlow or maybe with you guys or whatever, rather than, say, being a professor at a university.

30:18 Is there, like, a drain of, like, as professors sort of get their research going, do they get sucked out of academia?

30:26 Yes.

30:26 For a typical example.

30:29 I know it's a problem for data science.

30:30 I just don't know about machine learning.

30:32 It seems like there's a parallel, right?

30:33 Yeah.

30:33 Oh, there absolutely is.

30:34 Now, the main example of this is Uber going into Carnegie Mellon and grabbing, really, all of their computer vision researchers a few years back.

30:44 But I think you largely see more of a collaboration in sorts.

30:49 So sometimes top professors from universities will have appointments with some companies like OpenAI or Google, and they'll still work with their universities and their research labs on some of their projects, but at the same time work with the industry partner in a way.

31:11 And that also works as an incredible recruiting tool for these companies.

31:15 Yeah.

31:16 And it also seems like it's probably the best of both worlds, right?

31:20 You still get to be a professor.

31:21 You still get to do your research and teach people, but you also get some real-world experiments and experience.

31:26 You're not just, you know, in your lab poking around, right?

31:30 You get to see really what people are trying to do with it.

31:32 And there's a big difference between trying to just fill out grants and get money and resources through academia versus having some sort of private or public company.

31:46 And if you need just infinite AWS clusters, you can get it.

31:53 Yeah, like I need 100 GPUs in a cluster.

31:58 Can we do that?

31:58 Yeah, sure.

31:59 Here, you push this button.

32:00 Yeah, yeah.

32:01 It was kind of funny.

32:02 Someone did a blog post on like huge spike in AWS GPU demand the week leading up to the paper deadline for NIPS, which was like a few weeks ago.

32:14 It is impressive.

32:17 Yeah, we've got to finish our research.

32:18 We've got to get this going.

32:19 Yeah, it's...

32:20 Yeah, yeah.

32:20 Yeah, so you guys probably do a lot of cluster computation stuff like that.

32:25 Do you use GPUs and other specialized chips or anything?

32:29 For some of the models that can take advantage of the parallel compute on GPUs, like deep learning models, yeah, absolutely.

32:36 But a lot of our larger models are still used on AWS, but not necessarily GPUs.

32:43 Just straight EC2 clusters or something like that.

32:46 Yeah, yeah.

32:46 Yeah, all right.

32:47 Very cool.

32:48 So we talked about the math requirements, linear algebra, calculus, probability, things like that.

32:53 But if people, you know, they want to try this out and they want to get started, like how maybe they know Python, but nothing else.

33:01 Like how would they get started?

33:02 I would recommend getting inspired by a model or a problem, maybe a NIPS paper or something from ICML or even a blog post that shows some pretty cool results on like natural language processing.

33:16 And dive in that way.

33:17 Yeah.

33:18 Yeah.

33:18 Do you have some examples of like what a problem might look like?

33:21 Would they have those at Kaggle?

33:22 Yeah, absolutely.

33:23 That's exactly how I got inspired.

33:26 Going to Kaggle and there are a whole slate of machine learning problems that you can dive into.

33:32 And oftentimes those communities will discuss their approaches and share some of their methods.

33:38 So that is really good, almost a code first way to dive in.

33:43 But like I was saying earlier, to build up the math chops to really be able to do AI research, I recommend trying to reproduce paper results.

33:53 So you pick out a research paper and you go through it.

33:58 You try to understand the models and the algorithms, the assumptions, the experiments, and then you try to implement that from scratch.

34:06 It's not easy.

34:08 You're going to fail a lot.

34:10 But it's the best way to learn.

34:12 And fortunately, if you're doing this in Python, a lot of the Python community is very helpful.

34:18 Yeah.

34:18 So what are some of the tools?

34:19 Maybe the ones we talked about, TensorFlow, Carreras, things like that?

34:23 TensorFlow and Carreras, they have some great prepackaged models that you can take out of the box and even some that point specifically to the papers that they're from.

34:32 So you can get up and running pretty quickly.

34:35 Yeah.

34:35 Maybe a really simple way to get started might be to take somebody who's written a paper, done some project, but it's only in MATLAB, and say, let me try to reproduce this in Python.

34:44 Like, just even started from their code, right?

34:46 Yeah.

34:47 Yeah.

34:47 That's a good idea.

34:48 You kind of, like, take it, like, let me not actually try to solve the problem, but just take it technically through the steps of, like, creating it in Python and seeing how that goes.

34:57 And then you've kind of got that experience.

34:58 And then you can go and try to, like, actually solve it independently.

35:02 Yeah.

35:02 Ultimately, it comes down to just trying to build a link, a mental model of how the math is connected to the software implementation.

35:11 And if that's building the code and then trying to figure out how the algorithms and equations are defined there, that works best for you.

35:20 Go for it.

35:21 If it's more of a bottom-up approach, do that.

35:23 Yeah.

35:23 Yeah.

35:24 Cool.

35:25 So you talked about your dev workflow being very sort of loose, and you start from just let's play around and not worry too much about patterns and factoring our code and doing testing stuff.

35:37 What are some of the pain points that you might end up with from, like, that workflow?

35:42 Because we move fast and it's a lot of research and prototype code, we can build up some technical debt.

35:49 And this largely comes because people, researchers here, want to build the next thing, not put in another few days to refactor their experiment into beautiful abstractions with 99% test coverage.

36:03 And they're probably, they're researchers, right?

36:04 There's more mathematicians and scientists, not pro developers who really care about, like, this design pattern or something, right?

36:11 We have a mix, fortunately.

36:13 So there are some that are very academic researchers who just do some kind of ugly things in the code.

36:21 And fortunately, we have our code review, our pull request review process that can help teach them and be a tool for building better software engineers.

36:32 But then we also have a lot of people here that are very strong engineers and have built up their software chops coming from companies like Google or Facebook.

36:42 Yeah, of course.

36:43 And that's pretty serious development right there.

36:45 Yeah.

36:46 So do you use, like, special, like, any particular tools for, like, refactoring code or for profiling or things like that?

36:56 Yeah.

36:56 So profiling code is really useful around here because, I mean, the biggest mistake a program can do is trying to optimize our code without actually knowing where the pain points are.

37:08 Yeah.

37:08 I find that even harder in this type of computational stuff where it's sometimes not intuitive.

37:13 Yeah.

37:14 Yeah.

37:14 And Python comes prepackaged with some useful profilers.

37:19 I prefer to actually use Kernprof line profiler because some of the Python built-in profilers will really just show you, like, this function is your bottleneck.

37:31 But going line by line, I've found to be much more helpful.

37:34 Yeah, sure.

37:35 So, yeah, here's a 100-line function.

37:38 It's slow.

37:38 Like, great.

37:39 That helps me a lot.

37:39 Yeah.

37:40 Yeah, exactly.

37:41 And maybe, I guess also, if you had your code more factored, right, into little methods and things like that, breaking across even packages and things, maybe it's easier for it to say, well, this is slow.

37:53 Like, okay, well, I clearly see the function that is slow.

37:55 But if you're just playing around and experimenting, like, you maybe don't have it so fine-grained, right?

38:01 So it's even worse than normal, maybe.

38:03 Well, we try to keep our code well-designed for both conceptual level and practical level, where conceptual is like functions and objects are abstracted to follow the math.

38:15 Practical is more of running end-to-end experiments efficiently.

38:20 So conceptual is important because we want well-designed code so we can extend it naturally with future research.

38:29 So that might be where things like computational bottlenecks aren't really obvious.

38:33 But when you try to run these experiments end-to-end, that's when you see, like, oh, well, we need to go back and profile and optimize.

38:40 Yeah.

38:40 Yeah, that makes sense.

38:41 Yeah, so that kind of touches on, like, this tension between research and software priorities.

38:47 Yeah.

38:49 How do you keep those in balance?

38:50 Like, what are the trade-offs there?

38:52 Well, it's interesting because the research rewards, experiment results, and maybe sharing some cool figures and visualizations with the team are more near-term.

39:04 While the software rewards, like benefits of a clean, well-documented code base, those are farther off.

39:12 So it makes it difficult to prioritize good software like refactoring and cleaner abstractions.

39:18 I found it very helpful to try to communicate, overly communicate, actually, the value of high-quality software.

39:25 Like, maintainable code base makes onboarding new engineers much smoother, and they can get up and running and contributing much faster.

39:34 And then also, I try to reward, in a way, examples of good code.

39:40 An example recently was there was a good use of Dunder slots for fast attribute lookup in a PR I was reviewing the other day.

39:51 Yeah, so maybe tell people about Dunder slots.

39:53 Like, it's used in the right time, in the right place.

39:56 It can, like, dramatically change memory and performance, and it's interesting.

40:00 But we haven't talked about it much in the show, so what's Dunder slots?

40:03 Oh, sure.

40:04 So my understanding of it is, basically, when you have a class in Python, and you have attributes for that class, it automatically will create this Dunder dict object.

40:15 And if you don't need that representation for every attribute of your class, it could be a lot of overhead to do lookups in that Dunder dict.

40:26 So sometimes you can use Dunder slots, which avoids the creation of Dunder dict, and you can define your attribute lookups there.

40:36 That sounds right.

40:37 So there's, you can put in a class just anywhere, and just say Dunder slots equals, and you give an array, a list of basically the field names, the variable names.

40:48 And then those are the only variables it can have, right?

40:51 So in normal Python classes, you could dynamically add another field to it, or things like this.

40:56 And it uses that Dunder dict to manage that.

41:00 But every instance of the class has a separate dictionary with separate names of the keys and things like that.

41:06 And so the slots basically means there's none of that, none of that creation, allocation, assignment, all that kind of stuff.

41:13 Yeah, so it's really amazing.

41:14 All right, so somebody did this, you're like, oh, wow, look at this thing.

41:16 Yeah, so it was just like a quick shout out on our Slack channel, just saying, hey, this is awesome.

41:21 Prop to somebody, yeah, that's really cool.

41:24 Yeah, yeah.

41:25 This portion of Talk Pythonry is brought to you by us.

41:29 As many of you know, I have a growing set of courses to help you go from Python beginner to novice to Python expert.

41:35 And there are many more courses in the works.

41:37 So please consider Talk Python training for you and your team's training needs.

41:42 If you're just getting started, I've built a course to teach you Python the way professional developers learn by building applications.

41:48 Check out my Python jumpstart by building 10 apps at talkpython.fm/course.

41:53 Are you looking to start adding services to your app?

41:56 Try my brand new consuming HTTP services in Python.

42:00 You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML data formats.

42:05 Do you want to launch an online business?

42:06 Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs.

42:11 This 16-hour course will teach you everything you need to launch your web-based business with Python.

42:16 And finally, there's a couple of new course announcements coming really soon.

42:20 So if you don't already have an account, be sure to create one at training.talkpython.fm to get notified.

42:25 And for all of you who have bought my courses, thank you so much.

42:29 It really, really helps support the show.

42:31 So I looked at the Vicarious website to sort of check out what you guys are up to.

42:37 And I looked at the team page.

42:38 And there are a lot of people that work there.

42:39 Way more than I expected, actually.

42:41 How many people do you work with?

42:43 So we're at about 50 now.

42:46 And I started eight months ago, and I was number 30 or so.

42:51 We're growing a lot.

42:52 And we're looking to keep adding more engineers and researchers.

42:57 And we're expanding also to a big new office building and everything out here in San Francisco.

43:04 And there's a lot of us here.

43:07 And the really interesting thing is that there's a lot of us from very diverse technical fields.

43:14 So we have some people that are theoretical neuroscientists, some people that specialize in robotic grasping and manipulation and everything in between.

43:25 Sounds like a pretty cool space to be in.

43:27 Yeah, yeah.

43:28 It's fun.

43:28 Nice.

43:28 So you guys are working on this general artificial intelligence, which is maybe the most controversial kind, but also has the greatest possibility for helping society out, right?

43:41 Like, it's one thing to make a car that can drive itself.

43:44 It's another to make like a robot that can do all these different jobs, right?

43:49 And learn and so on.

43:51 So, you know, Elon Musk came out with his statement that we're living in a multiverse and he's scared of AI and things like that.

43:59 Like, how do you guys, do you guys talk about this kind of stuff?

44:02 Do you put any credence in it?

44:05 I'm not so sure about the multiverse.

44:06 I'm pretty skeptical of this.

44:08 But the thing becoming too smart, is this something that you guys actually are seriously concerned about?

44:15 It's something that we seriously consider.

44:17 I wouldn't necessarily say concerned.

44:21 It's funny.

44:22 Whenever I meet random people and say what I do, I end up having this conversation a lot.

44:28 I bet you're like, oh my gosh, here we go again.

44:30 Yeah.

44:31 No, we're not living in a multiverse.

44:32 This is just like, damn off the...

44:35 Well, AI will do amazing things for society, like transforming healthcare from reactive medicine to something that's predictive.

44:45 And essentially eliminating motor vehicle accidents.

44:50 I dream of the day I can ride my bike around San Francisco and not worry about someone slamming into me.

44:56 Yeah.

44:56 Yeah, absolutely.

44:57 But on the spectrum of fear and optimism, where the former are guys like Nick Bostrom, who wrote that super intelligence book.

45:06 Yeah.

45:07 Elon Musk and the latter, where the optimists are those of like the futurists, Ray Kurzweil.

45:13 I would say most AI researchers, and myself included, fall somewhere in the middle, leaning towards optimism.

45:22 So I would say, as with previous technological revolutions, there's definitely concern over the big societal changes, and rightfully so.

45:32 Jobs will be lost to automation and industries transformed.

45:36 But it's much easier to look at the existing jobs as examples, painting the picture of, oh, wow, AI is going to replace all of these, we're going to lose all of these jobs.

45:45 Than it is to imagine a future where there are new industries that are yet to be created and all the jobs to find.

45:53 Every change in human history has not wiped it out and people just went and sat in the corner, right?

46:00 Like the industrial revolution, the move away from farming, the move to knowledge workers, like all of these things have been done.

46:07 Exactly, exactly.

46:08 We will go on.

46:10 But I would say the real concern is the speed with which this transformation will take place is significantly faster than the agricultural and industrial revolutions.

46:20 Fortunately, companies are taking this into real consideration.

46:24 Elon Musk's OpenAI is one.

46:27 Here at Vicarious, we have a group dedicated to AI ethics.

46:31 Oh, wow.

46:31 Okay, that's pretty interesting.

46:32 Yeah, I definitely think the change will go quicker.

46:36 You know, like the change for the industrial revolution, things like that, they banned people's working careers.

46:43 So it's not like next year the thing that you did was completely gone and it was totally fine the year before, right?

46:52 And so I think there's going to, I think you're right, there's probably going to be some kind of step where there's like a serious shakeup.

47:00 One example of this that comes to mind is the U.S.

47:03 The number one job title, job role for men is driving of some sort, like trucks and things.

47:12 You know, self-driving cars, self-driving trucks and semis and deliveries could put a serious dent in, you know, guys' jobs.

47:21 And if that happened too quickly, I can see like some serious social unrest because if you have a large young group of men who are not employed and have no positive outlook, that could really turn badly, right?

47:35 Yeah, it could.

47:36 And you see some early examples of this with companies like Uber and Lyft who aren't necessarily replacing drivers, but they're transforming an industry in a way where taxi drivers are the source of this unrest because now they've opened up the workforce to this kind of car sharing or ride sharing.

47:59 Yeah, I would say it's similar to what the Internet did for knowledge work, right?

48:04 Like it used to be you would go to your office and you would compete with the people in your city for those jobs.

48:10 And then all of a sudden the Internet came along and open sourcing and globalization and everything.

48:14 You were all of a sudden competing with everybody, not just the people in your town.

48:17 And it's a little like that.

48:20 Like how many people are going to actually go and become taxi drivers versus like, hey, I have a car.

48:24 I have a smartphone.

48:25 I got two hours and need some money.

48:27 Let's just go do it, right?

48:28 It really opened up the space of potential people in the marketplace.

48:33 Yeah.

48:33 I agree that there's we'll probably get through it pretty smoothly and there's there's very likely going to be something amazing on the other side.

48:41 Let's think maybe one more question on this on this theme for you because I know you I know you have these questions a lot.

48:48 But what do you think the biggest change that AI is going to bring that people will like obviously notice?

48:56 Right.

48:56 Not something super subtle that makes a change that they don't necessarily see.

48:59 But like what are people going to go?

49:01 Wow.

49:01 Like here's what AI has brought us in 10 years or 20 years.

49:04 Well, it's kind of like the mountain climbing dilemma that we were talking about earlier where new AI technologies roll out, but it's very incremental in a way.

49:16 And you don't necessarily notice that Facebook's algorithm for identifying whose faces are whose has improved a lot.

49:24 But it did.

49:25 And that was the outcome of some really impressive AI research.

49:29 But something that I think I hope people will look at and just be completely floored by is how AI is going to transform health care and medicine.

49:40 Yeah, that's one of the things I definitely think as well is diagnosis basically.

49:44 Yeah.

49:44 Yeah.

49:45 Like I had mentioned earlier, health care has always been very reactive.

49:48 Someone is sick and has some sort of symptoms and we guess at what is going on.

49:55 And that is all suspect to what kind of quality of health care you can see.

50:01 And fortunately, we live in a country where we have a lot of good health care doctors.

50:06 But in third world countries, this absolutely is not the case.

50:09 AI I see as more less diagnostic, more preventative in a way.

50:15 So you know that you're getting sick months before you actually show any symptoms.

50:20 And the treatment is minor instead of catastrophic.

50:22 Yeah.

50:22 Yeah.

50:23 Interesting.

50:24 So one of the rumors, you know, there's not an iPhone 8 yet, but one of the rumors around the new iPhone coming out is that Apple is working on a dedicated AI chip that goes in the iPhone.

50:35 What do you think of these types of developments?

50:37 This is really interesting because we've been hearing for years a lot about this neuromorphic chip development, which was basically companies like Intel and ARM or whatever doing some research towards, okay, what is the next platform for computing?

50:54 And recently we've seen or heard about Apple's new chip.

50:58 And Google also has their TPU, which is a Tensor Processing Unit.

51:03 And these are devices dedicated to running specifically their algorithms on devices, I guess, on iPhone and iPad.

51:12 And I don't know too much about the hardware of these devices, but they would be optimized for computing inference on deep learning models and such.

51:23 Yeah.

51:24 I think it really opens up the possibility for something interesting, but I have no idea what it's going to be.

51:28 We'll find out in a few years, I guess.

51:30 Yeah.

51:31 The exciting thing is a lot of the work in AI is no one knows what it's going to be.

51:37 Yeah.

51:37 That keeps it fun.

51:38 All right, Alex.

51:39 Maybe we should leave it there for the AI topic.

51:41 I really appreciate you sharing what you guys are up to and your thoughts on AI research.

51:46 Yeah.

51:46 I'm glad you guys are interested.

51:47 Yeah, it's cool.

51:48 All right.

51:49 So now it's time for the final two questions.

51:51 So when you write some of your deep learning Pythonic code, what editor do you open up?

51:57 Oh, I definitely go for Sublime.

52:00 There are some add-ons like GitGutter I like a lot and PyFlakes Linter.

52:07 And then for C++, I'll use CLion.

52:09 Yeah.

52:10 Okay, cool.

52:10 Yeah, definitely the plugins for Sublime are cool.

52:12 And I played a little bit with CLion, but I just don't have enough C++ code to write.

52:16 So it looks neat, but I haven't done anything with it.

52:19 And notable PyPI package.

52:22 Oh, there's so many.

52:23 The one I mentioned earlier, KernProf Line Profiler, has saved my neck many times.

52:29 And then also, I like working from the command line a lot.

52:34 A lot of people in research will use Jupyter Notebooks for the interactive visualizations.

52:40 But working from the command line, I like PTPython.

52:43 Oh, yeah.

52:44 I love PTPython.

52:45 That's great.

52:46 Yeah, I use that as well.

52:47 Yeah, the tab complete is so clutch.

52:49 But then also, arg parse, I love a lot because I can just throw an experiment in a script and

52:55 then have some parameters that I define at the command line.

52:58 Yeah.

52:58 All right.

52:59 Awesome.

52:59 Those are three good recommendations.

53:00 So final call to action?

53:02 Well, if you're trying to get into AI research, like we were discussing earlier, you can definitely

53:06 check out some of these toolkits that have prepackaged models like Keras and TensorFlow and PyTorch.

53:12 But you should try to implement, for example, a convolutional neural net in numpy code.

53:18 That's how you really learn what's going on.

53:20 All right.

53:20 So let's start from the ground up and figure it out.

53:23 Then use the tools, huh?

53:24 Yeah.

53:24 Yeah.

53:25 All right.

53:25 Excellent.

53:26 So thanks so much for being on the show and sharing your story with everyone.

53:29 Appreciate it.

53:29 Yeah.

53:30 Thank you for having me.

53:30 You bet.

53:31 Bye.

53:31 This has been another episode of Talk Python to Me.

53:35 Today's guest has been Alex Lavin, and this episode has been brought to you by Linode and

53:41 Talk Python Training.

53:42 Linode is bulletproof hosting for whatever you're building with Python.

53:46 Get your four months free at talkpython.fm/Linode.

53:51 Just use the code Python17.

53:53 Are you or a colleague trying to learn Python?

53:56 Have you tried books and videos that just left you bored by covering topics point by point?

54:01 Well, check out my online course, Python Jumpstart, by building 10 apps at talkpython.fm/course

54:07 to experience a more engaging way to learn Python.

54:09 And if you're looking for something a little more advanced, try my Write Pythonic Code course

54:14 at talkpython.fm/pythonic.

54:16 Be sure to subscribe to the show.

54:19 Open your favorite podcatcher and search for Python.

54:21 We should be right at the top.

54:23 You can also find the iTunes feed at /itunes.

54:26 Google Play feed at /play and direct RSS feed at /rss on talkpython.fm.

54:32 Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.

54:37 Corey just recently started selling his tracks on iTunes, so I recommend you check it out at

54:42 talkpython.fm/music.

54:44 You can browse his tracks he has for sale on iTunes and listen to the full length version

54:48 of the theme song.

54:49 This is your host, Michael Kennedy.

54:51 Thanks so much for listening.

54:52 I really appreciate it.

54:54 Smix, let's get out of here.

54:56 Stay tuned.