#232: Become a robot developer with Python Transcript
00:00 Michael Kennedy: When you think about the types of jobs you can get as a Python developer, you probably weigh the differences between data science and web development, but did you consider programming robots in Python and not just toys, but serious productive machines? It turns out there's a gap in the industry where we could use more Python developers in robotics. That's why I'm happy to have Ricardo Tellez here to give us an overview of the software development side of robotics programming with Python. This is Talk Python To Me, Episode 232, recorded September 11th, 2019. Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem and the personalities. This is your host, Michael Kennedy, follow me on Twitter where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via @TalkPython. This episode is brought to buy Linode and Tidelift. Please check out what they're offering during their segments. It really help support the show. Ricardo, welcome to Talk Python To Me.
01:09 Ricardo Tellez: Thank you very much, it's my pleasure.
01:11 Michael Kennedy: Yeah, it's great to have you here. It has been way too long since we've talked about robotics in Python on the podcast. We were just talking and I think it's been at least four years since the last time we've talked about this, and the world...
01:25 Ricardo Tellez: Wow!
01:27 Michael Kennedy: Is, of course, changing so much in technology and Python and robotics as well. So I'm really excited to have you on the show to talk about that.
01:34 Ricardo Tellez: Yeah, it's my pleasure, I think that is once the time to start to continue talking about robotics in Python, yeah.
01:40 Michael Kennedy: Absolutely, absolutely. Now before we get to that, let's start with your personal story. How did you get into programming in Python?
01:46 Ricardo Tellez: Well, I started programming in Python when I was doing my PhD, but there was just kind of, only for making this small experiment work. So it was actually...
01:57 Michael Kennedy: Sure.
01:57 Ricardo Tellez: There was no program in Python, just copy-pasting and...
02:00 Michael Kennedy: What was your PhD in?
02:02 Ricardo Tellez: It's in Artificial Intelligence. I was using neural networks to move robots, to make them learn how to move and so on. That was just the beginning, but then after that then I started working to a company that is called PAL Robotics. We were building their humanoid robot, and then there is where I started to program more things in Python, especially when we started to use ROS, the Robot Operating System. So at that point in time because ROS provides to you this interface with Python and it was quite useful, so that's when I really started to program in Python.
02:40 Michael Kennedy: Sure, what were you programming before you started using Python significantly?
02:43 Ricardo Tellez: In C++.
02:44 Michael Kennedy: C++, yeah.
02:45 Ricardo Tellez: Basically C++, yeah.
02:45 Michael Kennedy: Uh-huh.
02:46 Ricardo Tellez: It's for the company in robotics, C++ is very important because usually you need to have very fast products. So, I mean, your robot has to deliver very fast, say, for example, let's say, face recognition, the speech recognition, control of the movement, so that requires as much as possible, and for that then C++.
03:09 Michael Kennedy: Yeah, absolutely. You think about the performance of Python, I think it's really interesting story because pure Python, the performance is not amazing, but a lot of times what you're doing with Python is either you're talking to some package that has a lot of the behaviors in C or C++ or you're talking to other systems that themselves are running in C or some other language like that, like a database or whatever, right. So I do think it's really interesting. Do you see people mixing Python and C++ or even things like Cython?
03:43 Ricardo Tellez: In robotics?
03:43 Michael Kennedy: Yeah, yeah, in robotics.
03:45 Ricardo Tellez: Definitely, especially if you're using ROS. So with ROS that is perfectly created, this integration. You can have this little nodes, for example, that like a kind of behavior changing node or changing the behavior of the robot based on some pattern recognition or something. So the pattern recognition system will run in C++ because it needs to move very fast, but this behavior system doesn't need to be to go to the millisecond. It could be half a second in order to change the behavior, so that could be running in Python and then it simplifies a lot, your work and your maintenance of the code.
04:27 Michael Kennedy: Absolutely, and even controlling stuff probably, right, like if I'm going to orchestrate, hey, C++ layer, get this image, tell me whether you recognize a thing and then let me decide in Python what to do and kind of orchestrate the lower levels. I think that might also be common, yeah.
04:44 Ricardo Tellez: Exactly, that's exactly the case. That what you mention is exactly the case. And by using ROS, you don't actually care what one program is written in another, they can't tell transparently, so they don't know if the other one is written, and they interchange the information transparently.
05:03 Michael Kennedy: Yeah, that sounds really cool. We're going to dig into more of that for sure, but let's just really quickly tell folks what you do day-to-day so they know your perspective where you're coming from, your experience?
05:13 Ricardo Tellez: Okay, so I was working on that company, PAL Robotics, building the humanoid robots, but then I decided to build my own startup. That is about developing programs for robots based on ROS. So we have to platform online where you can develop those programs, create your programs in ROS, either C++ or Python, and then you test on the same platform by using simulations of the actual real robot. So you got already provided this, so the integration is very, very simple. You don't need to install anything in your computer and you operate directly with the web browser, you create the programs there with the environment that we provide and you test it on the simulation. And once you have this running, then you can switch the code into, you can deploy the code into the real robot. Actually you can test, you can switch your code running from the simulation and then now let's click and switch to the real robot until everything works perfect.
06:14 Michael Kennedy: That sounds like a really fun job actually to be working on. What's the name of the company? How do people find to this place?
06:21 Ricardo Tellez: Okay, our company is called The Construct. Like The Construct is, so many people ask us, why is this name? So basically The Construct, we wanted to build this place where robots can learn because there is a simulated world where they can play and they can try and so on. And then that's a Construct concept that is in the movie of The Matrix. You know The Matrix when they get in? The world is built by the Construct and actually our logo is one of the sofas that, of the chairs where Morpheus is explaining to Neo that, hey, that's the Construct and so on.
07:03 Michael Kennedy: The famous red pill, blue pill scene.
07:05 Ricardo Tellez: Yeah, that chair.
07:07 Michael Kennedy: Oh, cool, yeah, I see it now and it's theconstructsim.com, I'll make sure to link to it in the show notes for people. That's really cool. Yeah, so this sounds like a really fun project to be working on, and I think this idea of having a simulation definitely has to democratize robotics, right. It's one thing to be able to write a little code, to look at an image and do something, it's a whole another to put a humanoid creature together and do that.
07:33 Ricardo Tellez: Yes, exactly. And that's a problem for the people who want to be a developer. Most of the audience of your podcast are developers, they are people that they're interested in developing programs, but what I found is that they usually are not interested in hardware issues usually. That's why they decided to become software developers in the first place, yeah. So that makes perfect sense, but there is a lack, there is a huge lack of developers in robotics. So a person, it's just us, the roboticists that we are creating, the programs, but, I mean, the quality of that, I think it could be hugely improved if we have the help of real developers. And then the problem is this hardware thing, you know. And they're interacted. So with the simulations, we can provide this environment where the developers don't have to deal with the hardware, they test the programs, the simulation in the same way as developing applications for an iPhone, you can use those simulators.
08:40 Michael Kennedy: So you said I could go to my web browser and actually code up a simulation there. Is that in Python or is that in C++? And what's the experience like?
08:49 Ricardo Tellez: Okay, so you have a typical IDE, so an Integrated Development Environment, and typical that you have autocomplete and so on. And then you can program, so you create the control programs. The programs are made in robot and they send the sentences or decide which direction to move or actually move the robot. Everything that could be written in Python or C++, but then there is the simulation itself, the simulation itself. In case that you don't want to use the ones that we provide, you can create your own, of course. We use a program, an engine for simulation that is called Gazebo, that is open-source. So anybody can download it also, and then you have to create a simulation for that engine, and that is created by using format that is called URDF. That is basically an XML file.
09:41 Michael Kennedy: Cool, so you can create your own little world. What kind of simulations do you provide? Like what kind of robots, could I do self-driving cars, humanoid things, delivery robots?
09:50 Ricardo Tellez: You have all types of robots, we have even humanoid robot, human size humanoid robots with including their controllers for walking, so you don't have to make that robot walk because it's already provided with all the controllers. You have drones, you have cars, in order to make self-driving you have to build the program, okay. And then we have what is called mobile manipulators. The mobile manipulators are robots that are intended to be in the warehouses, for example. And let's imagine an Amazon warehouse and then this robot has to be able to move on that warehouse to go to the shelf, that is where is the product and then grab the product. So that's a mobile because it moves, a manipulator because it can manipulate the object.
10:43 Michael Kennedy: Right, it's got an arm or something, yeah.
10:45 Ricardo Tellez: They do exist, the robot, but the problem is the software. Again, so the problem is the software. So the hardware is there, you can buy this kind of robots, but they are not able to grab the thing. And it's a problem of software.
11:01 Michael Kennedy: Yeah, well, it sounds like you're solving a little interesting problem. Now before we get into the topic more broadly, I just want to talk about really quickly, you also have a podcast that people might be familiar with called the ROS Developers Podcast. And I'll link to that as well. Just tell people who maybe want to listen to a whole bunch of stuff about ROS as well after the show about your podcast.
11:23 Ricardo Tellez: Yes, yes, definitely. So in this podcast what we do is, we interview people that are experts in ROS, and especially about how to program robots with ROS. And then they explain to us how you can do this, how they have gone to create, make a robot create a map, so they teach which packages they have used, where to download the code already there, and we try to have fun also, yeah.
11:49 Michael Kennedy: And it sounds like a good show, I'm enjoyed listening to an episode or two. Now let's just talk little broadly, when I think about the Python developer space at least, maybe I have this wrong, but I see it broken up into a couple of areas. I see there's a big chunk of web developers writing Flask, Django, Pyramid apps, talking to databases, worried about scalability, uptime, Docker, etc. There's another significant group doing data science with NumPy, Pandas, Jupyter, Lab, etc. and they're doing their thing. And then I see the world is kind of like other, and that other is super, super-diverse, right. It could be physicist, it could be biologist, it could be financial folk, right, DevOp. There's just so many things. I guess one of the areas that seems really interesting is this area of becoming a robotics developer. So is this like a growing area that people might look into? Is this a popular area for Python? It sounds like it is. What's the story around there?
12:56 Ricardo Tellez: Yes, definitely. So I think that becoming a robotics developer is now the moment to start thinking about it. It's I think of robotics developers like when we were thinking about data scientists, some years ago, let's say 10 years ago.
13:13 Michael Kennedy: Yeah, like 2011, 2012, 2013 when they were all sort of switching to into Python and more programming oriented stuff rather than say R, or something that's more statistical based, right.
13:24 Ricardo Tellez: Yes, or deep learning also. In deep learning 10 years ago they were just a few of them, and now it's completely full, it's filled, sorry. It's a completely filled of people. Everybody is doing data scientists. And then now the situation for becoming a robotics developer is a very good one because as I mentioned there are just a few people and also mainly we are roboticists, mainly of the people is just roboticists or very hardware oriented people, and then if you are the developer, you have a chance there because you know the skills better than anyone, and that is programming.
14:03 Michael Kennedy: And the tools sound like they're tools that people would maybe already be familiar with or very similar to stuff they've already done and is probably not a huge field. Millions and millions of people building robots necessarily in terms of developers, but it's probably good jobs, interesting jobs, you probably going to move around and do cool stuff and not just sit in a cubicle, right, and work on optimizing marketing conversions or whatever.
14:29 Ricardo Tellez: The thing is that this robotics market is exploding and actually I have here some forecasting for 2024, for example, is now at present specifically only for development with ROS. It's the market is about 312 million, 312 million, but it is expected in 2024 to be about 500 millions, the market. So it's almost doubling.
14:59 Michael Kennedy: Yeah, yeah, almost doubling, yeah.
15:00 Ricardo Tellez: Yeah, and then I can tell you, for example, that recently Amazon has entered last year. Amazon entered into the game and they have the Robotics division, and BDR also entered into the game some years ago, and Google is going to enter this year.
15:20 Michael Kennedy: Oh, wow!
15:21 Ricardo Tellez: And Microsoft also, Microsoft has entered last year also presenting their own ROS version in Windows. So very, very big players are getting into there and you know that big players, they don't go there just because they're bored, is because they are feeling that there is something there to get.
15:40 Michael Kennedy: Right, they want to ride the wave.
15:42 Ricardo Tellez: Yes. That's why think it's a very good possibility. Also there is a very famous conference, it's called the ROSCon, is the ROS Conference official. And actually it's in two months is going to be the next edition, and there all the companies at the show, we are hiring, we are hiring, we are hiring, we are hiring. There is a demand for software developers in robotic.
16:07 Michael Kennedy: This portion of Talk Python To Me is brought to you by Linode. Are you looking for hosting that's fast, simple and incredibly affordable? Well, look past that bookstore and check out Linode at talkpython.fm/linode. That's L-I-N-O-D-E. Plan start at just $5 a month for a dedicated server with a gig of RAM. They have 10 data centers across the globe, so no matter where you are or where your users are, there's a data center for you. Whether you want to run a Python web app, host a private Git server or just a file server, you get native SSDs on all the machines and newly upgraded 200 gigabit network 24/7, friendly support even on holidays and a seven-day money-back guarantee. Need a little help with your infrastructure, they even offer professional services to help you with architecture, migrations and more. Do you want a dedicated server for free for the next four months? Just visit talkpython.fm/linode. I think a lot of people grew up dreaming of working with robots and all these things, and then you realize that it's hard, the hardware is tricky, and maybe the jobs aren't there, but it sounds like maybe that's becoming a reality. That's cool.
17:16 Ricardo Tellez: Yes, definitely. And the main problem that they can face maybe is about dealing with the hardware, but as I mentioned before, then dealing with hardware can be kind of substitute by using the simulation. So you can replace a lot of work with hardware by using the simulation and then also by using ROS API. That's the combination of the both.
17:40 Michael Kennedy: Yeah, so tell us a little bit more about that. I mean, your company has this virtual world sort of integrated online space for doing these. Does ROS itself have capabilities for running in a simulation mode or something like that?
17:54 Ricardo Tellez: Yes, together with ROS when you install, ROS is open-source, right. So anybody listeners there, you can download right now if you want. And together with ROS, you are installing the official simulator, that is called Gazebo, like a gazebo. And then this allows you to simulate your robot, actually there are a lot of simulations already there, and then you can download also in your computer and execute. Then having said that, good luck, okay, because what, also you download, you know what happened, you download, and then it shows these effect that I call, it works on my machine, you know.
18:35 Michael Kennedy: Yeah. Have you seen the certification program? There's in a official, It Works on My Machine.
18:41 Ricardo Tellez: Okay, there is a certification program, no.
18:45 Michael Kennedy: It's really hilarious, I'll definitely put a link in the show. So there's somebody made a joke about this at some point and did a blog post. And so, let's see, to officially become certified is, you compile your application assuming that's a compiled language.
18:59 Ricardo Tellez: Uh-huh.
19:01 Michael Kennedy: Getting the latest from recent code changes out of source control is totally optional. You launch the application website, you cause one code path to be executed. The preferred way to do this is with ad hoc testing. Omitting this step is fine, if fewer than five lines have changed and then you check in your code, and now you are certified, you get the badge.
19:22 Ricardo Tellez: You're certified.
19:24 Michael Kennedy: Isn't that great? You get the simulation setup on your machine and then it's tricky to deploy this or share it. Is that what you're saying?
19:32 Ricardo Tellez: Yes, it is tricky because there are a lot of versions, the versions are changing constantly, and also there are a lot of requirements from other libraries that they are changing also again, so you get this effect that multiplies. And, of course, I'm not telling that developers, the listeners of the podcast, they cannot do it. No, I'm not telling this, okay, I'm talking about efficiency. So many of the people we talk to about our product, they say, no, why would I pay you if I can do it? And it's completely fair because they can do it, but most of our clients they can also do it. It's not that they cannot, but they don't want to. That's the...
20:13 Michael Kennedy: Yeah, they want to focus on the core problem they're trying to solve and not all the stuff around it, right.
20:19 Ricardo Tellez: Exactly.
20:19 Michael Kennedy: Okay, cool.
20:20 Ricardo Tellez: There's another point here is about working into teams so, or, for example, that our system is very useful and used actually by, for example, by universities because the teacher can create the demonstration simulation with a code that makes the robot grasp something, and then you can share with all the students, 200, 300, 1000, whatever. It will work in the exact same way as for the teacher, so the teacher knows that this is going to work for this guy, and they don't have to prepare anything.
20:56 Michael Kennedy: Yeah, it's not like, well, you got an old version of Mac OS and it has the wrong dependency here, so it won't work or you don't have the right permissions or all that. I mean, it sounds a little bit like the problem that Jupyter Notebook solve for teaching data science and other types of courses where you can just say, here's a URL, you just go use this, and it's all setup just the way we needed to be, all right.
21:19 Ricardo Tellez: Exactly, actually in our product we use Jupyter Notebooks for the documentation and demonstration. I will send you one example, one we call ROSject, ROSject is a ROS project, which contains everything. It contains the simulation, the code, the documentation and dataset, and so it's a simple link, and then I will send you one example to you, so you can have a look and maybe share with the audience.
21:44 Michael Kennedy: Yeah, yeah, put it and share that, huh.
21:44 Ricardo Tellez: And that you'll see that it opens a Jupyter Notebook and then it says the Jupyter Notebook. Oh, do this, and then the robot will do that, and then you can see that it actually work.
21:54 Michael Kennedy: That sounds really fun. Yeah, definitely share that with everyone, that will be fun to play with. Now let's talk a little bit about the different types of robots. So I have a robot in my house and it looks about the size of a plate, and it drives around and it gets stuck in my furniture. When it's not stuck in my furniture, it's trying to vacuum up stuff around the house. So we have vacuums and actually those are pretty cool, even some of those like Roomba and stuff, there's some that are programmable, right.
22:20 Ricardo Tellez: Many things to say about it. You can stop me anytime, okay, because I have a lot of a lot of things to say. So first thing I'd like to say is vacuums is, I'd say that is the only real robotics business that has been successful. I think, I don't know the numbers, okay, of all the robotics companies, but I mean in the case of service robot.
22:46 Michael Kennedy: Sure.
22:46 Ricardo Tellez: So the vacuum cleaner is called a service robot.
22:49 Michael Kennedy: Consumer level robots, right, yeah.
22:51 Ricardo Tellez: Exactly, so those are robots that are working, they're doing their work in environments where there is people around. They have to figure out how to manage changes in the environment, not industrial robot, okay, I'm not talking about those, but about service robot, I think that the vacuum cleaners, the iRobot or any other brand, those are the only ones robotic companies that have been successful and made a business. The rest of the ones they are leaning on investment, and rounds of VCs and so on.
23:22 Michael Kennedy: Yeah, it's all future valuation and VC money and hope.
23:26 Ricardo Tellez: Yeah, and hope, exactly. So that's the first thing I wanted to say about the vacuum cleaners because they deserve all the respect.
23:34 Michael Kennedy: They actually are a success story, right, in a sense.
23:37 Ricardo Tellez: Exactly, a success story. And then the other thing to mention about programming, the version of iRobot is called Roomba. There is a version, I don't remember exactly its name, but it's programmable, so you can program it. Then some of the guys, guys of the developers of the ROS, some of the guy decided to use it as the base for creating a simple robot that all the ROS students they can use it and they can buy for low amount of time and then start performing task with ROS and so on. It's called the TurtleBot. The TurtleBot in the inside has a Roomba, a programmable Roomba, that is called... Oh yeah, I remember, it's called Kobuki, Kobuki.
24:23 Michael Kennedy: Yeah, Kobuki, okay, that's pretty awesome. They should just leave it vacuuming just so wherever the robot works, it's just super-clean. I had no idea they did that, that's really interesting. I knew you could program, I didn't know that it was being like repurposed for this, these other areas.
24:38 Ricardo Tellez: Yeah, yes.
24:41 Michael Kennedy: Another area where I think robotics, I don't know even, you tell me if this is the right word, but certainly machines operating autonomously in a world seems to have taken hold somewhat, maybe not mainstream commercial success, but kind of crazy success for how complicated it is is cars, right. Like Tesla even just the Teslas are unbelievable and how that they can interact with their environment and then you've got Cruise Automation, you've got Google, you've got Nvidia. Uber actually building truly autonomous cars, but even going for riding a Tesla on a city street is a very weird experience.
25:22 Ricardo Tellez: It's amazing, but it still haven't reached the Level Five of autonomy. It means that it can't go right now, yeah, not yet. Not yet, and it's going to be very difficult to reach that. They are expecting to release very, very soon the system, but personally I don't believe it that is going to be very soon because it's very, very complex. It's very complex here.
25:45 Michael Kennedy: The edge cases seems so hard, right. You get 95% perfect, but the consequence of failure...
25:51 Ricardo Tellez: Correct.
25:51 Michael Kennedy: Is so high. If you run over a person one every 100,000 times, that's still a really big problem.
25:58 Ricardo Tellez: Exactly, that is the point. That is the point. For the main thing it works, it works. So I have been with some people from other companies that they are building this and they gave me a ride and the experience is amazing. So it's in a very good condition, but for all the conditions, for all the types of people, the types of clothes, the types of materials, so that's not working, is not working yet, yeah.
26:23 Michael Kennedy: Yeah, not yet. I'll share a really quick story with folks.
26:26 Ricardo Tellez: Sure.
26:27 Michael Kennedy: We rented for four or five days a Model 3 Tesla and we were driving and my wife was, I really needed for my wife to try it out, and we were driving down the street, and she really wanted to try the self-driving, so she turned it on and it was fine, but it got to a place where the stripes on the road were faded or it's the, Tesla became uncertain of the environment, and it told her to put her hands on the wheel and take control of it because it needed her to be ready because it was getting to this uncertainty zone, and she puts her hands near it, but not physically touching the wheel, so it didn't know she was holding on or ready to hold on. And it got really mad, it told her to do it again, and then after a couple of times of saying, do this now, it just said, no more self-driving for you and it pulled over and stopped. It just said...
27:12 Ricardo Tellez: Oh, wow!
27:13 Michael Kennedy: If you're not going to take the wheel, we're done self-driving, no more self-driving this entire drive until you shut the car off for awhile, then you can self-drive again.
27:20 Ricardo Tellez: Huh, no way.
27:22 Michael Kennedy: Isn't that funny? She's like I won't drive anymore. I'm like, it said to hang on. She's like, oh, I was hanging on, you were almost hanging on it, it didn't know. Anyway.
27:33 Ricardo Tellez: So now we're going to have fight with the partner and with the car.
27:37 Michael Kennedy: Exactly, exactly, like the car got mad at me. So, I mean, these are the ways they can deal with it now, but it's going to be, I do think though that cars have come surprisingly far in robotics.
27:48 Ricardo Tellez: Yes.
27:48 Michael Kennedy: As far as I thought. I thought this problem is just too hard for cars. They were actually quite close to this Level Five, they're not there for the problems we already talked about, but they're way farther than I'd have guessed 10 years ago.
27:59 Ricardo Tellez: Yeah, yeah, yeah, that's right. That also was a surprise for all of us. So it was like there was nothing and then suddenly boom, then it started the development. I think that it was, everything started with DARPA Challenge. I don't remember exactly the name.
28:15 Michael Kennedy: Yes.
28:15 Ricardo Tellez: That was one of the first. They have to cross a desert and then the first edition there was nobody, and the second edition there was the team from Sebastian Thrun who is the Founder of Udacity. Sebastian Thrun, and then he made it, and then on the next one says, so many of the cars they did it, but that's when it started, then from that point that's it. Then they created a company, this company was bought by Google, and then when Google started to do that, then it kind of like it explodes, it explodes. So now there's companies, driverless cars, companies many, many around the world.
28:57 Michael Kennedy: It's really amazing. There's a super-good documentary by NOVA, which is this public broadcasting organization in the US, a series called The Great Robot Race, and I'll link to it. And it takes you, it's like an hour and a half deep techy dive into this DARPA Challenge that you talked about, and it's really amazing.
29:19 Ricardo Tellez: Okay, I'll take a note to watch it.
30:43 Ricardo Tellez: This is my personal opinion based on my experience and that's what we teach in our academy. So beside this development environment, we teach people about how to become a robotics developer in our academy. That is also an online academy. It uses the same engine as the development system, so it's everything online. You'd see simulations of robot and then you learn. So what we teach there is that if you don't know anything about C++ or Python, then you should start with Python. So you start learning Python and then you start learning ROS with Python. So how to program in ROS by using Python. If you know Python already, then okay, then still start by learning ROS with Python. And the only option for starting with C++ is because, I don't know, it's your only option for getting a job, you are very, very stressed in that. The only option that you have to leave, but if you want to optimize your path, then start always by learning Python. ROS with Python, and once you know the basics of ROS with Python, then you can decide whether to go back and learn ROS with C++. Again, and why is that? Because ROS with Python is hiding, not many but a bunch of details that may...
32:08 Michael Kennedy: The important details, yeah.
32:10 Ricardo Tellez: Yeah, very complex to understand when you're new there. So it's easier for you to understand really how ROS works and how to make a robot based on ROS to do a task, to finish a task. So you can master that with Python a lot faster. Then in order to go back and relearn with C++, you have to reconsider if you're going to dedicate yourself for working, programming robots for industry or companies or you're going to dedicate to program robots for research. That is also a big space where developers can dedicate is to program robots just for doing research or making robot to learn new things, or discover new things with robotics. In that case Python is a lot better because in this case what is vital is to present the result as fast as possible. And for that, I mean, Python is the winner, is always the winner in that case so. Even if execution is lower, it's good enough to record the video and present the paper, and that's what is required in research.
33:19 Michael Kennedy: Yeah, absolutely, that's really interesting when you think about performance between Python and C++. If you're going to run some code just a couple of times or something and you factor in from the time I start working on the idea until I have the answer, maybe C++ runs five times faster, but it takes you a day in Python and a week in C++, right. Which language is faster? Right, like that's an interesting question as well.
33:41 Ricardo Tellez: Exactly. Python, Python definitely.
33:43 Michael Kennedy: Yeah, for sure. So one of the dramas, challenges, I guess, success stories coming up, but it's been a long journey these days is legacy Python versus modern Python or Python 2 versus Python 3, right. What's the Python 2, 3 story around robotics?
34:02 Ricardo Tellez: It's a pain.
34:04 Michael Kennedy: That is yeah.
34:06 Ricardo Tellez: It's a pain, it's a pain. I mean, so I'm talking here always about programming robots with ROS. Okay, so what's the problem with ROS that ROS works with Python 2.7 up to now, the latest version of ROS still works with 2.7. Then ROS has, it has started 10 years ago, and so now it has matured for 10 years so, but it became clear that this structure of the current ROS was not going to be sufficient for real robot applications for companies to adopt ROS into their products massively, because of that they decided, the organizing committee of ROS decided to create a new version of ROS that is called ROS 2. And then when they decided this, they started like three or four years ago to develop, to start development and they decided to do it in Python 3.7. So if you are going to jump into ROS now, then if you're going to jump into ROS 2, then you're saved. Okay, because you are going to learn Python, you're going to use Python 3.7. So the future is everything, yeah.
35:26 Michael Kennedy: You can use all the new libraries, TensorFlow, NumPy, whatever, the latest version of all the good things, yeah.
35:32 Ricardo Tellez: Even there are all the libraries that we use in robotics like the ones created by OpenAI. OpenAI, you know about that company.
35:41 Michael Kennedy: Yeah, tell us about it.
35:42 Ricardo Tellez: Okay, it actually was founded by Elon Musk, the Founder of Tesla, so the CEO of Tesla. And they're concentrated on creating very good reinforcement learning algorithms using Python. They use Python only, and they use Python 3, of course. So that's very cool because we use those, then you can use it in ROS 2, but if you're going to jump into ROS 1 because is the current standard, so most of the places you're going to use ROS 1 yet, then that's based on 2.7, Python 2.7.
36:19 Michael Kennedy: And you're kind of stuck in that world, right.
36:21 Ricardo Tellez: Yes, well, the next version, the next distribution of ROS 1 that is going to appear in 2020, it's going to be based on Python 3 already.
36:32 Michael Kennedy: Okay, well, that's a pretty positive story.
36:34 Ricardo Tellez: That's going to be a festival. I mean, it's going to be a big party about everyone because imagine everyone already changing their code into that other version, so it's going to be crazy.
36:46 Michael Kennedy: Yeah, that will be wild, but that's really good. Now maybe there's some code I've prototyped in Python and is working well, but I need to move it into production, I want to put it onto a real robot, maybe that robot has limited computation power or whatever, and I want to make it go faster. What steps do I need to take? Do I rewrite part of that code in C++? Have people used Cython on parts of their Python code to make it go faster or other things like that? What's the productizing my robot story?
37:19 Ricardo Tellez: I don't know about the Cython. How do you call it?
37:22 Michael Kennedy: Cython, it's like CPython without the P. So what you're going to do is you're going to take a small portion of Python, put type annotations on it, and then compile it to native code as if they were C, through a C Compiler and it goes many, many times faster. So if you've got a, like I've got a Python program and there's some tight loop or some one module that's 200 lines long or something, you need that to go fast, you could hit that with Cython, rename it to, I think it's C-P-Y, I forgot the extension, but rename it. And I don't think that's right, but anyway you rename it to a different file and you hit it with Cython and then it just becomes like a C module that gets imported by your other Python code.
38:02 Ricardo Tellez: Okay, so that's the first time that I heard about that, so I cannot tell you what is the difference between that, but usually what we do in those situations, in those situations is usually is first one, rewriting C++. So that's the, I mean, that is straight, write in C++ or another thing that is done is also to put a bigger computer. So you put a more powerful computer and robot, and that's it. If you cannot do that, also you can run in an external computer because ROS is how it is built, it doesn't care in which computer you're running the different programs in ROS, and they can communicate and transfer information transparently. For them is like if they were in a single big computer. So that's another option.
38:54 Michael Kennedy: Like a kind of distributed computing environment type of thing.
38:56 Ricardo Tellez: Yes.
38:58 Michael Kennedy: Okay, is it common to have kind of microservice architecture where I've got a robot and it's doing stuff, but then it's making calls over the network to get more information or resolve information, things like this?
39:11 Ricardo Tellez: It's actually for product, for robotics products is very, very common because they want to put, to ship into the hardware as less as possible and also they want to make you kind of trapped into their system, so you need that but there is a very famous case.
39:32 Michael Kennedy: Hey, the $4.99 for the continuous service. Monthly, right, or whatever the price is.
39:38 Ricardo Tellez: Exactly, so there's a very famous case that just had a couple of months ago, it's called Cozmo, Cozmo robot. You know that robot?
39:47 Michael Kennedy: No, I don't know this robot.
39:48 Ricardo Tellez: A small robot that you can have on your desktop, it was amazing. It looked like a, you know this movie of the robot of Disney?
39:58 Michael Kennedy: Yes, the kind of big fluffy rounded white. Yeah. Yeah, I know, oh man, I forgot too. That was a great show. Hero 5 or something like this.
40:07 Ricardo Tellez: No, no, no, I'm talking about that, about this other movie that there is a robot that it stays on the planet, is cleaning all the mess.
40:14 Michael Kennedy: Oh, yeah, WALL-E.
40:16 Ricardo Tellez: WALL-E, exactly, WALL-E.
40:16 Michael Kennedy: Yes, yes.
40:17 Ricardo Tellez: Exactly, so it looks the...
40:19 Michael Kennedy: that one is a sad movie actually. Yeah, anyway, sorry, go ahead, so it looked like WALL-E.
40:24 Ricardo Tellez: No, like the other one, the character. The female robot, white one. So the face of this Cozmo, it was engineered, I mean, by using graphic designers and developers just because cartoons in this sense. So it was super-cool robot, it was bot. Unfortunately it was not a success in business terms, so they have to close the company. And now this robot is useless, so you cannot even use it because it requires the network in order to understand your vocal command.
40:58 Michael Kennedy: Oh, I see.
40:59 Ricardo Tellez: For that to program.
41:00 Michael Kennedy: Yeah, because once the company goes out, then the services die, and if they lean on the services too much, then it becomes a useless junk.
41:08 Ricardo Tellez: Exactly.
41:09 Michael Kennedy: Yeah, yeah, that's a bummer.
41:10 Ricardo Tellez: There is a famous, even more famous example, that is called Jibo. Do you know Jibo? Did you hear about Jibo?
41:16 Michael Kennedy: No.
41:17 Ricardo Tellez: No? That was first site of the Time magazine. This robot was there and it was closed last year, the company, and then it was also the same problem, was the same problem because it was not successful in terms of business, and so they couldn't maintain and they, and so everybody that bought this, they kind of just throw it out because it's useless now.
41:42 Michael Kennedy: Oh, no, that's such a bummer. Yeah, I've had that happen, I had a Pebble smartwatch and it was beautiful, but they went out of business and like almost everything about it, it's still kind of cool, but it's way less than it was because the services are gone. So we're talking about putting some sort of computation unit on these little robots, maybe talk about what those computers look like, what those operating systems look like because we could have a very small little robot, the size of your hand and maybe it has a micro-Python on a $5 chip in it, and maybe we have the robot that's like a vacuum and it's got, I don't know Raspberry Pi running Linux, or maybe...
42:22 Ricardo Tellez: Yes.
42:23 Michael Kennedy: We have a car that's got a Core i9 CPU that's like the top-of-the-line Intel CPU or something like, what's the spectrum of computing you put into these robots and operating systems and all that?
42:39 Ricardo Tellez: Yeah, exactly what you mentioned. So we can do small robots that only require, maybe even an Arduino, only with Arduino you can make it work. You can make the robot perform the task that you wanted, then you have a Raspberry Pi and the like, then you have also the computer, the typical PC computer, for example, there was one very famous that is called Intel NUC, that is a square, like a cube, is small like this. Now the ones that are getting a lot of traction and very big success are the Jetson from Nvidia because they include some GPUs, and then they allow you to do some deep learning inside the board already there.
43:26 Michael Kennedy: Right, okay.
43:26 Ricardo Tellez: So in case that you want to use the data from a Point Cloud device. A Point Cloud is for a Kinect device, so that you can recognize objects or people.
43:37 Michael Kennedy: I hadn't heard of this Jetson thing, this is really cool and it's...
43:40 Ricardo Tellez: Yeah, Nvidia Jetson.
43:42 Michael Kennedy: They're very small and they look like they don't even have fans. They just have really big heat sinks on a...
43:47 Ricardo Tellez: Yes.
43:47 Michael Kennedy: how big. What is it like 10 centimeters by 10 centimeters, something like that maybe?
43:52 Ricardo Tellez: Yeah, yeah, yeah, so it's very, very small, yeah.
43:54 Michael Kennedy: Yeah, $100 for a Jetson Nano, that's pretty awesome.
43:57 Ricardo Tellez: Yes, and the power that it has is very cool. There is the Jetson Nano and then there is another version, is a Jetson, whatever. And then there is another one that is more powerful that is called a Xavier.
44:09 Michael Kennedy: Yeah, Xavier.
44:11 Ricardo Tellez: And it's even more powerful.
44:12 Michael Kennedy: These are pretty intense, they've got some really cool ones. Yeah, and some of them even have the GPU looking things embedded into...
44:19 Ricardo Tellez: Yes.
44:21 Michael Kennedy: So that's another thing maybe if you need more power, if you have maybe you want to, you talked about some of the machine learning stuff, and, of course, your background is in this area as well. GPUs probably play an important role for certain types of robots, I'd guess.
44:34 Ricardo Tellez: Yes, exactly, especially for example, it makes sense in image recognition. Whenever you need your robot to recognize where it is the location or the people or the objects by using a camera, then that's one of the cases. Also in the speech recognition also. And then also they just use a lot the GPU for training, for launching reinforcement learning algorithms that make the robot learn, but those usually are using the simulation because the real robot is very difficult to train the robot because in order to train a robot, you need to put into a certain conditions, then make them take some actions and when they fail, you say, hey, wrong. And then you had to put it back. Yes, yes, it's like teaching a kid or something.
45:25 Michael Kennedy: Yeah, teaching a child, yeah, I see.
45:27 Ricardo Tellez: A child, exactly. And then you say, yeah, very good, very good, and then the robot gets excited, oh, yes, super-cool. And then you have to have somebody to put it back to the original position, and also all the environment get studied, so that's not good approach to do it with the real robot. And usually this is done in the simulator. And once you're in the realm of the simulator, you have the car computers that they have whatever you want.
45:51 Michael Kennedy: Sure.
45:52 Ricardo Tellez: Related to the operating systems that we use there, usually we install Linux, Ubuntu specifically, and then on top of that is where we install ROS because ROS even if it stands for Robot Operating System is not an operating system, is a framework for programming inside an Ubuntu system.
46:14 Michael Kennedy: I see, so maybe it's operating system for the robot, but it needs its own computational operating system, which is Linux, yeah.
46:22 Ricardo Tellez: Exactly, that's it. That's it. And now Microsoft has started to create a version of ROS for Windows also. So you can install Windows instead of Ubuntu, but is still not finished, so still very early stage.
46:37 Michael Kennedy: Yeah, that will probably bring in a lot more developers as well though. Windows is still a big platform that people develop on in a lot of places, so having ROS there sounds like a good idea.
46:47 Ricardo Tellez: Yeah, they are putting a lot of effort into that.
46:50 Michael Kennedy: Cool.
46:50 Ricardo Tellez: Yes.
46:51 Michael Kennedy: I see robotics as a interesting way to bring young people into programming as well, right. Like as you get older, you're willing to just plow through the things. Okay, I've to learn the for loop and then I learn the class and I learn the functions and eventually I'll do something interesting, but kids they don't care, they're like, this is boring, I'm out of here after five minutes, right, but if they could see a robot. My daughter is in sixth grade, my youngest one and she had whatever electives in the morning is robotics. Actually she just started this week, I'm not really sure what they're doing, but it's really I think a good way to teach kids, and I think that might be the Microsoft angle as well to some degree.
47:27 Ricardo Tellez: Okay, that's a strange for me from my point of view of roboticist. For me, I mean, it's very clear to use Linux. I mean I'd never go to Windows to program robots. For me it's like, what?
47:38 Michael Kennedy: Yeah, of course.
47:40 Ricardo Tellez: I mean, all this big thing that you don't need, that you don't need because you don't need to have the terminal and the commands there and then everything works. You have full control of everything.
47:52 Michael Kennedy: Well, I think there's these environments where kids are already sitting in front of the Windows PC and they're going to use it or they're also, Microsoft is also working on Edge IoT computing stuff. And I think it kind of fits into that realm. And I think that even they have some embedded Linux stuff that they are doing over there, which is kind of ironic and funny.
48:10 Ricardo Tellez: Okay, I didn't know about that.
48:12 Michael Kennedy: Can't remember exactly. I think I can't remember what it's like, something Sphere, it's at the edge of my knowledge. I should stop talking about it, but I do believe they had some embedded Linux somewhere, if I find it, I'll put a link in the show notes afterwards.
48:26 Ricardo Tellez: Oh, you mean Red Hat.
48:27 Michael Kennedy: No, no, not Red Hat, there's something from...
48:30 Ricardo Tellez: Red Hat, they bought the Red Hat version, isn't it? Didn't they?
48:34 Michael Kennedy: It's called Azure Sphere, that's what it is. Yeah, so it's Azure Sphere, and it's basically a hardened, I think it's Linux, let me see, I'm not sure, but it's a hardened IoT device that basically has automatic over the air updates and a whole bunch of various security stuff and so on, but yeah, I got to look into it to make sure that it's Linux, but I think so. Anyway, the world is getting confusing, right, like it didn't used to look like this.
49:04 Ricardo Tellez: Yes, is getting confused. And related to what you mentioned about for the kid, yes, definitely, the more graphical, the better. So there is this, that is called with the programming for robot using blocks, that is called Blockly. Yeah, okay, Blockly is one of them, but this is based on another one made by the MIT, basically is programming robots.
49:29 Michael Kennedy: Scratch, maybe.
49:30 Ricardo Tellez: Blocks, yeah, Scratch, Scratch. Yeah, yeah, that's it. Scratch, okay. Okay, so there exists also a version of Scratch in ROS. So actually you can, if you are interested on doing.
49:42 Michael Kennedy: Oh, okay, so you could started really early, I see. Okay, that's super-cool.
49:47 Ricardo Tellez: Exactly, so for that in that case for kid and for people with lower level of knowledge in terms of mathematics, physics.
49:57 Michael Kennedy: Maybe they're not typing really well yet or some like that, yeah.
50:00 Ricardo Tellez: Or programming skilled, yeah, exactly. So that makes sense.
50:04 Michael Kennedy: Yeah, very cool, all right, just to wrap-up this Linux thing. Azure Sphere is a Linux-based operating system created by Microsoft for the Internet of Things applications. It's the first time Microsoft has publicly released an operating system right in the Linux kernel. All right, so very interesting, the world is confusing and very mixed. It's a fun time. All right, so let's wrap-up this conversation, Ricardo, by talking about how people can learn ROS and program robots with Python. We kicked it off by saying it's a good time to be a robot developer, Python is a good language to do it. What if people want to get into, what do they do?
50:37 Ricardo Tellez: In that case there are several ways of learning, and then that depends, it's always the same. You want to invest more time or more money, so that's the situation. So let me give you the options that I usually provide. One is going to the official wiki of ROS. So ROS is at present being created and developed by now is a company, it's called Open Robotics. And they release it for free, it's open-source, you can do whatever you want with it, and then at the same time the evolution of wiki with some tutorials with a lot of information there about how to learn ROS. So that's the official wiki of ROS. That will be the first place. It's a little bit dry as a first step, it's a little bit dry, but I learned by using this. So I learned using this because there was nothing else at that time. Then the other option, yeah, so we being very early.
51:36 Michael Kennedy: Coming early you got to just go through the docs and figure it out.
51:40 Ricardo Tellez: But I have to say that for the time, those documents were amazing, the documentation. They put a lot of effort into the documentation because they knew that that was a very good point to attract people. So at that time that was like, wow, that's amazing. I mean, it's dry, but is amazing. Now some years afterwards, so it's a little bit more complicated, but you have other options, for example, books. There are a lot of books in Amazon, if you go, you will see a lot of books. We have our own series of books that teach you the basics of ROS or how to use it for navigation, but there are many, many other books already available. Use the books, they're very nice. Then another way is to use the YouTube video. So in YouTube you'll find a lot of videos, lot of videos. We also provide this videos in our channel of our company, and actually we do a ROS live class on YouTube every Tuesday at 6:00 CET.
52:44 Michael Kennedy: Oh, cool. Yeah, you just cover some, it's like a live stream, you just cover something different each time and people can just drop in.
52:50 Ricardo Tellez: Yes, it's a different subject, it's a different particular, very concrete problem.
52:57 Michael Kennedy: Something you can do in an hour, right.
52:58 Ricardo Tellez: Exactly.
52:58 Michael Kennedy: What did you do last time?
52:59 Ricardo Tellez: Yeah, so what is interesting is that the participants that connect, they're programming at the same time with me. So I provide to them on this live class with ROSject that they get directly into the class and then they start automatically into their browser with all the environment, the simulations and the instructions. So we are doing together, so I'm explaining, do this, you see here is, do that, do that. And then the guys they are doing and then as I send requests here and so on. So, and then last time what we did was about making several robot create the map of an environment. So several robot, not only one robot, but several, they were moving around and creating the map. That's the concept in robotics.
53:47 Michael Kennedy: Yeah, that's super-cool.
53:48 Ricardo Tellez: Well, that is another of the options using the YouTube videos. And I'm telling you about those, of course, I'd tell you that you go to our academy is the Robotic Ignite Academy, but that's a pay-in one. So the previous one, the official wiki is for free, the books is the price of the books, the YouTube videos are for free, so everything is free.
54:11 Michael Kennedy: You've got to watch an ad maybe, but other than that it's free.
54:14 Ricardo Tellez: Oh yeah.
54:14 Michael Kennedy: Because of YouTube, right.
54:15 Ricardo Tellez: Yes, exactly.
54:15 Michael Kennedy: Right. That's just YouTube itself.
54:17 Ricardo Tellez: Yeah, yeah, exactly, that is right, that is right.
54:20 Michael Kennedy: Yeah, and then you have your academy.
54:21 Ricardo Tellez: Yeah, the academy is online, and it's not based on videos, it's based on practice. So you have to practice and do the things because whatever you read there, if you don't practice, it's not useful at all. And especially in ROS, you need to see when you do this, what the robot does? And this is the system that we have implemented in our platform, in our academy. It's a system that where you get the instructions, so now you're learning these, then very quickly do this exercise, so you practice and you do this. So that's more practice focused, but it has a cost, it has a cost. And I think that those are the main options that they have for this. Of course, you have conferences.
55:08 Michael Kennedy: Sure.
55:09 Ricardo Tellez: Yeah, around the world, summer schools.
55:10 Michael Kennedy: Yeah, podcast. Now that's more awareness I'd say, not in total straight on learning. It sounds like you can use the simulators, either your online one or the Gazebo one or something like that, and you don't necessarily have to start with hardware. Is that right?
55:24 Ricardo Tellez: Exactly, that is right. Actually our online simulator is Gazebo. So we have created a version of Gazebo at, in the inside in our server is Gazebo running and then we have a frontend in web development for that. So is exactly the same thing that you will run in your own computer. And then that makes you be separated from the hardware.
55:47 Michael Kennedy: That's a great way to learn. Okay, a lot of options it sounds like and a lot of interesting possibilities out there. So, cool, well, thanks for sharing that. Now before you get out of here, I have to ask you the final two questions. If you're going to write some robotics code in Python, what editors to use?
56:01 Ricardo Tellez: What I used to use before it was Sublime. Sublime was very good. Also I used the Qt Creator, but Qt Creator was more for C++. And then I was doing kind of a hack, hack for Python, but then Sublime was the best. And now at present I'm using a web editor, IDE, which is called Theia, or Theia, I don't know how to pronounce in English.
56:26 Michael Kennedy: Yeah, something like that, uh-huh.
56:28 Ricardo Tellez: And that's the one that I'm using. Very cool, it has Autocomplete, you can jump into the functions, you can see the documentation, and so it's very cool.
56:37 Michael Kennedy: Oh, very interesting. All right, I'll put that in the show notes, so people can check that out. That's interesting.
56:41 Ricardo Tellez: It's open-source, it's open-source, so you can download, install in your computer or in your web server and program into a server. It's very cool.
56:51 Michael Kennedy: Nice, and then notable PyPI packages, what's the important stuff that people should know in this world?
56:57 Ricardo Tellez: I mentioned already, the OpenAI Reinforcement Learning algorithms, that's very cool. The Gym system is a Gym system for learning about reinforcement learning, not only for robotics, but for making, for example, programs play Atari games.
57:16 Michael Kennedy: That's cool. Use games as their world, right.
57:19 Ricardo Tellez: Exactly, and then they learn to play those games, like the Pacman, the Tetris. So that's OpenAI Gym, OpenAI Gym.
57:28 Michael Kennedy: Nice.
57:29 Ricardo Tellez: And then another one that I like very much is SIDEKIT. SIDEKIT is a set of Python libraries for many, many, many different techniques of artificial intelligence, not only for reinforcement learning, but many others, and supervised learning, supervised learning and even genetic algorithm. So I'll recommend you those and some other too that are very commonly used in robotics are OpenCV that probably all the people already know, and TensorFlow, of course, for deep learning.
58:05 Michael Kennedy: Very cool, those are great. OpenAI Gym looks awesome, I've put a link to that as well. That's a really nice option. All right, Ricardo, final call to action, people who want to become robotics developers, what do you tell them?
58:17 Ricardo Tellez: So we need you because, I mean, we need programmers that can program really, really well, and the robotics field is lacking this. Actually there is a very cool video, is super-cool video. It's a video where a person is showing how a robot is an arm based robot, it's a mobile manipulator. This robot is cleaning a room from all the mess. So you can see that the robot is grasping all the books, putting there on the place, then taking all the clothes from the ground, putting into a basket, taking the basket, bring in there. So you can see this speed plus 20 or something, okay, very slow, but it can do everything, everything. Then at the end, the guy of the video says, yeah, so I've shown you that these robot can do that, but the thing that I didn't show you is that I was controlling the robot.
59:14 Michael Kennedy: Ah!
59:15 Ricardo Tellez: So he was teleoperating the robot, but what's the conclusion here. The conclusion here is that the hardware, it's already there in order to have a robot that can clean the mess of a room. The only important part that is missing is the software, is the head of the robot that makes it. So that's why we need developers that can build those programs.
59:39 Michael Kennedy: Awesome.
59:39 Ricardo Tellez: So that's the problem.
59:40 Michael Kennedy: Yeah, well, it sounds like a lot of interesting stuff to work on. Thanks for sharing the story and giving us a look inside the robotics world.
59:47 Ricardo Tellez: My pleasure, it really was very fun. As I mentioned to you, that's the first time that somebody has interviewed me on a podcast, and it was an amazing experience. Thank you very much, Michael.
59:56 Michael Kennedy: You're welcome, thanks for being on the show. This has been another episode of Talk Python To Me. Our guest on this episode was Ricardo Tellez and has been brought to you by Linode and Tidelift. Linode is your go-to hosting for whatever you're building with Python. Get four months free at talkpython.fm/linode. That's L-I-N-O-D-E. If you run an open-source project, Tidelift wants to help you get paid for keeping it going strong. Just visit talkpython.fm/tidelift, search for your package and get started today. You want to level up your Python? If you're just getting started, try my Python Jumpstart by Building 10 Apps course, or if you're looking for something more advanced, check out our new Async course that digs into all the different types of async programming you can do in Python. And, of course, if you're interested in more than one of these, be sure to check out our Everything Bundle. It's like a subscription that never expires. Be sure to subscribe to the show. Open your favorite podcatcher and search for Python, we should be right at the top. You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. This is your host, Michael Kennedy, thanks so much for listening, I really appreciate it. Now get out there and write some Python code.