#255: Talking to cars with Python Transcript

00:00 KENNEDY: Modern cars have become mobile computer systems, with many small computers running millions of lines of code. On this episode, We plug a little Python into those data streams.

00:11 KENNEDY: You'll meet Shea Newton, who is a Python developer and who has worked on autonomous cars and is currently at ActiveState. This is Talk Python to me. Episode 255 Recorded February 25, 2020.

00:38 KENNEDY: Welcome to Talk to Me, a weekly podcast on Python, The language, the library's the ecosystem and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm at in Kennedy. Keep up with the show and listen to past episodes at talkPython.FM and follow the show on Twitter via @talkpython. This episode is sponsored by Data Dog and Clubhouse. Please check out what they're offering during their segments. It really helps support the show.

01:02 KENNEDY: Shea, welcome to _Talk Python to Me_.

01:03 NEWTON: I thank you for having me.

01:04 KENNEDY: It's great to have you here. And by have you here I mean literally have you here. I've done a few recordings on location in Portland, but this is live from the Talk Python Studio.

01:16 NEWTON: Yeah. Great Set up. I'm happy to be here.

01:19 KENNEDY: Yeah, Thanks. I'm really happy to have you Here is. Well, it's gonna be a lot of fun to talk about Cars.

01:23 KENNEDY: When I grew up cars were these, like, dirty, greasy things. I hated work on cars. Like, probably I would have done different stuff with my life if I didn't hate working on cars so much. I actually used to race motorcycles and we will rebuild engines and we would take them apart. And it was ridiculous. Those things had to be rebuilt, like every month the way they worked. Luckily, it was not very expensive. The parts,

01:23 so I didn't really hate mechanics, but I don't know,

01:23 I just banged my hand and I just didn't like working on cars. Always admired people could. But I was really good at computers. And now cars are computers. Yeah.

01:59 NEWTON: Yeah, I think people are out there hacking in the grease and in the computer of their car to Yeah, Yeah, absolutely.

02:03 KENNEDY: Yeah, yeah. Super Cool. So that's the topic for the show today is to talk about talking to your car with Python, which I think is gonna be real fun. (Yeah, Yeah).

02:13 KENNEDY: Nice. Now, before we get to that, of course, let's start with your story. How'd you get in a programming in Python?

02:17 NEWTON: Yeah, I went to the University of Idaho for a computer science degree. It was just something that was itching the back of my brain that I wanted to learn how to do. And you jump straight into it

02:29 NEWTON: Today, I work for ActiveState as a developer advocate. And I'm actually just kind of out spreading the word a little bit about what cool things that they're working on.

02:37 KENNEDY: Yeah, well, tell us what a developer advocate does.

02:42 NEWTON: Yeah. Good question. Yes. So some of the day to day for me is to open source communities maintainers just about ways that active state can work with them about the cool things that they're doing. Developer advocate, reach out. Talk to community. Talk to open source maintainers.

02:59 KENNEDY: My impression is that there's like, these different roles that developer advocates play and they can vary really dramatically. Sometimes that's giving speeches at, like, conferences or user groups. Maybe it's like I'm going to travel on site to customer site and they're trying to adopt some technology, and I'm gonna sit with them, almost be like a drop in consultant expert. There's also the ones who may be right blog posts or whatever. Like yeah, all The above.

03:29 NEWTON: Yeah, a little bit of all that. There's some block post writing. I was just a developer. Week 2020 conference in Oakland. I'll be at InterOp in Austin later this year. Yeah, just kind of toes in a lot of the communication sides of the work.

03:43 KENNEDY: Yeah, it's not the main topic, but what do you think the qualifications are? Also, I should maybe just throw out there back in numbers, escaping my mind. I did an episode with Paul Everett, Matt Makkai, Cecil Phillip about being a developer, advocate for their different companies. So people really wanna dive until they can go check that out. But what do you think the qualifications are like? How do you go from? Yeah, I can get this code to compile. Or I could write a website and we'll ask now I'm representing the company is like an expert.

04:15 KENNEDY: (Yeah, Yeah) Look for I think like wait. I think

04:18 NEWTON: part of it is an empathy for the developer I think there's the technical know how, But there's there's two sides of, like what even developer advocate might mean. So there's advocating to the developer from the company. But there's also advocating for the developer internally,

04:32 KENNEDY: almost being a channel from the community back to the team.

04:34 NEWTON: Yeah, saying this is the kind of thing that's important. This is the kind of thing that people are actually needing to use it for the times that they've come up against. And I think that that's something that that I like to bring to the job.

04:46 KENNEDY: Well, it sounds like a really fun job. (Thanks.) Yeah.

04:50 KENNEDY: Now he's all this. You got into programming at Idaho University, Idaho University University. University of Idaho?

04:58 KENNEDY: How about Python? How'd you come to that?

05:00 NEWTON: So even kind of in the earliest days. So that curriculum was kind of based on C and C++, But

05:06 NEWTON: as I moved through the program, Python just showed up as the glue that held things together along the way And quick scripts, rapid development for things that we were working on outside of class. Independent projects. And the longer I was in the world of programming, the more Python showed up as this glue between things, and sometimes I was able to devote a whole project to it, and sometimes it was just one off. Things that that I learned is I went.

05:32 KENNEDY: It's in a nursing take. I definitely think that Python is much more applicable because it can consume or glue together these other things, right? Yeah, I was just talking to somebody and they were talking about It's really cool that we can take Python and we can then hide away really old fortran code for linear algebra. Or we can hide away the C Library that nobody really wants to work with. But we can, you know, use the C API to create a package around it, and no one has to know. But it's like, you know, 50 year old tested code. Yeah, nobody can get to work, but it doesn't matter.

06:05 NEWTON: I think that's so powerful. Yeah, I think that another thing that's interesting is, well, my sort of hypothesis is that companies that say that they develop in maybe Go(lang), or C or some other language if you if you ask them if they are also using Python I'm almost certain that they'll all say, Oh, Python, too, You know, like right? Yeah,

06:27 NEWTON: it's just there. It's just so, so ubiquitous.

06:29 KENNEDY: Nice. Yeah, very cool. So you just saw it, Like showing up everywhere. Like I probably need to know this.

06:35 KENNEDY: Yeah, I need a little.

06:36 NEWTON: I needed to get much better. Yeah.

06:37 KENNEDY: Yeah, a little bit. Like you have to know Javascript if you're gonna work on the web.

06:41 NEWTON: Yes. Yeah, yeah. Absolutely

06:43 KENNEDY: Super cool! Okay, so that's little bit about you. There's a couple of angles I want to take here all around talking to cars with Python First, let's start with science fiction. That's kind of not science fiction. (Sure)

06:43 It feels like science fiction. So, to me, when I was in school, lies in college, you know, I took just a couple of programming classes for some requirements, not a CS degree, but to me, it felt like artificial intelligence. It was people working in LISP, and they were trying to pass the Turing test. Like, if you have a chat, literally a text chat, with my program and you are tricked into thinking it's a real human. Well, then it must be real right?

07:26 KENNEDY: and that, you know, never really worked very well. One do is not very impressive, right? And then, you know, it felt like AI was never really going anywhere. And then all of a sudden, you know, we rented Tesla's not too long, just for a few days, and it was just driving around on its own in traffic like that is really far from a text chat that kind of works. Yeah, right. So that's why I say science fiction, because I feel like you know, that part of science fiction is kind of arrived.

07:59 NEWTON: Yeah, we're seeing a lot of incredible advances, and I think that machine learning and artificial intelligence is accelerating that. Absolutely.

08:07 KENNEDY: And you think Python is?

08:08 NEWTON: It's the defacto way they go about that for sure. (Really?) Yeah. In my mind, yeah, for yeah, handling data for machine learning. I mean, the library's are out there. It's just super powerful tool for that.

08:19 KENNEDY: Now. You did a little bit of work around this kind of stuff. Can you give us a sense of just like, what kind of libraries might be involved or like, what kind of even just what kind of code. Do we write? What kind of sensors are there like, What's that world look like, As a developer?

08:31 NEWTON: Oh, yeah. For cars specifically for Python. There's a layer on top of ROS called ROSPy, which is an incredible platform for talking to sensors and collecting data. So sensor data get the view of the world around. You are actuating controls on a robot.

08:53 KENNEDY: So ROS is the robot operating system. Yeah. Yeah. And ROSPy is the Python API for talking to ROS. Yes. So part of the way this might work is you just treat the car like a robot.

09:06 NEWTON: Yeah, absolutely. I think that we can, as our cars, like you said, get more and more like computers. And as we want computers to drive them, they're just robots. Yeah.

09:16 KENNEDY: Yeah, well, you know, jumping just a little bit ahead to some of the stuff will come back to this self driving a bit more maybe, but we were talking earlier about just. can we quantify how much this is true or not true about whether cars are like computers. And ah, if you go by lines of code, it's definitely in the realm of computers.

09:36 NEWTON: Absolutely lots and lots of lines of code.

09:39 KENNEDY: Random modern car. What would you say?

09:43 NEWTON: Yeah. Too many to count. Too many for one person to manage.

09:45 KENNEDY: So I did a quick search earlier, and I You know, according to Google, I'm not sure exactly what it's based on this on, but it says that the modern car, with their infotainment and their cloud stuff and all that sort of thing says uses basically 100 million lines of code. (That's boggling.) Yeah. Yeah. So that apparently comes from WIRED.

10:07 KENNEDY: So to give you a sense, that's 5% is much code, as all of Google as a company, like all of their service, is, that's wild. We also looked. And if you look at Windows 10 Windows is not typically accused of being like a then bloat free operating system. I mean, neither is macOS, either, really, but it's definitely not considered one like a lightweight, like mini operating system, right?

10:35 NEWTON: Yeah, it's the real deal.

10:37 KENNEDY: Well, Windows 10 has 50 million lines of code to the modern car. We're not talking self driving Tesla and nothing like Kia or something, right? (Yeah, absolutely) 100 million lines of code. That's a serious, bit of programming.

10:50 NEWTON: Yeah, it's Yeah, incredible effort of decades.

10:54 KENNEDY: So it’s pretty interesting just to think of how much computation is going on. Also, I think I heard somewhere that there's not just like a central computer, but there's a bunch of little micro computers and they need to interact with each other, right?

11:08 NEWTON: Yeah, absolutely little computers all over the car that are talking to them

11:12 NEWTON: to each other in their own sort of protocol Or the CAN protocol. Which priorities? For that?

11:18 KENNEDY: I wonder if car technology had been adopted later. You know, how much would this just look like microservices with discovery. Yeah, (yeah, great question), right, Because it's not JSON and buzzing around the wire, is it?

11:29 NEWTON: No, no, it's just raw bites,

11:33 KENNEDY: which is fine for performance and stuff. But I have no idea what the connectivity like the network cabling between these little microcomputers are. But there's aton of computation going on in these cars.

11:44 NEWTON: Yeah, absolutely. Every I mean from from the big stuff, like pushing the brake pedal to the little stuff, like turning on an overhead light. Everything is in a modern car is letting it. So the car's aware of it in many places, when you do things like that,

11:58 KENNEDY: Right? Traditionally older cars. Maybe there's some little micro controllers in, like an eighth set, eight track cassette player or something. But when you push on the break that pushed directly on a lever which pushed on some hydraulics which then pushed on brake calipers, or whatever they're called see, I don't know. I don’t work on cars.

12:15 KENNEDY: And Makes it stop right creates friction. Heat goes into the air and the car stops.

12:19 KENNEDY: But now, not so much. I mean, it's like all the stuff that you do turning on windshield wipers like my car is fly by wire, right? I pushed on the brake and push on the gas, and it interprets what that should mean for it to go. Yeah, it does. So there's there's a lot of data's zooming around, right? (Oh, yeah, yeah, yes.) You gave a talk on this at Portland. PDX Python meet up, right? Yeah, yeah, yeah, that's cool and all. Definitely linked to the video that was recorded there on a link to that video there. But you had this graph have, like all the data going along in a Kia some kind of Kia Rio or something accelerating, right?

13:01 NEWTON: Yep, yep. Just the data, you have to the noise you have to filter through And just what the bites look like as you're looking at activity growing. Yeah,

13:10 KENNEDY: right. Yeah. Super cool. Let's talk about that for a while. Let's talk about talking to cars. Surprising, not surprising I don’t know. It's a little surprising to me, I guess. But cars generally have the same architecture, For like their communication protocols and their hardware and stuff, right?

13:27 NEWTON: My understanding is the cars at least have to have an interface for this ODB2 port. Whether that's an international or a federal standard, I can't remember but a way for things like emissions tests to plug in and poll the vehicle for data. Very specific data. So there's that and most cars are still using this CAN protocol on this CAN bus. I think that that's pretty much the standard over these things.

13:52 KENNEDY: Right? Right, right. And I suspect that's largely due to suppliers in the OEMS. If you're gonna make a car, you don't make the brakes from scratch necessarily. Maybe you buy them from Bosch or you buy them somewhere else. And those things will have to be interoperable.

14:07 NEWTON: They do. And there's also a history of things already having safety qualifications. So the work that's involved in having a new technology brought into a car is incredible. And so if something already exists that has the safety rating for it, it can be very cost effective to just maintain.

14:25 KENNEDY: right, almost like medical right. It's like once this thing's approve. Yeah, it's a huge barrier to dethrone it or whatever, because you've gotto take anything new. I can imagine that. So, OBD onboard diagnostics. To get your registration, renewed a lot of places. Oregon, where we are in California and probably places in Europe.

14:47 KENNEDY: You've got a yearly every couple of years. Have your car evaluated to see how well it's doing. So those places when I had a car that burned fuel. When I would go there and they would get in and I would put me in this little closet thing and say, “Wait, wait for five minutes” in the car would like run and stuff would happen and then they would unplug it. It's okay. You can go or you can't. You have to get this thing fixed or whatever.

15:12 KENNEDY: So that's kind of some of the stuff that we're talking about right there, plugging into the same thing that we could talk to with Python.

15:18 NEWTON: Yeah, absolutely. So the talk that you mentioned was was specifically about using that OBD port to poll your vehicle for information, unpack and get a better look at it. Things like your emissions tester is plugging into that port using the accessible data. There's also, I think, the insurance companies are doing some driver monitoring.

15:39 KENNEDY: Yeah, absolutely. My daughter recently went through this. She's going to be 20. She's 19 now, and she bought her own car, and her insurance was surprisingly expensive for her. She's like, Well, this is pricey, But her insurance company did offer, like we will send you this dongle thing. You can plug in your car and it will monitor all sorts of things. Little monitor you're breaking rate, your acceleration rate, you're turning and like all these things, and then we'll give you a price based on analytics.

16:11 KENNEDY: I was like, There's no way that's going in my car. I don't need that thing, But it turns out, it was only there for two months. And then they said, Okay, we've got enough information. You can return it now.

16:24 Newton: Okay, so it's like an evaluation period.

16:24 KENNEDY: It is not like you permanently, at least for her company didn't permanently install it.

16:30 KENNEDY: But yeah, they were looking, I guess, into the same data.

16:33 NEWTON: Yeah, that's my impression. There's a lot of applications for It I think, just looking at the Wikipedia page or something. It's a little makes my eyes glaze over a little bit because it's so specific and (yeah) and particular. But there's also companies, there's one that I remember called common AI, I think that gives you a tour of the plug in there, and it helps visualize and represent that data in a little less, “I glaze over it” way.

16:59 KENNEDY: Yeah. Okay. Yeah. Super cool.

17:03 KENNEDY: This portion of talk Python To Me is brought to you by Data Dog. Let me ask you a question. You have an app in production that's slower than you'd like? Is its performance all over the place? Sometimes Fast, sometimes slow?

17:15 KENNEDY: Now here's the important question. Do you know why? With DataDog you will. You can troubleshoot your Apps performance with data dogs, end-to-end tracing. Use the detailed flame graphs to identify bottlenecks and latency, in that finicky app of yours. Be the hero that got that at back on track at your company. Get started today with a free trial at talkpython.fm/datadog.

17:39 KENNEDY: If we're going to talk to it, it's one thing to get a package little IOT thing from whatever insurance company they say. “Find the hole that looks like this and jam it in.” (Yeah), it would do crazy things like it would beep you if it detected a bad driving experience. You would hear Beep beep beep because now there goes money...

17:58 KENNEDY: anyway, instead of that, if we want to talk to it when I'm thinking OBD2 or CAN bus I'm not thinking USB-C, or Bluetooth or wireless has, like the plug-in like So how do I plug in? You know, something that talks Python into this?

18:13 NEWTON: Yeah. So basically, with the right cables, you can set it up to just go from the USB on your computer. When I developed this program, it was just using Ubuntu 18 Linux machine plugs to RS232? Something like that.

18:20 NEWTON: Like the serial port equivalent?

18:30 NEWTON: Yes. And then that converts to the specific OBD2 port. And from there, my kind of half joke is that talking to the car with Python is as easy as any other IO like, if you've ever written to, opened a file and written to it and read from it, that's the equivalent of what you have to know how to do for communicating with your car.

18:52 KENNEDY: Okay, once you get that all connected, that sounds pretty easy. Just opened up something different to talk to the port that it's on. (Exactly) Yeah. So in Python, the way you talk to files `with open(filename)`

19:03 KENNEDY: What is it like `with car` or like how do you do it?

19:08 KENNEDY: Yeah, you know what’s The libraries for talking to the car.

19:11 NEWTON: So talking to the car. There's just a library called Python CAN That's made thio interface with CAN Networks. And you just call open on it. Yeah.

19:20 KENNEDY: Okay, awesome. Really? It's good, right? Yeah. Yeah.

19:23 KENNEDY: And then, is it? I don't like streaming data. You just say I'm reading from it. Assumes that sees a bite. It just kicks it out.

19:29 KENNEDY: Or how's it work. Something like that?

19:31 NEWTON: You can poll for specific information, or you can listen to streaming data and grab, I think it's called a PID. I'm not sure if that's the same process idea or not, but, yeah, you can ask it for information and you can listen for information. So the example program that I'd written up asks for the standard, the version number of the standard that you're looking at in your vehicle.

19:53 KENNEDY: I see. You gotta figure out what the communication protocol version is.

19:55 KENNEDY: So you know what message to send?

19:57 NEWTON: Yeah things change. Yeah, for sure. And when you're dealing with the low real estate of bytes packed on the wire things, they're subject to change in ways that break the math, unpack them. So be careful.

20:08 KENNEDY: Okay. Yeah. So a lot of what's exchanged our can data frames and when they're called. Yeah, but these air like eight byte packets, and you just have to figure out what they mean.

20:19 NEWTON: Yeah, that's where kind of the standard comes in handy. So typically, it will say you have these eight bytes. And in order to do what you need to to get out of floating point number, there's a bit shift operation and a division and some scaler to multiply.

20:37 KENNEDY: Right Right, the first two bits indicate the message. Type second to bits indicate something like that? Yeah, totally.

20:44 KENNEDY: It's ultra packed. Yeah, yeah, yeah. Okay. I'm telling you, if this was created today, It would just be JSON.

20:53 KENNEDY: but it's built in a time. I don't know when this first came out, but it's gotta be like eighties.

20:58 NEWTON: Oh, I think I think it goes back further than that.

21:02 KENNEDY: You think so? Yeah, the computing power. They part like, you know, we can't send the name, but we can send a bit.

21:09 KENNEDY: interesting. So there's probably some higher level API or package as well that given one of those frames information.

21:17 NEWTON: There are my example was just the program I had written was just the hands on Go do it yourself, hack on your car or listen to what your car is saying. Python does have a OBD or OBD2 package for doing this stuff specifically, if you want to level up and start collecting that data or writing your own applications.

21:35 KENNEDY: Right. Right, Right. Right.

21:37 KENNEDY: Well, that sounds pretty awesome. I could imagine a bunch of bun use cases. Now, you said that maybe Tesla doesn't work with this format because

21:47 KENNEDY: some of the least that maybe doesn't have one of these ports, for example, potentially because the purpose is gather like emissions data. Yeah, of emissions. You're like, Well, I

21:57 NEWTON: remember some press coming out a while ago about at least one model of Tessa that that people

22:03 NEWTON: notice that there wasn't an OBD2 port for. And yeah, but from what I understand is that they didn't have to pass emission standards, so there was no reason to enforce it, right that they had that where other cars do. But I think recent Tesla's they've decided to incorporate it.

22:18 KENNEDY: I see, just to have some compatibility.

22:21 KENNEDY: I'm sure the native internals and Tesla's are probably much more modern. Yeah, it was. Ah, some company. I think Japan

22:31 KENNEDY: did like a tear down of a model three. It was Ah, a car. OEM Manufacturer, like they made transmissions or brakes or something.

22:38 KENNEDY: And they realize that for Tesla cars, there were many, many more parts that were from Tesla rather than third-party part's. interesting, and they're basically their self driving technology was all like Tesla's chips and whatnot, and they were

22:54 KENNEDY: basically concluding, like Tesla's like 6 to 7 years ahead. Because instead of waiting for all these OEMS to come along. they're just building it and there's nobody they depend upon. Whereas weir builds on on this House of cards type of thing for all of our suppliers. So

23:10 KENNEDY: yeah, pretty interesting. I'm sure there's different ways you can talk to Tesla's well, but these car companies, they don't like you to talk to their software,

23:17 NEWTON: so some are pretty protective. There's the public CAN bus that's easy to tap into that has the OBD2 port. And then there's private Can networks that are a little more invasive to get into. They're not publishing in easy to find places how to unpack those bytes. But there are cars like I think Lincoln has a car called the MKZ that you can buy a kit for, and I think it's actually like supported by Lincoln or they Yeah, but endorsed somewhat. And it's four by-wire control. It's for just hooking it up for autonomous vehicle testing. So at least Lincoln has seemed to have opened the door to this stuff.

23:57 KENNEDY: Wow, that's pretty interesting. I guess you have to have something that will actually do the steering on this train will like It's gotta have a little bit of, like lane-assist or something.

24:05 KENNEDY: Yeah, yeah, that's an interesting point.

24:08 KENNEDY: Or how else are you going to like turn the wheel or press the brakes.

24:09 NEWTON: There's one of the two of organizations in Europe is doing, Or last time I checked was doing testing with actual robots in the car. So, like an actual kind of humanoid shaped thing that would turned

24:21 KENNEDY: Like it pushes the gas or it rotates the armor. There's a rotating front thing.

24:28 NEWTON: Yeah, there's a lot of approaches to I think a lot of people are getting creative on how to do it, especially in cars where they don't necessarily have permission by the manufacturer

24:38 KENNEDY: t like, you know, we don't need to get some of those expensive cars. We're gonna get used whatever and just put the robot, in it? Yeah, and it'll be fine.

24:47 KENNEDY: Well, it sounds kind of crazy, but at the same time, right that cars are meant to be operated that way. Yeah, and so pretty high fidelity for driving.

24:55 KENNEDY: Yeah. Well, I remember when I bought my car, we had to sign that we would not affect or modify or attempt to circumvent any bit of the software. Yeah, like if you change this like one your warranty is voided to, we're gonna be really angry. You know, the antilock brakes, the the lane, assist the airbags, all that stuff there, like stay away.

25:20 NEWTON: Oh, yeah. It's a big, big, _Do not try at home_. Kind of situation like the stakes are very high.

25:26 KENNEDY: Yeah, but there's there's some stuff, like definitely reading the data. Seems like it's kind of okay.

25:30 NEWTON: Oh yeah. So as far as. “Do not try at home“ trying to effect a change in your car or, you know, cutting a cord and tapping into the private CAN bus or something like that. Something I wouldn't recommend as at home thing. Right. But yeah, the data from the OBD2 port is just public access. It's just read data our request, something specific, but yeah, it's your data. It's It's something that they're doing that they're obligated to do.

25:59 KENNEDY: yeah, I mean, obviously the insurance companies were doing it without real permission.

26:03 KENNEDY: So what kind of use cases do you see here? Like there's the diagnostic side of things. There's the insurance side of things, but I feel like there's probably some fun data science that you could do.

26:16 NEWTON: Yeah, it does seem like for the data curious. There's a lot of interesting data to collect about your own driving patterns about the fuel efficiency of your car. I just learned about some communities of people that are just trying to maximize their car for fuel efficiency with interesting fins in that kind of thing. And probably the data around how your car is behaving from something like this port would give you a little more insight into optimizing for things like that.

26:41 KENNEDY: Yeah, I would think so. I mean, that's definitely one of the first use cases I imagined was. Well, maybe we could create our own little dongle that would create that data. Then we throw it into pandas, and we start answering interesting questions and maybe even some ML models. To say people who drive like this, like this is the most important thing for fuel efficiency. Yeah, besides not just stomping on the

27:02 KENNEDY: gas, stomping on the brakes. Right? But

27:05 KENNEDY: maybe there's something that people are not totally clear about, where certain vehicles or it just

27:10 KENNEDY: seems like a fun research project.

27:12 NEWTON: Yeah, I agree. I would be fascinated to see where that stuff where a project like that would go. It seems like often when you set machine learning to a task, the things that it finds for you are wild and surprising.

27:24 KENNEDY: Yeah, there they could be totally surprising. So, yeah, I think it be fun to do some data science. What do you think about IOT type stuff? Like, what kind of devices could we build and then stick on that port, right? Like raspberry pi, maybe.

27:37 NEWTON: Well, with the right cables, you can hook up a raspberry pi, for sure. I mean, like I said that Linux computer I had You can

27:43 KENNEDY: get power in the car, so you just fire that like, yeah, maybe to a fuse over, just like plug it into the cigarette lighter. Yeah,

27:52 KENNEDY: Who uses those anymore.

27:53 NEWTON: One of the incredible things about Python is it basically just works everywhere, right? I mean, yeah, it's the community, and the development of the language has been awesome and its multi-platform. And so if Python can run on it, you can listen to what your car's doing.

28:07 KENNEDY: But wonder if you could get microPython or circuit Python to work. That's a whole another level of small that goes from raspberry pi to $5 micro control. It seems to me like one obvious requirement which, honestly, haven't done enough with, but I think it would work is if you just had usb support for

28:24 KENNEDY: that adapter. Right? And I have a circuit Python. I know it's laying right here somewhere, and it has a little micro USB communicate with the computer and like, like, write code on it and stuff potentially.

28:37 NEWTON: Yeah, Yeah. Just essentially, Yeah, something to collect collect on. Yeah.

28:41 KENNEDY: Okay. Well, yeah, I think it would be super fun. I'm not sure I want to mess with my car or not, but it definitely sounds like if I was looking for a cool little IOT project. You know, this is something that it might be really cool to plug into.

28:54 NEWTON: Yeah, Yeah, I think. just listening. Getting a feel for what the data looks like. Just that that level is kind of fascinating to me. But then studying, too. Unpack and look at the way your vehicle talks. But yeah, then taking it to, ah, an application would be Yeah, could be a great direction.

29:11 KENNEDY: You know, if people are looking for, like, a science fair project. Oh, yeah, or something like Alright, well, we're gonna ask you some of the parents to install this on their car.

29:20 KENNEDY: Well, so you don't have to have

29:21 NEWTON: to install anything so you don't have to put software on your car.

29:25 KENNEDY: No. I mean, like, you just plug in the device.

29:29 NEWTON: Oh, Gotcha, yeah, yeah. Just that on its own is a little spooky, like, Yeah,

29:30 KENNEDY: yeah, but you're like, I don't really want to put that in my car. I'm sure some parents, though they're like, but my kid wants to do data science.

29:37 KENNEDY: We’ll plug this into the car to see what happened.

29:39 NEWTON: I heard something a while ago that it might be spookier to plug USB into your car. Sometimes I don't remember what what vehicle it was. But somebody installed, doom to play on their little console. Because you could just put something that said, `Runmecar.exe` or something. Yeah,

29:58 KENNEDY: yeah, it's a big entertainment center or a specific car there. That was some

30:04 KENNEDY: way. Yeah, So just plugging

30:07 NEWTON: in anywhere in your car has some. Your car's listening, and it's trying to decide if it's something that needs to.

30:13 KENNEDY: What is it, like, some sort of audio device play and put up on the screen forever. And if it has a

30:19 KENNEDY: `runme.exe`

30:20 KENNEDY: Yeah, yeah, it's gonna run it. That's what

30:23 KENNEDY: I I bring that up.

30:25 NEWTON: Just because I think that plugging in and listening to the port for your diagnostics it's like, no more invasive than then it would be, to.

30:33 KENNEDY: I would suspect it's less right? Yeah, that port is read only. Yeah, right. As opposed to...

30:39 KENNEDY: It’s a really good point. I mean, it's definitely a little nerve racking to plug any USB thing into anything else.

30:46 KENNEDY: Yeah, because you don't know, right? It could have no windows has that like, it'll try to, like, boot up certain types of things. If it's in there, probably Mac does as well. And yeah, you don't put that in untrusted places super cool.

30:46 So let's maybe dive into a little bit more about the ROSPy stuff. So with the diagnostics thing, you can definitely, you see, was happening, right? The brakes were applied, the car did this or than and so on

31:15 KENNEDY: With ROSpy is more like like we said, like, kind of like a robot, right?

31:18 NEWTON: Yeah. Yeah. So I think a lot of companies that are doing research and development or maybe even just development or autonomous vehicles and robots are using ROS. It is really powerful for data collection.

31:32 KENNEDY: So what we've been talking about so far really is just about collecting and analyzing and visualizing and drawing inferences from data, right? Yeah, but Rospy probably could also take actions

31:42 NEWTON: for sure. For sure it could. And I think the power of like Rospy over just ROS is the ability to kind of rapidly develop in much shorter amount of code and also to hook into these, like super powerful machine learning and data analytics.

32:01 KENNEDY: Like I can write this in scratch with C. Or I could `pip install terrace` Yeah, and then we’re off to the races? Yeah. Do you think a lot of people are out there Are using, like, RosPi plus open CV, like computer vision stuff or things like that? Or is it?

32:17 KENNEDY: Gotta have some sort of sensor, right? Vision? Seems like some of some of the options.

32:21 NEWTON: Oh, yeah. Yeah, I think because Ros is open Source. There's a lot of people developing for specific sensors. And so, you know, a lot of times when the new sensor is out or some new form of vision you'll see development and incorporating into the ros ecosystem.

32:35 KENNEDY: I see, like LIDAR or something. Okay. Yeah. Did you ever get to play with LIDAR?

32:38 NEWTON: Yeah, I played. I played with with some LIDAR.

32:43 KENNEDY: Maybe tell people what that is.

32:45 KENNEDY: That's a great question. What is it? What is it?

32:48 KENNEDY: And why it won’t just blind everybody around him, right? If it's lasers.

32:51 NEWTON: Yeah, that's funny. I've had LIDARs on my desk and I've hacked around and watched myself on the visualization, but I don't I don't know a lot about them.

33:01 KENNEDY: like, basically my understanding is that shoots out like an expanding beam of lasers and then looks at the reflections on what lights up it can tell a distance and sort of 3D sensing.

33:12 NEWTON: Yeah, yeah. So the sophisticated letter sensors do have multiple layers, and they spend 360 degrees and you can get these beautiful renderings of the environment. But there's simpler ones that are more of a 2D thing, okay? Or just particular field of view

33:26 KENNEDY: Like a horizontal access or like 20 degrees left.

33:30 KENNEDY: There's a thing here, exactly. There's a gap for this distance, which might be all you need. If you're trying to develop a little like a Roomba or something, A Roomba doesn't care. Being down or whatever, perspective or hills. It's just like I'm stuck on a cliff like No, you're not. You're on a shoe, but I’ll help you out.

33:45 KENNEDY: But yeah, it doesn't really distinguish a whole lot.

33:49 NEWTON: Yeah, yeah, and I think that some of those sensors light are is one that can be prohibitively expensive when you really get all the bells and whistles. But like a simple field of view, sensor might be a lot less expensive and so you can put it into a consumer electronics. Like a Roomba.

34:04 KENNEDY: Right? Yes. So if you wanted to. I guess if people want to play more with the self-driving aspect but they don't really want to just hook this to a car and just understand what the car's to tell them. But they want to put on all these sensors. Maybe there's smaller devices that they could play with, and they could probably skip the OBD altogether And use straight ROSpy.

34:24 NEWTON: For sure. RosPy, One of their first kind of example projects just gives you a list of the hardware you need and some recommendations for Soldering Irons and a couple of sensors that are affordable to make your own self driving kind of cardboard box with a phone in it. You know it's totally cool

34:41 KENNEDY: That sounds like a very expensive self Driving box.

34:49 NEWTON: But no, it's amazing what what you're able to do and so hacking into these other systems, or creating your own like that, like that ROSPy example. You can experiment with path finding algorithms. You can experiment with computer vision. Yeah, there's lots of fantastic stuff built in at to ROS and accessible with ROSPy.

35:09 KENNEDY: This portion of Talk Python to Me is sponsored by Clubhouse. Clubhouse is a fast and enjoyable project management platform that breaks down silos and brings teams together to ship value, not features. Great teams choose Clubhouse because they get flexible workflows where they can easily customize workflow state for teams or projects of any size. Advanced filtering quickly filtering by project or team to see how everything is progressing, And effective. Sprint planning Setting their weekly priorities with iterations and then letting clubhouse run the schedule. All of the core features air completely free for teams with up to 10 users. And as talk Python listeners, you'll get two free months on any paid plan with unlimited users and access to the premium features You get started today. Just click the clubhouse link in your podcast, players show notes or on the episode page.

35:57 KENNEDY: So there's two projects of maybe that they're worth giving a shout out to one that I recently learned about. I think it's pretty new is called Donkey Car.

36:05 NEWTON: Oh yeah, that's a cool one..

36:10 KENNEDY: Yeah, so it's an open source DIY self driving platform specifically for small cars like little remote control cars, you could pick up with your hand or even hold in your hand. So it's RC car plus a raspberry pi plus Python and the Python side, that's tornado. Karas Tensor flow and open CV. And that looks really cool, doesn't it?

36:30 KENNEDY: Yeah, So if people are out there thinking about like, maybe I'll try something like that comes with the really simple API. And it says you do things like,

36:38 KENNEDY: make your own car that'll drive itself. Or drive a car with your phone or laptop or record images or train a neural net to drive a car on different tracks. Like maybe you drive it and then you say See, Do that?

36:49 NEWTON: Yeah, yeah, I think that's fantastic. And the fact that it's kind of a nice, out of box experience and you don't have to worry about getting something wrong and wiring it yourself.

36:58 KENNEDY: Yeah, they have a bunch of cool examples. You know, if I was like an engineering professor or something, I'd be super excited about all the stuff they got going on there. So the API is pretty simple. I looked at it. You just `import donkey`,

37:11 KENNEDY: Then you create a vehicle, and then you just like, grab the camera and you told to do things. And so it's a really nice API. Wow. Another one that you brought up as we were talking before we hit record was this idea of programming A Roomba? Yeah, right. I'm not sure everyone listening knows what a Roomba is like. A Roomba is one of these little self driving, self navigating. Self-vacuuming, vacuums.

37:36 NEWTON: I think they’re driving.

37:40 KENNEDY: As Much as a vacuum drives. We have one of those.

37:40 KENNEDY: My wife loves it. Periodically, It'll get stuck on stuff. She'll ask me to dig it out, Take it back to its home. So it's OK, but those were getting pretty sophisticated. And you condone control those and programming right, now?

37:51 NEWTON: Yeah. I mean, this is This is another good question of what's officially endorsed by the company.

37:55 NEWTON: But I've had some friends... You don’t want to return it. Yeah, that have hacked on. And it's it's cool because you have this baseline of performance that you're able to see with with your vacuum and and how how it handles the obstacles in your house. But then you can also try out different path finding approaches and just see how it works,

38:14 KENNEDY: right? It would be cool to put I don't know how much of, Ah, like a sensor array that that thing has on it. It feels like I know the old ones. It was kind of a blind thing. Feeling its way through the room and have a good talking to the wall. They're okay. I guess that's a chair. We'll just, like, work away. I think the newer ones are a little more advanced because we'll get, like, a map of where it cleaned. (Oh, wow.) Yeah, like a three. It's like, Yeah, yeah, top down view of where it thought the table was, You know, (so cool). It's much more advanced. Maybe you could get into those as well and get access to that information.

38:48 NEWTON: Oh, yeah, I think that that that'd be a great project. Yeah, seeing where the rumors have come. What sensors air there. And,

38:55 KENNEDY: yeah, that'd be super fun. So I'll link both the donkey car. Well, that's easy. Donkeycar.com But I’ll link through anyway and then the Roomba Python API. So not only do you get a program that you get to program it in Python. Yeah.

39:08 KENNEDY: Yeah, that's that's pretty cool.

39:10 KENNEDY: When you think about the self driving cars like how computationally intensive do you have to be just to do those types of things? Can you have, like a Really? They said raspberry pi is safe for a donkey car. But maybe you have to only run simple algorithms. I don't know. You need GPUs. I know NVidea working on a self driving car. Yeah, meaning that at some point

39:29 NEWTON: I think it's just a cz much as they're able to fit, I think that whatever's available, they'll use on. And I think that a lot of the argument for using low level languages is trying to get as much speed as possible.

39:42 KENNEDY: Right, Because if you could take a higher res image. Yeah, then that means a lot for figured out. What's out there. Absolutely cool.

39:50 KENNEDY: Well, I don't know. I'm inspired

39:52 KENNEDY: to work on something.

39:53 KENNEDY: I'm not sure I'm gonna go hack my Roomba, But maybe,

39:58 KENNEDY: maybe, yeah, it sounds really fun. And also now I want to know if my car has one of those ports. Yeah, Go check it out and see. All right. Well, this is really fun to talk about these things, and we'll link to some of the stuff out there. I’ll link to your talk. Maybe just give us a quick catch up on ActiveState while you're here as well. You know? Yeah.

40:16 KENNEDY: Had think was Pete on to talk about machine learning libraries ago, but we haven't talked about ActiveState for a while, So

40:23 KENNEDY: we hear a lot about Anaconda and those types of distributions.

40:28 KENNEDY: Feels to me like after ActiveState plays kind of in that realm.

40:30 NEWTON: Yeah. And traditionally active state has been, ah, distributions in the same kind of vein as as Anaconda, Right? Just a big collection of popular packages that you don't have to worry about pip installing in that kind of thing. Okay, Python and for Pearl also. But now they're we're putting together a self serve version of that. So you might think, Oh, the active Python distribution has a lot of the great stuff I need. But there's 500 packages and and I only need 40 right? So you can go into the platform and say, these are my my 40 packages that I'm interested in pre-build it, test it for these

41:11 NEWTON: os and architectures. And then anybody who wants to use my project can pull it down and just have everything good to go.

41:17 KENNEDY: Okay, Cool. And you can pick, like, different versions of Python or can even, configure how Python is compiled. All right, What level?

41:26 NEWTON: So. There's multiple versions of Python. I think we're up to 3.8 or 3.9 now, like up to date with modern Python. Thousands of packages available we’re still in beta. So there's some flux in what builds and the information about the combinations of builds that you're trying is valuable to us and will make things better.

41:48 KENNEDY: Okay, Yeah, It sounds like a good project. And you're working on some ROS things. There's, well, it sounds like possible installed isolated things.

41:58 NEWTON: ROS is something that I think is one of the powerful things about the future of this platform that we're doing. ROS is a project that's notoriously difficult to install and difficult to update. I've spent multiple days getting ROS configured the way I need it. And so the vision behind what ActiveState is doing is that configuration is done on this builder of the ActiveState platform it, you just pull it down. It's set up already for you the same way. A distribution would be.

42:26 KENNEDY: Yeah, I would. Sounds a little bit like Docker.

42:28 NEWTON: I think of it as more of ah, pipenv them than a full on docker. Okay, there's some runtime isolation.

42:33 KENNEDY: I see more of that. It comes in a way that you know that it's gonna be set up to work, eh? So much the isolation side of Docker.

42:39 NEWTON: Some of the power behind it. For ROS, I think, is the same. That applies to current things that we currently support. Like SciPy and NumPy that have C dependencies, that you can get tangled trying to build them on your machine. So the builder on the platform tests and builds and gives it to you for your specific platform so that you don't have to untangle that. And the future of what the platform will be able to do is things like ROS, where it's just a pull it down. It works for you rather than having to do any of that yourself.

43:13 KENNEDY: Well, that sounds nice. Yeah. I mean, this is the fun part of programming not the unfun part.

43:18 NEWTON: It's funny like we have this this expectation that when we install things like Spotify, it just works for us, right? That's prebundled and pre-built. But when it comes to like our developer environment, we still have to build everything from scratch. There's still this funny thing where developers air just okay, have to build my new thing, right? So I think the idea is we get to a place where your dependencies You could install as easily as you could your browser.

43:42 KENNEDY: Yeah, Well, that's pretty cool. When Is that coming?

43:45 NEWTON: Oh, so the platforms live now, and we're building up our catalogue of Python and Pearl for macOS, Windows, and LINUX and the package list is growing. And the potential for the language list to grow too.

43:59 KENNEDY: Yeah, Yeah. Cool. All right, well, let me just close this out with a couple of high level questions about cars real quick here.

43:59 So I opened the whole conversation, as in science fiction is like here and is now just science or engineering in some regards of self driving cars. But we're still not out there where just self driving cars are creeping around like I just read this morning, somebody had projected, like a human onto the asphalt and Tesla's were stopping.

44:26

44:26 Yeah, there's a lot of salt circles around cars

44:32 KENNEDY: have seen, like a car painted in.

44:37 KENNEDY: Yeah, it's like the elephant that's tied down with a little tiny stake, the but it’s the car version anyway. So having More experience in many people, what are your thoughts? And it's like, are you just amazed that where it is? Do you think we're going to get to the science fiction level of it or

44:54 NEWTON: Yeah, I think that I am amazed by what we're capable of with this technology. I think for me the aside from what we're capable of in our technology, I think because this is all so new and because there are things like 100,000 lines of code and were coming from a system that has a foundation in building on safety assurances, there's a question that still, I don't think fully answered, which is how safe is safe enough for this new stuff. I think that some of the auto manufacturers are concerned about that. Certainly state and local governments are concerned about that like what? How safe is safe enough for it to be on the roads of our city, right? And I think that those right now in my mind, that's one of the most significant challenges to AV technology.

45:38 KENNEDY: Yeah, and I think we might have to just get used to a different kind of safe. So what I'm thinking is if there is a person who like his video being just run over by a self driving car in some horrific way, even if there are, you know, 10% fewer wrecks and 100,000 people are not hurt. If there's a graphic example, that's going to stick in people's mind, that's gonna run on news and people our cars and see if we can't have this. Maybe they're They are unsafe somewhat, but they're less unsafe then, like people driving cars, potentially. I don't know if that's true, but yeah, it just might take us a different type of it. We might have to all get used to it differently, right?

46:20 NEWTON: That's a really interesting point, and I think that at the beginning, especially like when people are still trying to decide whether or not they are willing to trust this technology.

46:30 KENNEDY: you get in the car alleging that uber that no one's driving.

46:34 NEWTON: I think that it’s really important for the community of people developing this technology not to violate the public's trust. And I think that really catering are just really keeping the public in mind is an important part of how the industry moves forward. I think if they're going to maintain the trust of the people that they'd like to be in these cars,

46:52 KENNEDY: yeah, I also think it's a huge challenge to have Ah, hetero genius world, right? It's one thing to imagine a world with really clear painted lines and tons of self driving cars that all communicate like Hey, I'm over here, I'm I'm intending to turn or whatever. But if there's grumpy drivers at rush hour and there with, um like that's as a whole

47:13 KENNEDY: different level of complexity,

47:15 NEWTON: I like to think that each of the auto manufacturers, with their own sort of machine learning decision making, there's gonna be like different attitudes of Ford's versus you know GM's or something like the Fords are grumpy.

47:27 KENNEDY: The Cadillacs are super aggressive.

47:30 NEWTON: You never merge when there's a Cadillac.

47:33 KENNEDY: yeah,

47:34 KENNEDY: yeah, it's probably true, right? Like if they have different models running, you're gonna make different decisions. Yeah, absolutely. Well, we'll see if they learn.

47:44 KENNEDY: All right. Well, I guess you know, probably that's a good place to leave it for the conversation. But I'm both somewhat hopeful for this technology. I'm mostly just really, really impressed with it. Although I'm not one of those people. Like, I can't wait for self driving cars to come because I hate driving. I like driving or I don’t mind it. I actually don't have to commute that much. So it's no big, I guess. Maybe that's why I don't hate it. Yeah, but I do think it's interesting technology, and it it really blows my mind coming from the Turing test chat bot.

48:10 NEWTON: Yeah. I was just learning a little bit about Eliza when I like. I think it was written in the sixties.

48:14 KENNEDY: Yeah, that's like I think that was one of the ones thinking of. sure.

48:20 KENNEDY: All right. So before you get out of here, You got the two final questions. If you're gonna write some Python code, what editor do use?

48:26 NEWTON: Yeah, so I'm VS Code person, and Okay, I I have had great experiences with both VIM AND emacs and those were kind of driven by pair programming. I'd like to be able to jump in and pear. And so I've learned for the people that I was typically pairing for and something that is a little easier barrier to entry. Like VS Code makes it much easier for me to pair we regardless of preference, usually.

48:51 KENNEDY: or whatever, if you're not familiar with it. Yeah, I mean, just go. All right, now exit

48:51 Exactly.

48:58 KENNEDY: And the tooling for Python for VS Code is solid.

48:59 NEWTON: Yeah, I've never had a complaint there.

49:02 KENNEDY: All right. Good one and then notable PYPI package.

49:04 NEWTON: Yeah. So notable in a couple ways for me. Is the docup package? Okay, so it's for sure, my most used library just because it takes scripts that I write programs that eye right up to the next level of usability on the command line. But the other thing that's incredibly notable to me is that it's like maybe a project that's reached enlightenment. It's been quiet for two plus years or something. But it's just stable.

49:28 KENNEDY: stay, but it's okay that that's happened. Yeah,

49:30 NEWTON: for sure. There's like warnings out there that you shouldn't use a package if it's been quiet for a certain amount of time if there's not enough activity. But I think there are some rare ones out there that are just stable.

49:46 KENNEDY: Awesome. Well, that's a very good one.

49:48 KENNEDY: but well, you know, I think should be on the show. People are excited about this. Maybe they want to go get some diagnostics, information the car or they want to try out some of the self driving AI magic. What would you say?

49:59 NEWTON: Yeah, I say jump in. ROS is a great place to start talking to a robot. Talking to a car isn't harder than file IO, if you could do that, you can talk to robots.

50:09 KENNEDY: That's pretty impressive. Awesome. Are you? Well, thank you for being here. And thanks for being the show.

50:13 NEWTON: Absolutely. Thanks for having me.

50:14 KENNEDY: Bye.

50:16 KENNEDY: This has been another episode of Talk Python to me. Our guest on this episode was Shae Newton. It's been brought to you by datadog and clubhouse

50:25 KENNEDY: Datadog gives you visibility into the whole system running your code. Visit talkptyhon.fm/datadog and see what you've been missing. They’ll throw in a free T shirt.

50:34 KENNEDY: Clubhouse is a fast and enjoyable project management platform that brakes down silos and brings teams together to ship value, not features. Fall in love with project planning. Visit talkpython.FM/clubhouse

50:47 KENNEDY: Want to Level up your Python? If you're just getting started, try my Python jump start by building 10 apps course. Or, if you're looking for something more advanced, check out our new async course that digs into all the different types of async programming you can do in Python. And, of course, if you're interested in more than one of these, be sure to check out our everything bundle. It's like a subscription that never expires. Be sure to subscribe to the show. Open your favorite pod catcher and search for Python. We should be right at the top. You can also find the iTunes feed at /iTunes. The Google play. Feed at /Play in the direct RSS at /RSS on, talkpython.fm. This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code