Choosing JupyterHub and Python over MATLAB

00:00 The Nobel Prize in Economics recently went to Paul Romer, a convert from proprietary software

00:05 like MATLAB, over to Python and the SciPy stack. Paul said, the more I learn about proprietary

00:10 software, the more I worry that objective truth might perish from the earth. That's quite a

00:15 statement, but what if your organization is deeply committed to proprietary software such as MATLAB?

00:20 Well, don't despair, because Peter Karzarnoff, a professor at Portland Community College,

00:25 is here to share his experience converting his courses over to Python and JupyterHub.

00:29 This is Talk Python to Me, episode 201, recorded January 23rd, 2019.

00:34 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the

00:52 ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter,

00:57 where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm,

01:01 and follow the show on Twitter via at Talk Python. This episode is brought to you by Linode and

01:07 Stellar S. Please check out what they're offering during their segments. It really helps support the

01:11 show. Peter, welcome to Talk Python.

01:13 Thanks, Michael. I'm so excited to be here.

01:15 Yeah, it's really excellent to have you on the show. You and I have spoken throughout the years

01:19 over Twitter and other mediums, and it's great to finally get to talk to you in more depth.

01:25 Yeah, it's great to connect. I can't wait to tell my story about choosing Python over MATLAB

01:29 and using JupyterHub in my classroom.

01:31 Yeah, absolutely. I think that is such a good choice, and it's happening more and more. There's

01:35 some really interesting quotes from the guy who won the Nobel Prize in economics.

01:43 I think that was Mathematica, Python over Mathematica. But basically, the knowledge of the free world was

01:52 in danger because of these proprietary systems, and Python and open source was the thing that would

01:59 keep it free and keep sharing the knowledge, which is pretty interesting. So definitely a trend you're

02:05 seeing around a lot of data science teams I worked with and stuff as well. All right, so we'll get to

02:10 that, and we'll dig into that, and it'll be fun. But let's start with your story. How did you get into

02:14 programming in Python?

02:15 I got involved in programming kind of late. I'm an adult, and I didn't get started until about three

02:21 years ago. Growing up, I wasn't a gamer, and at age eight, I wasn't like pounding away at like a

02:27 Commodore 64 in order to like write new programs. My dad, he was a biochemist. He wasn't a programmer,

02:33 and my mom, she did art stuff for the school district, and she wasn't a programmer either.

02:38 And like there were some kids in high school who like programmed their calculators to play games,

02:44 but I wasn't that kid. I played soccer, and I went camping and stuff like that.

02:48 That's cool. You know, that's probably a good way to grow up, honestly.

02:51 Yeah.

02:51 I was always kind of blown away with those people that would program like their HP calculators and

02:56 stuff because they didn't have a proper keyboard. It was like texting. Like remember when you'd have to

03:00 hit like one or two, maybe it was two, three times to get a C to come out. Like who could program that

03:06 way? But they did, right?

03:07 Yeah, but they like did it. But that wasn't me. So I did my graduate work at UW, and I worked on making

03:16 plastic transistors and plastic solar cells. It was really cool research, and that was kind of the

03:21 first time that I did some programming, and I used LabVIEW. Have you heard of LabVIEW before?

03:26 I've heard of LabVIEW. It's like this graphical programming interface, but it's...

03:31 Yeah.

03:31 Is it hardware focused?

03:33 Yeah, it's really just focused on hardware, and you kind of drag these wires like around the screen

03:38 in order to connect components. So I was just using that to interface with hardware and like measure

03:45 devices and collect data and stuff.

03:47 Cool. Tell me about this plastic hardware stuff that you were doing. Like,

03:51 did you find a way to make plastic that was conductive, or was it like wrapping chips, or what

03:57 was it?

03:57 We were in graduate school making N-type semiconductor plastic. So regular chips that are in your computer

04:04 or your phone, they have both P-type and N-type silicon in there, and that's what helps the

04:09 transistors shuttle electrons across. And we were making these specialty plastics that would shuttle

04:15 electrons when you applied what was called a gate voltage. So we were basically making like

04:20 these little transistors just out of plastic. It was really, really neat.

04:24 Wow, that's crazy. Could you 3D print those these days?

04:26 Probably. And we were thinking about maybe you could roll-to-roll print them, like you print a

04:31 newspaper. Maybe you could inkjet print them, like you're printing out a regular piece of paper.

04:36 But the research was just very fundamental, like trying to measure core electrical properties of

04:42 these plastics.

04:42 That's pretty excellent.

04:43 Yeah.

04:44 Okay, cool. So you had to learn this LabVIEW draggy-droppy programming language so that you could work on this

04:50 hardware, right? And I find a lot of people actually get into programming along these lines. They're like,

04:54 well, I didn't really think I'd be a programmer, but this thing I thought was really cool required this

04:59 step to like basically make it go. And maybe it's pretty small, but you know, there it was, right?

05:04 Right. Then I started working at Portland Community College, where I work now.

05:08 And I was digging through these things. They're called course content outline guides. They're kind

05:13 of like course descriptions that are up on the web. And I noticed that this course content outline guide,

05:20 this course description had the word his in it. And the phrase was, the student may work at his desk.

05:27 And I was like, this is not good. Like I've got two daughters. Like I want them to go into engineering.

05:32 I really like it. We shouldn't have like his in this description. So I started looking through all of the

05:39 other course pages that were up. And I quickly realized that there are way, way too many to do

05:44 by hand, right? Like we have over 1300 classes and I can't click on every single one and read through

05:50 every single one. So I was telling one of my colleagues about this and he was like, oh, you should,

05:56 you know, learn Python, like try this Python thing. And you could like write a little program that could

06:00 do it. And I was like, okay, I'll give it a try.

06:04 It sounds really hard, but okay.

06:06 It sounds hard, but I'll try. And so like, I found this package, beautiful soup. And like, I wrote this

06:11 little script and, it was probably about a week. And I got this thing where I scraped every single

06:17 one of these content guides, about 1300 classes and found like any time that there was a his,

06:24 he, or like him in them. And it was just like amazing that this thing that I was clicking through

06:29 individual links and like reading all the texts. Like I could just like type Python and then my

06:35 script.py. And then like, it all just came out into one CSV. It was like amazing.

06:40 Wow. That's pretty interesting. A couple of thoughts. One, I think that's definitely a noble effort

06:46 because we already have enough barriers to folks who are underrepresented getting to,

06:53 into STEM fields, into programming and stuff. Like we don't need to, to make it less comfortable.

06:57 Right. Right. So I think that's good. Is this thing open source? Like if somebody worked at

07:02 Johnson County community college or whatever, right? Like, like, could they borrow your thing

07:07 and turn it loose on theirs? Yeah. I mean, the code is up on GitHub. If somebody wants to try,

07:12 like this is my first Python project. So I hard coded in a lot of the URLs and stuff like that.

07:18 So it's not totally transferable right now, but all of those like gooey kicks that the Python

07:25 ecosystem has been talking about. Maybe I could turn it into one of those gooeyes and somebody

07:29 in another community college could just like click it and make it work for them. That would be pretty

07:34 cool. Yeah. That'd be totally cool. Like you put some gooey on a little CLI on it or like if somebody

07:38 wants, they could create a web app and then you just give it like the base URL of your college and

07:43 it'll go through and it'll do it. Right. Right. Give it, give it a little CSV download. That'd be sweet.

07:48 So my other non-technical question here though, it's like, I definitely agree that this is a good

07:53 thing to be aware of and to try to eliminate, but how do you eliminate it? I mean, like replacing every

08:00 him with a her seems like a step that is in like, it's the idea of it is, is right, but it also doesn't

08:07 seem right. Like, so to me, I don't know, this is an English question. Like this just bothers me in

08:12 English. There's not a way to speak about a person in a non-gendered way really. So I don't know. I

08:17 try to use they, them, their, but like obviously they're singular, but you know, like what would

08:23 you propose here? Like as part of this? For those, you could say at the student's desk or at their desk

08:29 or at a person's desk. You don't at all need to use any of those like gendered pronouns. Right. As long

08:35 as there's more than one student and there always will be, it'll be fine. Right. It's just like,

08:38 it's when it's singular. I'm like, man, I'm going to say they, and I know it's singular and

08:42 it's probably wrong, but to me it feels better than he, when it's a her or her, when it's a he.

08:48 And you know, it's just like, I don't know. It's just weird. We need a better bit of English here.

08:52 All right. So great. So that's how you got into programming. As you saw in the Python is how you

08:57 saw this problem. You're like, your friends, like, you know what? Like a little bit of requests,

09:01 a few foreign loops and some beautiful soup. And you got this problem solved. Actually,

09:06 you said you teach at Portland Community College here in lovely Portland, Oregon.

09:10 What do you do day to day? Like what kind of classes do you teach? Things like that.

09:14 I teach engineering at Portland Community College and we're a community college. So we teach the

09:19 first two years of a four-year engineering program. And we also teach two years of a two-year degree

09:26 in engineering. So some of the students go on to jobs and some of the students go on to four-year

09:32 universities. So I teach some of the intro classes. We call that ENGR 101. And I also teach the engineering

09:39 programming class. And then in the second year for engineering students, I teach some of the mechanical

09:45 classes like material science, manufacturing, and machine design.

09:49 Cool. Those sound interesting. First of all, maybe we should just define what a community college

09:54 is for folks who are not in the United States. I feel like this is kind of a U.S. type of thing.

09:59 Like in the U.S. we have four-year colleges and we have super expensive four-year colleges.

10:03 And then we have high school. And high school is especially bad in the U.S. in terms of preparing

10:09 people for a job. Like in Europe, I feel like maybe what they learn is not necessarily better. But

10:15 in the end, they either have a vocational track that they can sort of follow on, especially in Germany,

10:21 or like they go a little bit farther in high school so that they actually have a little more skills

10:27 when they come out. So in the U.S., like having a high school degree almost is like not having a

10:32 degree at all, right? So maybe just talk about that so people who are not familiar with this idea know

10:38 what it is. Yeah. So in the U.S., after high school, you could go to a regular four-year school,

10:44 you know, like the University of Michigan or Stanford or something. But they're also these community

10:48 colleges. And they're usually two-year programs. And they kind of have two separate aims.

10:55 So one of them is as a vocational school. So you could go after high school and we have,

11:00 say, an automotive department. You learn how to be an auto mechanic. Or we have a welding program.

11:05 You learn how to be a welder. Or we have like a pre-nursing program. So you learn how to be a

11:11 medical assistant. And then the other function that the community college is served here in the U.S.

11:17 is to prepare students and get them doing their first two years of college. And then they'll finish

11:23 their third and fourth year of college at a regular school. So it's both that kind of vocational school

11:28 as well as kind of like the first two years of what you consider regular college.

11:32 Yeah. Okay. Great. And it's also typically much, much cheaper than a four-year school.

11:39 Even a state four-year school. But like especially the other ones. So folks who go here,

11:43 maybe they're not academically ready for a four-year school. Maybe they're not entirely sure that that's

11:48 for them. So they're going to take a less intensive route to like find their way. The reason I'm bringing

11:54 this up is it's sort of relevant to the conversation later around like expensive proprietor software and

12:00 so on. Yeah. Yeah, exactly. So at PCC right now, I think our credit costs $111 a credit. So that's

12:08 kind of $1,500 a quarter or so. And if you look at private four-year colleges, you know,

12:14 they're at least $45,000 for a year now. So community colleges are much, much cheaper.

12:21 It's like a 30X difference or something. Yeah. Yeah. Yeah. Yeah. Just for people listening,

12:25 I actually went to a community college for my first couple of years out of high school and just thought

12:29 it was a fabulous experience. It went out to do well in other colleges afterwards. Right. So it's

12:35 definitely a nice place to be. I think it's got the right sort of low-key start. And the other thing,

12:41 I guess, that's interesting to me is that you're teaching engineering courses. When I knew about

12:46 community college, it was like you could take calculus and differential equations and chemistry

12:51 and physics, but there was not like a full-on engineer that was like to prepare you for other

12:56 engineering courses. These directly transfer to Portland State or to Oregon State. Some of the students

13:02 go to the University of Washington. So you take engineering classes, but our students also take

13:07 things like calculus and chemistry and physics and those courses too. Sure. Sounds fun. Kind of makes

13:13 me wish I was back in college, sort of. I love it. Like this is honestly like my dream job. Every day I

13:19 walk in, I'm like, oh, I love this place. I love this place. How cool. All right. So let's talk about

13:26 the computing software that you have for your class. So you're teaching these engineering and

13:31 engineering programming classes and programming labs and stuff. So traditionally, this is the realm of

13:37 things like MATLAB and Mathematica, which are super expensive things. They only exist on the computer labs

13:44 at the school because like no student is going to go, yeah, 1500 bucks for like a license. That seems like

13:51 a good idea. Oh, and then you've got to buy all the other add-ons, right? Like, yeah, it's really

13:56 the use of these tools while they're pretty good at solving engineering problems really means

14:01 like you're, you're sort of stuck there, right? Yeah. Yeah. So our college has taught MATLAB in this

14:08 programming course for about four years and it would cost students either $50 or $100 for the license.

14:15 And then it also costs the college. It's like an academic license or something they can get.

14:19 Yeah. Yeah. They get a student license and it says, I think, EDU, like at your command prompt in MATLAB,

14:25 just to let you know that you're still a student. And, another problem with that is after two years,

14:31 your license would expire and some students stay a little bit longer than two years. And so

14:36 they'd come to me before they go to four year and they're like, ah, Peter, like I got to reinstall

14:41 this new license. Can you get me another license to MATLAB? And it just felt like, like we were

14:48 choosing certain students to succeed because there's not MATLAB installed in the multicultural center or

14:54 MATLAB installed in the veterans resource center. Like it's just installed in these two computer labs and

15:01 then our engineering lab. And for $50, like that might not seem like a big deal if you're a big

15:07 programmer and you've got, you know, this big job at Google or Facebook, but for some community college

15:13 students where food security is an issue, $50, that can be a really, really big deal.

15:19 That can definitely be a big deal. I remember the shock of just the price of books.

15:23 Oh yeah.

15:24 Yeah. And the, and, and books in the course were over a hundred dollars. The book was awesome. It was

15:31 really, really good, but it was super expensive. And then after the course, what are you going to do

15:35 with that textbook again?

15:37 Try to sell it. You just try to sell it.

15:39 Right.

15:40 It's the whole college books thing, but I'm just blown away how expensive those are. Right. But that's one possible path. The other path is maybe something that's really good at computation. That's kind of like MATLAB.

15:53 Right.

15:54 That's free and more accessible, right?

15:56 Right. That's free, more accessible. So that's why we started investigating Python because we were like, okay, we can do these arrays in MATLAB, but Python has NumPy. We can do numerical computing.

16:09 Or okay, MATLAB, we can build these plots in MATLAB, but Python, we have matplotlib. Or in MATLAB, there's this serial module where you can talk to hardware. Oh, well, in Python, we've got PySerial.

16:22 Exactly.

16:23 So I sort of started investigating like each time I was doing something in MATLAB, like, can I do this in Python? And the answer was always yes. And sometimes, especially when we were doing like HTTP requests or like trying to get data from like a web

16:39 API for like tie data or something you're going to plot, it is just so much easier in Python to do that than it was in MATLAB. Like it just felt like we were having to do these backflips whenever we were trying to like read anything that was external to like the MATLAB sort of ecosystem. And so we decided to make the jump to teach Python in the course instead of teaching MATLAB.

17:01 That's really cool. Now, all those reasons make tons of sense to me. But I can just see, you know, the 59 year old, almost retired engineering professor going, you know, MATLAB's worked pretty well. And I've got six years left, and I'm not a programmer.

17:20 Right.

17:21 As a department, we're not doing this. How did that go?

17:23 There's still like some open disagreement, I'd say about it. And there's some are some benefits to MATLAB, like the installation story is really clear. And there's only one IDE, like you use MATLAB, and that's it. And there's only one way to write the code, run the code, which is use MATLAB. Like those are good arguments. But I don't know, community colleges are about like innovation.

17:46 And doing quick iteration on courses, like we can change the way that we teach courses so much faster than four year universities can. So I just felt like it was an opportunity for the community college to like, lead in this area instead of just follow what other courses were doing. And now maybe down the line, when the students go in, they'll be like, yeah, we can do all of this in Python. Like, can I do my third year, like numerical methods class in Python? Like, maybe it'll bleed over the other way.

18:15 Yeah, I bet that it will actually, I bet that you'll have these experiences where people learn Python here. And then they're like, you really want me to use that other thing? Like, I know, we could do it so much better with x, y, and z package, right? Like, right, it's probably gonna, you know, plant some interesting seeds. But there's some other things to consider, I guess, like, what were some of the things you did to decide on Python?

18:40 I mean, obviously, you're personally a fan of Python, you know, is pretty well, but there's some other academic examples, right?

18:45 I wanted to make sure that it would not only be useful for students, but that like real engineers use Python. So like the Large Hadron Collider, like they use Python, or like the Allen Brain Science Institute, they use Python, or Instagram, or YouTube, Facebook, like they use Python. So like real engineers are using Python.

19:08 And I actually just went and toured a company last week, that makes pacemakers here in Portland. And that company uses Python, too.

19:17 Oh, how interesting.

19:18 There are real engineers out there using Python, like every day. So I think it is quite applicable. Plus, not every single engineer is going to end up and every student student in engineering doesn't always end up being an engineer, you know, they might be a teacher, they might end up working in a medical office, I don't know. And having a general purpose language like Python, is a lot more useful than having, you know, MATLAB, like if you work in a medical office, what are you going to do with that? I don't know.

19:45 Exactly. I do think it's interesting to consider like what you do outside of this discipline. If you have the skills to write something in Python, that in and of itself is a very marketable thing. You know what I mean? Like if you're like, man, I can crush it in LabVIEW. People are like, great, you know, Excel, right? Like, it's not going to really translate, right? Unless you're working in hardware or something like that, right? Like if it's, if you want to move out, I think it's really, really interesting.

20:14 This portion of Talk Python To Me is brought to you by Linode. Are you looking for hosting that's fast, simple, and incredibly affordable? Well, look past that bookstore and check out Linode at talkpython.fm/Linode. That's L-I-N-O-D-E. Plans start at just $5 a month for a dedicated server with a gig of RAM.

20:32 They have 10 data centers across the globe. So no matter where you are or where your users are, there's a data center for you. Whether you want to run a Python web app, host a private Git server, or just a file server, you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network, 24-7 friendly support, even on holidays, and a seven-day money-back guarantee.

20:52 Need a little help with your infrastructure? They even offer professional services to help you with architecture, migrations, and more. Do you want a dedicated server for free for the next four months? Just visit talkpython.fm/Linode.

21:06 Yeah, and of course, Paul Romer and his collaborator won the Nobel Prize in economics using Jupyter notebooks and stuff, which is pretty trick. So that's awesome. And then also, this is partly why I opened the conversation this way. It's like, when you think of how much that costs, right? Maybe it doesn't cost an insane amount of money to run MATLAB.

21:25 But when you look at trying to keep tuition as low as possible to make community colleges for the community, right? Things like that. So the price of not paying for MATLAB, it can't hurt, right?

21:37 Yeah, and it can't hurt the colleges' bottom line either. They've got to pay for MATLAB. And then also then the textbooks for MATLAB, those are closed source too. So if you can move to Python, an open source free language, you can move to an open education resources textbook, like a free textbook. And then the college doesn't have to pay for the MATLAB license.

21:58 Like we calculated, that's about $30,000 in one year of savings just by moving to Python if we do that in the first year course and then this engineering programming course. So it's like little bits, but like those little bits really do add up.

22:14 They do. That's cool. I mean, maybe you could almost offer like another course that pay an instructor to offer a course that otherwise wouldn't be available rather than sending that over to MATLAB, right?

22:24 Right. Or how many students could you give scholarships for that at a community college? That's a lot of students, right?

22:29 That'd be actually excellent. Yeah, that's great. Maybe talk about some of the road bumps. Like what were some of the challenges?

22:34 There were a lot of road bumps. So initially installation, that was a big road bump because we work at a big institution. So Portland Community College is actually the biggest college in Oregon. There are around 70,000 students, depending on how you count.

22:49 So that's bigger than the U of O, University of Oregon, bigger than Oregon State. And so doing things like changing infrastructure, that's something that happens fairly slowly.

23:00 And we wanted to make sure that we had Python installed when the quarter starts so that the students could use it. So that was one of the big roadblocks.

23:08 Yeah, yeah, that's a huge challenge.

23:10 When you're talking about those instructors, like you've been doing it for so many years and they only have a little bit to retire, like that is a huge barrier.

23:18 Like telling the administration or like the IT people, you need to install this piece of software on every computer on campus.

23:24 Like that can be a real non-starter at some places.

23:28 Yeah, I can imagine.

23:29 What kind of hardware and OSs do you use? Is it mostly Windows? Do you have like Mac machines? What do you got?

23:35 We got all Windows machines and...

23:37 Windows 10?

23:38 Yeah, Windows 10 at the college. And the student machines, students, I've like done surveys and students overwhelmingly use Windows. There might be one student that uses a Mac or there might be zero students who use a Mac.

23:52 How interesting.

23:53 In the class, yeah.

23:54 I mean, you look at the price of a Mac versus like a decent Windows laptop, though. It's like three to four times the price, right? So it makes sense.

24:02 Right. And also some students use either Chromebooks because they're even cheaper or they use tablets. And so the installation story of Python on a Chromebook or Python on a tablet, that's not great right now either.

24:15 Hey, I saw the Apple ad. It told me my iPad can replace my computer. I got this keyboard. Let's go. It's like, oh, you're an engineering student. What are you thinking, man?

24:23 Right, right, right. I'll probably write a little blog post about it. I got Python going like on my Chromebook, but it's not super easy.

24:31 And if you're like a student starting out learning programming, that's not going to be something that you can do.

24:36 No, that's definitely a challenge. Yeah.

24:38 Yeah. And then like the other challenge just was textbook. So in the course with MATLAB, we had a textbook from a regular big textbook manufacturer and it had end of review problems, you know, and it had example questions and stuff like that.

24:53 Everything an instructor would want in a textbook.

24:54 And there are awesome, like great Python books out there, but very few of them have homework questions at the end.

25:02 That was like something big that we were missing.

25:04 And also we wanted to make sure that if we were moving to this free programming language, that the textbook would also be free, that we wouldn't like offset the cost of the software by like having a $300 book.

25:16 Yeah. Now Wiley's rich instead of MATLAB, whatever.

25:19 Right, right. Exactly.

25:20 I would say like installation and then figuring out like textbook and then course content.

25:25 Those were kind of like the big roadblocks to adopting Python here at the community college.

25:30 That makes a lot of sense.

25:31 So we're going to talk about JupyterHub, which is a solution that you settled on.

25:35 But just a few other things that come to mind as I think about this is there's coder.com, which is Visual Studio Code plus Docker container in your browser.

25:43 So that would work on like a tablet or a Chromebook.

25:46 There's Python Anywhere, which is pretty awesome.

25:49 So there are a few solutions, right?

25:52 But they're either brand new and didn't exist a year or two ago when you were doing this, or they're maybe not quite as comprehensive.

25:58 Like I don't know if Python Anywhere does any like data science-y stuff.

26:02 That's easy for you.

26:03 I just have no awareness of that.

26:05 So maybe let's talk about JupyterHub before we move off the student story.

26:09 I want to ask one more question, right?

26:11 So this is, I guess, programming for engineering, but it sounds like a lot of them have to take this course.

26:17 I suspect there's a little bit more of a challenge to write Python and run Python than there is to write MATLAB.

26:24 I'm not 100% sure.

26:26 I've done enough MATLAB that I felt like you could just start typing in like the library.

26:30 There's no import this or important things that you had to use.

26:34 It's a little more formal to do Python.

26:36 So how is that received by the students?

26:38 We're like, yes, we get to learn a real programming language.

26:40 Or was it like, oh, are you kidding me?

26:42 Now this is even harder.

26:43 I guess I'd say it was the wealth of options was maybe the difficult thing for students.

26:49 Because like you were saying, you could run Python on Python Anywhere.

26:52 There's also Replit.

26:54 There's also Python Tutor.

26:56 There's also like Google Colab.

26:58 Like there are all these different ways.

27:00 And one thing that intro students I think crave is just like, tell me how to do it.

27:06 Like you pick the tool, just tell me how to do it.

27:09 And so what we decided on was to use JupyterHub in my class.

27:14 And so that sort of settled that.

27:17 You don't have to like install Python.

27:20 And you don't have to like go to any of these other sites and use their interface.

27:24 Like we're just going to be using JupyterHub.

27:27 That's what we decided on.

27:28 Okay.

27:28 Yeah, that makes a lot of sense.

27:29 It's basically Jupyter Notebooks hosted.

27:33 It's a server for hosting Jupyter Notebooks on a central server.

27:36 So did you get like a super nice Linux machine?

27:40 Lots of cores so they can go do math on it?

27:43 Well, sort of.

27:45 Did it actually not need it?

27:46 Actually, my laptop is plugged in here.

27:48 Right, right.

27:50 So let's like back up just a little bit.

27:52 So to make sure that everybody knows what a Jupyter Notebook is.

27:56 So Jupyter Notebook is a way to write and run Python code.

27:59 And it also contains things that are called markdown cells.

28:03 So you can also write text in them.

28:04 So they're this great way for writing, running Python code, and then explaining what the code does,

28:10 which is super useful when you're teaching an engineering class to have both of those things available.

28:15 Explain what the code does and then write the code.

28:18 And these Jupyter Notebooks, they just run in web browsers.

28:21 So I was thinking when we were doing the course, like, well, if it can just run in a web browser,

28:28 then maybe a student can like just open up a web browser and run it.

28:32 Like they won't need to install Python in their computer.

28:35 And so I was like Googling around and I found like Jupyter Hub.

28:38 Like it exists.

28:39 Like Project Jupyter created this.

28:41 And I was like, awesome.

28:42 This is exactly what I want.

28:44 Yeah, that's cool.

28:45 That's great.

28:46 So I installed it on a virtual machine on DigitalOcean.

28:50 And what I found was the very first quarter that I did this, it was in a hybrid class.

28:57 And that means some of the class is in person, some of the class is online.

29:01 And when they did the computing labs that were online, the students would log in at different times.

29:07 Not everybody tries to log in at exactly the same time.

29:11 When it's just totally online.

29:12 And just a real small virtual machine, like a $10 a month virtual machine worked just fine for that.

29:19 And the first day that we went into the class that was all in person and everybody tried to log in at exactly the same time,

29:29 that kind of crawled the server like a little bit and it took a little while.

29:33 So sort of what I'm realizing now is I either have to like up the size of the server that I've got or the JupyterHub documentation talks about using Kubernetes and deploying Docker containers instead of running it all on one individual server.

29:48 But totally thanks to Talk Python.

29:50 Like I used one of the credits from DigitalOcean.

29:53 That's how I ran that first course.

29:54 Wow, that's awesome.

29:55 Yeah, how cool.

29:57 So I was actually able to like get a bigger server than the one that I currently have now because of that credit.

30:02 Yeah, you know, it's just amazing what you can do with computational resources and virtual machines these days.

30:09 When I was in, when I was getting my math degree at four-year college, the last year we got this computing, like shared computing server.

30:17 It was like a used computer they had bought secondhand from some other place, right?

30:22 It was a Silicon Graphics, big mainframe type of thing.

30:26 And they paid a quarter million dollars for this used computer.

30:30 And, you know, like seven or eight people could log into it and do stuff.

30:34 And it was kind of okay.

30:36 And then here you are running like all that on, you know, 10 bucks.

30:39 It's really impressive.

30:41 All right.

30:41 Yeah, yeah.

30:42 It's so cool.

30:43 So, you know, I think there's a lot of advantages to having these notebooks.

30:48 I mean, certainly you have JupyterHub and run them there.

30:50 Like they also, if you put them into GitHub, GitHub like will execute them and actually render them live.

30:57 And there's NB Viewer and there's Binder.

30:59 There's all sorts of stuff, right?

31:00 Yeah.

31:01 And Google Colab will now run them too.

31:04 So there are just a lot, a lot of ways that even without starting a virtual machine that you can either view or interact with Jupyter Notebooks.

31:14 It's a pretty amazing technology.

31:15 And then one of the other things about it is Jupyter stands for Julia, Python, and R.

31:22 So you don't even have to run just Python in Jupyter Notebooks.

31:26 Like you could also run the R programming language or you could run Julia or there are a bunch of different backends that you can plug into it too.

31:34 And then there are like these extensions too.

31:37 So I found this really neat extension that you could plug in and it allowed you to do draw.io flowcharts like right in the JupyterLab interface.

31:49 So students can like draw the flowcharts that describe, you know, their loops or their if statements right there in the JupyterHub interface, which is pretty cool too.

31:58 That's really awesome.

31:59 It sounds like this solves, like that virtual machine you went and created, it sounds like that solves it for your instance of the class.

32:06 But I can see as a college, as a department going, okay, well, we can't always just, you know, call Peter the week before and have him use some credits to get a VM running mic.

32:16 What's the long-term plan there?

32:20 Well, so one of the reasons that I set up JupyterHub was I was worried that the college wouldn't install the software before the quarter started.

32:28 And it also gave the students like a way to log on and like run Python before the class even started.

32:35 And I think for some students, that's really nice to like just get their feet wet and be like, oh, this is what we're doing.

32:41 And like, oh, this isn't too hard.

32:44 And they can't come in and use the lab computers during like winter break or whatever.

32:48 So it did turn out that the Anaconda distribution of Python did get installed on the college computers.

32:54 So now at the college, I have the students open Jupyter Notebooks using either the Anaconda prompt or just using the Windows Start menu and opening a Jupyter Notebook that way.

33:04 And at least for us in engineering, like the Anaconda distribution is super helpful.

33:11 It has Matplotlib.

33:13 It has NumPy.

33:14 It has Pandas.

33:15 It's got PyCereal.

33:17 Like all the packages that you would want to install, all of those are already included.

33:22 And doing like pip install installations and virtual environments.

33:26 Like if you've been doing it for a while, that sort of seems like, oh, yeah, that's just easy.

33:30 You just create a new virtual environment.

33:32 For a student, it's really kind of complicated and opaque.

33:35 Like, what is this?

33:37 Yeah.

33:37 Like, I just want to open a file and make it run.

33:39 Like, why am I doing this?

33:40 Right.

33:40 Right.

33:41 And so using Anaconda, then the students don't have to install anything.

33:45 Like, you import NumPy and MP and it just works.

33:48 So are they generally using their local computers these days?

33:51 Generally, they're using in the classroom their local computers.

33:55 But I actually just like logged on to the JupyterHub server about half an hour ago.

34:00 And there were like a couple of students that had logged on.

34:03 So students are still using it.

34:05 Yeah, that's pretty awesome.

34:06 So earlier you spoke about this challenge of the textbooks.

34:12 And, you know, like I said, I remember the textbooks just being like a huge expense that I didn't see coming.

34:18 Like I saw tuition coming.

34:19 I didn't see $500 a semester in books also.

34:23 Like, geez, where am I going to get that money from as a kid, you know?

34:26 Right, right.

34:26 And there are, like you said, a bunch of great Python books that people can just sort of colloquially learn Python.

34:33 Like you read, it's interesting, right?

34:35 But not in the formal school sense, right?

34:39 Like chapter three is going to have this test.

34:41 And here's some example exercises like it.

34:43 So you're working on a book for this, right?

34:46 Along with making JupyterHub go so the students could see how to run Python before the quarter started.

34:52 I also wrote this open source book so that the students could read the book before the quarter started too.

34:59 And I was kind of inspired.

35:01 Like Jake Vander Plaas, he wrote the Python Data Science Handbook.

35:05 And that was written in Jupyter Notebooks.

35:07 And there are a couple of other books that were written in Jupyter Notebooks.

35:10 And I was like, I don't want to both like write the code and then I have to like test all the code.

35:16 And then like 3.7 is going to come out and then I have to retest all the code.

35:20 Like why not just write the whole thing in a Jupyter Notebook?

35:23 And then like the code that produces the plot, that's the plot that's like right there in the book.

35:29 So I wrote the book in Jupyter Notebooks and then students could view it in NB Viewer or go on to Binder and run them themselves.

35:38 And then I also used a package called mkdocs to build a web version of the book so that the students can just go to a website and see what their homework is and read through the material.

35:50 Yeah, this is really nice.

35:51 So problem solving with python.com, right?

35:53 Yeah, that's the textbook website.

35:56 And it just got published on Amazon.

35:58 So if you want a hard copy, that'll cost a little bit of money.

36:01 But all the students can use the web version or look at all the notebooks for free.

36:06 And some other authors have sort of adopted that same kind of idea that if you get the hard copy, you have to pay a little bit of money.

36:13 But if you're going to use the PDF or like use the web version, that's free.

36:17 And that's what I did it for my book too.

36:19 Oh, yeah.

36:19 It looks really interesting.

36:21 I like the exercises seem cool.

36:23 The review questions seem cool.

36:24 You have a Kindle version, which is maybe some weird intermediate thing where, you know, I really love the Kindle because you can take notes and highlights, right?

36:32 Like the web pages are great, but it's super hard to use it as reference, you know?

36:36 Yeah, and it's sometimes nice to be able to read the Kindle in bed or you're like on the bus.

36:40 You know, it's nice to be able to have that version too.

36:43 But part of writing the book, big challenge was how do you output to all of these different formats?

36:50 Like you have to create a website version.

36:52 You have to create the print version.

36:55 You have to create a PDF.

36:57 You have to create the Kindle version.

36:58 And I don't want to write four different books all at the same time.

37:02 So Project Jupyter, they've got this nbconvert package and that'll allow you to export Jupyter notebooks to various other file formats.

37:12 And lucky if you know Python, then that can help you with some of this stuff to be able to get the web version up and get a separate PDF made and get the separate Kindle version made.

37:23 So I'm just really, really thankful for all the people at Project Jupyter and all the hard work that they did.

37:28 That's awesome.

37:29 So the source of the book is a series of Jupyter notebooks?

37:32 Yep.

37:33 What are more?

37:33 Many.

37:34 Okay.

37:35 Yeah, yeah.

37:35 Many.

37:35 So there's sort of like one big directory and in there is a couple subdirectories for each chapter.

37:41 And in each chapter, there are between, I don't know, six or ten books.

37:45 So there's a notebook for each section of the book.

37:48 And then those all get put together to build a website.

37:51 Another real benefit that I didn't realize until students started using it was that you can make edits on the fly really, really quickly.

37:59 Like I can just, you know, pull the GitHub repo down and start editing one of the notebooks and then save it and push it back up.

38:06 And then the students like version is changed.

38:08 So having like a class of students is a great way to see where all like your little typos, grammatical mistakes, what's confusing.

38:16 Like it's probably like the awesomest editing system.

38:19 I don't really mean to use the students as editors, but they've just been great, like sending me suggestions to improve the book.

38:25 Well, the normal iterative cycle on textbooks is years, right?

38:30 Right.

38:30 It's not near live.

38:32 So that's a pretty cool aspect.

38:35 This portion of Talk Python to Me is brought to you by StellarS, the AI-powered talent agent for top tech talent.

38:43 Hate your job or feeling just kind of meh about it?

38:46 StellarS will help you find a new job you'll actually be excited to go to.

38:52 StellarS knows that a job is much more than just how it sounds in a job description.

38:55 So they built their AI-powered talent agent to help you find the ideal job.

38:59 StellarS does all the work and screening for you, scouting out the best companies and roles and introducing you to opportunities outside your network that you wouldn't have otherwise found.

39:10 Combining deep AI matching with human support, StellarS pairs things down to a maximum of five opportunities that tightly match your goals, like compensation, work-life balance, working on products you're passionate about, and team chemistry.

39:22 They then facilitate warm intros, and there's never any pressure, just opportunities to explore what's out there.

39:29 To get started and find a job that's just right for you, visit talkpython.fm/StellarS.

39:35 That's talkpython.fm/S-T-E-L-L-A-R-E-S.

39:40 Or just click the link in your show notes in your podcast player.

39:45 Do you encourage them to use GitHub issues and PRs and things like that, or do you just say, hey, send me a note that there's something wrong here?

39:52 Yeah, so far with the students, I've just said, hey, send me a note.

39:55 And if anyone wants to send me a PR, and absolutely, if anybody wants to use the book in their class, please do so.

40:02 So far in the course, we haven't gotten into Git and version control and how to do those sorts of things.

40:09 So I thought it was a little bit much to ask.

40:12 And students, it's easy for them to send me an email.

40:14 So so far, that's how it is.

40:16 But if anybody else reads the book and wants to make changes to it or wants to use one chapter but not use the other chapters,

40:23 they can definitely just go to the GitHub repo and pull it down and use what they need.

40:28 Interesting.

40:28 Maybe add an appendix for something or other.

40:31 Yeah.

40:31 And I took this idea from Allen Downey.

40:33 He's wrote a couple Python books, like the Think Python book.

40:36 At the end of his book, he had this list of contributors.

40:40 So every time I think somebody sent him something, then he put it in a list.

40:44 So I'll start listing anybody at the back of the book and the contributors if anybody sends me any edits.

40:49 Yeah.

40:49 OK.

40:49 Well, that's a great idea.

40:51 Yeah.

40:51 So people should check that out.

40:52 It seems like a really nice resource.

40:54 Even you think it's good for students who just want to learn Python and have they're not in like your ENG 101 course?

41:02 So if you're not an engineering major and you're not going to go on to a four-year engineering degree, I think it's still a really good intro resource because it's designed for students who don't have any experience in programming.

41:14 And also, if you're into data science and you want to get into data science, the tools that engineers use are a lot like the tools that data scientists use.

41:23 So in the book, there's a whole chapter on NumPy.

41:26 There's a chapter on Matplotlib and building plots.

41:29 Pandas and data frames are used.

41:32 And then some other packages like PySerial for communicating with hardware.

41:36 Also, SymPy for doing kind of symbolic math, for doing math with Python.

41:41 That's also in the book.

41:43 So if you're into data science, it could also be like a good resource for you, I think.

41:48 Yeah, I think it would be.

41:49 When you first told me about it, I was thinking it's mostly an engineering book that has some Python in it.

41:54 But it's mostly a programming book where the examples and the intended audience are engineers, right?

42:01 Yeah, exactly.

42:01 Yeah, that seems to me like that would be really useful.

42:03 When I was in engineering, I did like one year in chemical engineering before I decided I didn't want to be in chemical engineering.

42:11 And they told me during this period, I said, I would love to take a programming course.

42:15 Can I take like a C++ course?

42:16 They said, look, that can be an elective.

42:19 But you first have to take Fortran.

42:22 That's going to be the most important computer course you ever take in your life.

42:25 And you've got to study that.

42:26 And then you're allowed to do an elective C++ course, right?

42:29 It's like mid-90s or something.

42:30 And I just, I would love to have a book like this that covered a modern language that wasn't already old in 1995.

42:40 Yeah, and hopefully, hopefully, whatever the students learn in my course right now, and then in another 20 years, hopefully that will still be useful.

42:48 We'll be at Python, what, version 10 by that point or something, right?

42:51 Yeah, exactly.

42:52 I've heard talk of Python 4.

42:54 I have no idea what that's going to mean.

42:55 We're working on the versions as a community, I believe.

42:59 Oh, I guess it's worth asking, right?

43:02 This is Python 3?

43:03 This is Python 3.6.

43:05 And I call the book the 3.6 edition.

43:08 And I'm working on, for the spring, the 3.7 edition.

43:12 And then I'll change my virtual environment that I run all the notebooks in, make sure it all runs with 3.7.

43:18 The new f-strings, I think that that is a killer feature in Python.

43:23 And so the book does use f-strings.

43:25 It just makes inserting numbers and other strings into strings so, so much easier.

43:31 So it's definitely Python 3 only.

43:33 Perfect.

43:34 It's time, right?

43:35 Yeah, it's time.

43:36 It's been time for a while, but it's really time now.

43:38 I see you have a chapter on MicroPython.

43:41 Yeah.

43:42 And also external hardware.

43:43 That's pretty cool.

43:44 What do you guys do with that?

43:45 With external hardware, we connect the computers to little Arduinos, little microcontrollers.

43:51 Students can control LEDs or they can read temperature sensors, things like that.

43:56 And we used to use MATLAB to do that, and it was pretty cumbersome.

44:00 But the PySerial package allows you to communicate with external hardware.

44:06 So Python talks to these little Arduinos, these little microcontroller boards through the

44:12 serial interface using PySerial.

44:14 And the students each quarter do these projects with these little microcontrollers.

44:20 So, for example, one student group made a project that opened a set of doors that allowed this

44:27 car to run down a track.

44:29 Or another student group had a set of photo gates, and they measured how fast the car ran.

44:34 Or another student group put a light sensor on this aquaponics fish tank grow system and measured

44:42 the amount of light that the plants were getting.

44:44 And it sort of ended up being that the students would have to learn two different programming

44:50 languages.

44:50 They'd have to use MATLAB, and then they'd have to learn the Arduino programming language,

44:56 which was just sort of like C to do their projects.

44:59 But Python has MicroPython.

45:01 So you don't have to learn Arduino.

45:04 You can run Python on the little microcontrollers.

45:07 It's awesome.

45:08 It's so amazing.

45:09 Yeah.

45:09 This quarter, the students will use those little microcontrollers and run MicroPython on them.

45:15 And then they can use most of the regular Python commands that they learned throughout the rest

45:18 of the course.

45:19 Yeah, that's really interesting.

45:20 Because going from MATLAB to C, that's a big step.

45:22 Definitely.

45:22 But hopefully going from like Python to MicroPython, like that won't be as huge of a jump.

45:28 And I'm also looking forward just to see the things that the students are going to be able

45:33 to do with them.

45:34 Like they always come up with these awesome, awesome projects.

45:37 One student group had a coffee cup and the temperature thermometer.

45:41 And their idea was they were going to have it like send you a text message when your coffee

45:45 got cold.

45:46 Like I'm just super excited to see what they turn out to be this quarter.

45:49 That's cool.

45:50 You've got five minutes to finish your coffee before you won't like it anymore.

45:52 Go.

45:53 Yeah.

45:54 Oh, how cool.

45:55 I can certainly see this IoT device programming aspect of being an engineer really being useful,

46:02 right?

46:02 Like engineers build things, they have sensors, right?

46:05 Understanding like the control mechanisms seems great.

46:08 Engineers solve problems in teams.

46:10 That's what I always tell my students.

46:12 And so like they use computers to solve problems, not just to design computer stuff.

46:19 So we try in the course to like make them solve a real problem.

46:22 And MicroPython and like those little teeny microcontrollers, like you can solve real problems

46:27 with programming.

46:28 It's really neat.

46:29 Yeah.

46:29 A real in the physical world with atoms, not bits, right?

46:32 Yeah, exactly.

46:33 Exactly.

46:34 Yeah.

46:34 Cool.

46:35 I guess one other follow-up thing I'd like to ask you about this whole JupyterHub and

46:40 Jupyter Notebooks is what tools or advice do you have for people who may want to use JupyterHub as

46:47 their like homework, student work area, right?

46:51 Do you use things like NB Grader or do you use like is there, what advice do you have for

46:56 people I guess?

46:56 Well, so if an instructor wants to do this, I guess one of the big pieces of advice is you

47:01 have to think about authentication early.

47:04 Like how are the students going to log on?

47:07 Because you could have them all get GitHub usernames and passwords and do it that way.

47:12 Our college uses Google apps like Gmail and stuff.

47:15 So you could use like your Google logins or you could create individual usernames and passwords

47:21 that you like mail out to individual people.

47:24 So like thinking about authentication, I think is like one of the things early on that you should

47:30 think about.

47:30 And then another one is just data persistence or like state persistence.

47:36 So when the JupyterHub starts, it creates these new servers and there are different ways to

47:42 run it.

47:43 If you run it in a Docker container, it's possible that all of your work, once you close down your

47:49 server, like you close your web browser, all that goes away.

47:52 So that's the way that Binder works is that you can run these notebooks.

47:55 But if you save like homework number one and you close it, like and you come back later,

48:00 it's gone.

48:01 That'd be a little frustrating.

48:02 Yeah, that would be totally frustrating.

48:03 So you can't quite do like a whole course in that.

48:06 So you have to think about how are you going to allow data to persist on the server if users

48:11 are creating it.

48:13 Another little concern of mine was all of the labs and the assignments are on GitHub.

48:17 So how can I pull all of the labs and the assignments into the student's environment so

48:24 they don't have to go to GitHub and look at them.

48:27 They're like right there in their Jupyter notebook when they're going to the

48:30 they log into JupyterHub.

48:31 And there's this neat little extension called nb-git-puller, I think.

48:37 And this nb-git-puller extension, you click a link and then it automatically like pulls down

48:43 a repo from GitHub and sticks it into one of the JupyterLab or JupyterHub instances.

48:50 So that was like a really neat package to find.

48:53 Oh, yeah.

48:53 That is pretty cool.

48:55 A one-way git repository access.

48:58 That's nice.

48:59 Also, how did you deal with like, did you set it up so students each had their own private

49:04 set of things but you could see them?

49:07 Because obviously if one student can see another is like that takes like copy and homework to

49:12 another level, right?

49:13 Yeah, totally.

49:14 There are different ways to set up JupyterHub.

49:17 But the way that this one is set up is that when a student logs in, they basically get a new user

49:23 on a virtual private server on the Linux server.

49:26 And so their user on that Linux server doesn't have access to really anything outside their

49:33 own home folder.

49:34 So each one of the students has a different Linux user.

49:37 They have a different home folder.

49:38 And so they can't really see anything outside of just their local like user profile.

49:46 Yeah, OK.

49:47 And if a student like is super, super clever, like actually JupyterLab has command line available

49:53 and they're like able to like hack into the server, send me an email and we'll talk about

49:58 it because you're like way, way above like where a lot of students are if you're able to do that.

50:03 And maybe it's OK if you're copying somebody's homework.

50:05 Yeah.

50:05 If you figure out how to hack the server.

50:07 Yeah, interesting.

50:09 All right.

50:09 Yeah.

50:09 But it sounds like you could use Linux permissions and roles and whatnot to make it work.

50:13 Yeah, yeah.

50:14 That's one way to do it.

50:15 Then doing the Docker containers and Kubernetes, that's like another way to do it.

50:19 That's pretty cool.

50:20 It sounds like a fun project.

50:21 Yeah, it's super awesome.

50:23 And I built a set of documentation because as I was building the JupyterHub server, I wanted

50:29 to make sure like I can do this again the next quarter.

50:32 So I have a little website built with MKDocs that shows every step that I went through in

50:38 order to build my JupyterHub deployment.

50:40 So if anybody wants to build their own JupyterHub server, they can definitely take a look at that

50:46 online and see if they can build their own.

50:49 It's actually it's not too hard.

50:51 Like you need to do a little bit of Linux stuff and you've got to start the virtual private

50:55 server.

50:55 But like you can do it if you're teaching engineering programming at college.

50:59 Like you can launch JupyterHub.

51:00 You can do it.

51:01 Yeah, that's certainly true.

51:03 Cool.

51:04 Yeah, that probably about rounds it out for our main conversation.

51:07 You're trying to turn the tables on me here.

51:09 I see you threw out some stuff that say you want to ask me a couple of questions.

51:12 So I don't know if I have a good answers, but I'm game.

51:16 Go ahead.

51:16 All right.

51:17 So one of my big questions is I'm just so grateful to everybody at Project Jupyter and

51:25 Min Regan Kelly and Carol Willing and everybody who works even at NumFocus to help build like

51:31 NumPy and Pandas and all of the tools that we use.

51:35 What is a great way to thank people that build some of these open source tools?

51:41 I'm not at the level where I can like directly contribute to the NumPy source code.

51:46 Like what is a good way to thank people and say thank you for what you did?

51:50 Obvious way might be to do like add a new feature or something, right?

51:53 But that's a different level maybe.

51:56 I think starting just by saying thank you actually is really valuable.

52:00 So much of the open source stuff I think is thankless, right?

52:03 People will get angry because some feature doesn't work and they'll go rail against the maintainers.

52:09 And like just having a counterbalance like all these people are appreciated.

52:13 Maybe sharing your story, which obviously you're doing here, right?

52:17 But as a more general comment, like your work has really enabled me to teach, make Python

52:23 and engineering more accessible to more folks, right?

52:25 Like hearing that I think is a lot.

52:28 And then if you're in an organization that can somehow contribute back to it, you know, I

52:35 mean, I realize that a lot of people aren't.

52:37 But if you work, say, like at a big company, you know, maybe your company could somehow sponsor

52:41 that project to help, you know, make it not purely volunteer, but help make that a job.

52:47 One of my pet peeves is there are many, many billion, billion dollar companies built upon things like

52:55 Pandas, Django, Flask, SQLAlchemy.

53:00 And if those companies just sent back 0.01% of their profit, even not just revenue profit, just to say, look, we probably should maybe support the thing that is the foundation of our business a little bit like that would just there's like a handful of people like I probably could count them on my hand.

53:20 Literally, that maintain Flask, right?

53:22 Like, how much do these giant companies need to contribute to actually just make those people's lives great to just sit around and contribute to Flask, right?

53:31 So I feel like if that was if that's something that people could do, that would be amazing.

53:36 But I think just saying thank you, maybe contribute a tutorial, maybe contribute a case study, or, you know, it doesn't have to be a new feature, right?

53:45 It could be even documentation or, hey, let me I see your docs are on Python 2 for your tutorials.

53:51 Can I upgrade that to Python 3?

53:52 Because I know that's, you know, just like little because that's the last thing that folks that are trying to write the software want to do is like rewrite their tutorials.

53:59 You up for a couple more tough ones?

54:01 Yeah, hit me.

54:02 All right.

54:03 All right.

54:03 So this is like a secret question, but I hear it from students a lot.

54:09 Are you still afraid that some of the software you build is going to break?

54:14 And does that feeling ever go away that you're not thinking in the back of the head?

54:18 Oh, my God, what I just built like it's going to crash.

54:21 Yeah, that's a great question.

54:22 I certainly have little scripts and apps and utilities that are like super sketchy because they're like I needed them once or twice to do a little thing.

54:32 Those will probably break because they're not very well put together because they're just a quick one offs.

54:38 But at the same time, I'm not afraid because like they're not really important.

54:40 I could just if I need them again, I can kind of fix it.

54:43 But things that do make me nervous are like the training website that people have paid tons of money for so that they can take their classes.

54:51 Maybe they've got a project next week.

54:54 And it takes a long time to feel like it's not going to break anymore.

54:58 But I feel like it's not going to break.

55:00 All right.

55:00 Like not in any like super significant way, but it takes years.

55:06 So let me just put it to you this way.

55:08 If I go on vacation, like, for example, I went skiing two weeks ago up on Mount Hood, went stayed up there in Timberline for a couple of days.

55:18 And I was thinking, I really don't want to take my computer because I'm going to have to leave it in the car and like sub zero weather and it might hurt the battery.

55:26 But what if the website goes down?

55:30 Maybe I'll bring my computer just in case.

55:32 And I'm, you know, still and I would say that's one.

55:34 Like so I decided, no, it's been fine for the last six months.

55:37 I'm not going to touch it before I leave.

55:39 It'll probably be fine.

55:41 Worst case, I'll buy an SSH client on my iPad and try to get in there and fix it that way.

55:48 There's always like a little bit of a like a little bit of an uneasiness because you just you never know.

55:53 Right.

55:53 Like it could be things out of control.

55:55 Right.

55:55 Like maybe something happens in the data center or you just don't know.

55:59 But generally it does go away.

56:02 But it takes a long time on any given project, I think, before it feels OK.

56:05 Yeah.

56:06 I just think that's like being afraid of that your thing will break.

56:09 It's not something that programmers talk a whole lot about, but it's something that I do hear from students and that I feel myself.

56:15 So I just want to make sure that like actually a lot of other people sometimes felt that way, too, and that it was normal.

56:21 Yeah.

56:22 I'll share one more story with you.

56:23 The very first website that I wrote that did e-commerce was for a pretty expensive product for quite a bit of students.

56:30 It was a training thing.

56:31 It was not mine, like my own thing.

56:33 This is for a company I work for.

56:35 And they would come and buy like $4,000 things.

56:39 And I remember writing that and just being super nervous about shipping this offer.

56:46 I'm like, people are going to go put their credit card in here.

56:48 We're going to charge them $4,000.

56:50 What if I mess it up?

56:51 Like what if this is supposed to be dollars and it's cents or it's supposed to be cents and I put in dollars?

56:56 Like just, you know, the consequence of failure was really high.

56:59 Like what if I end up charging somebody, you know, $40,000 or like just like those things are super nerve wracking.

57:07 But, you know, the more you go through them, right, the second, the third, the fourth time, you're like, ah, okay, I'll just be careful.

57:13 But this is how it goes.

57:14 So, yeah, it doesn't ever go away all the way.

57:17 But, yeah, it gets a lot better.

57:18 So, my final question is what's the hardest problem that you ever worked on that just had a super simple solution?

57:24 You're banging on your head for like two weeks and then you were like, oh, it's just this.

57:29 Yeah.

57:29 There's been a couple of those.

57:31 I would say there's certainly some around like APIs.

57:34 And you're like, oh, man, I just can't get this to work.

57:36 And I go, why didn't I just, the order is like this.

57:38 But I do have an engineering related one for you that's probably like this.

57:43 I was working on this project with one other developer.

57:47 And we were, this is actually coming from MATLAB code into C++, C#, something like that.

57:54 This is before Python.

57:55 And it was doing real-time wavelet decomposition of like a time series.

58:02 And the times, it was something like you have all this state, about 250 measurements per second.

58:08 So, you have four milliseconds to process a measurement, right, like per measurement.

58:13 You don't have much time.

58:14 And we take all this MATLAB code and we convert like thousands of lines of wavelet decomposition code and stuff.

58:22 And just looking at this going, I sure hope this works because I can barely like understand what I'm doing here, right?

58:27 But it also has to be fast enough.

58:29 It has to be real-time.

58:30 So, it can't take more than four milliseconds per bit.

58:33 Or if you queue it up, you know, like times the number of things.

58:36 And so, we get to it and we run it and it's not fast enough.

58:40 Like, oh my God.

58:41 Like, we're going to have to optimize that wavelet code because I cannot change that.

58:46 Like, I got it working but really I don't understand more.

58:50 And it turned out, I turned a profiler against it and it was using a list instead of a dictionary to hold a bunch of stuff.

58:56 And it had to do like lookup based on some cases or like what would be better by key rather than by position, right?

59:03 And we just switched the data structure from a list to a dictionary and it got like 10 times faster.

59:08 Cool.

59:09 So, boom.

59:09 It fell under four milliseconds.

59:10 It was done.

59:11 Problem solved.

59:12 Right?

59:13 But like it was, this program is too slow.

59:15 What is it doing?

59:16 Oh, it's just doing like index of type operations on the list.

59:21 Like, that's 80% of the CPU usage is list lookup.

59:24 Switch it to dictionary and that 80% goes away.

59:27 Boom.

59:27 Fast.

59:28 That's it, I guess.

59:30 For me, it just seems to have come down to go to PyPI.

59:34 Yeah.

59:34 I was trying to surf these webpages and then it was like, oh, just use Beautiful Soup.

59:38 Or like, oh, I was trying to get a website.

59:40 Oh, just use Request.

59:42 So, now that's kind of the first thing that I do is go to PyPI.

59:45 Yeah.

59:45 I remember that being a big problem.

59:47 If you either come from a language that doesn't have a rich standard library like C++ or something or C or you just don't know, you have no awareness, you're like, oh, I got to do this thing.

59:56 So, how am I going to implement it?

59:57 Like, that should not be your first thought.

59:59 Your first thought should be, can I pip install an answer here?

01:00:02 Right?

01:00:02 Right.

01:00:03 Can I pip install a solution?

01:00:04 For sure.

01:00:05 For sure.

01:00:05 Nice.

01:00:06 All right.

01:00:06 Thanks for the turnaround on the questions there.

01:00:09 That was fun.

01:00:10 Before we call it a day, though, I got to ask you the two questions, as always.

01:00:14 If you're going to write some Python code, what editor do you use?

01:00:17 Well, I like this question because students often want to know which editor to use, too.

01:00:21 So, I've got kind of a multi-pronged answer.

01:00:23 If I'm writing a one-file thing, I'll use a Jupyter Notebook or Jupyter Lab.

01:00:28 And if I'm writing a multi-file thing, often I use Visual Studio Code.

01:00:33 And if I've got something really big, you know, like a Django site or like Flask, then I use PyCharm.

01:00:40 All right.

01:00:40 Those all sound like good answers to me.

01:00:41 I agree.

01:00:42 Right on.

01:00:42 And then, notable PyPI package?

01:00:45 So, I've got kind of a niche one, but a package I've been using is called SCIMDRAW.

01:00:50 And the SCIM stands for schematic.

01:00:52 So, it's a package that draws circuit diagrams, like puts resistors and capacitors and stuff in circuit diagrams.

01:01:00 And I've used that package to make test questions.

01:01:03 And there's some figures in the book that I used it with.

01:01:06 So, if you've got to draw some circuit diagrams, like check out SCIMDRAW.

01:01:10 That's pretty awesome.

01:01:10 All right.

01:01:11 Well, definitely enjoyed talking to you about this, Peter.

01:01:14 It was really fun.

01:01:15 And I'm glad you found your way from MATLAB to Python and SuperHub.

01:01:19 That was awesome talking to you, Michael.

01:01:21 Thank you so much for having me.

01:01:22 Yeah, you bet.

01:01:23 So, final call to action.

01:01:24 People are excited about this.

01:01:25 Maybe they want to use your book or contribute or they want to like try to set up a similar thing in their environment.

01:01:31 What do they do?

01:01:33 So, my final call to action would be if you want to take a class, think about community colleges.

01:01:37 They have programming classes too.

01:01:39 If you're an instructor, please use my book or send me a PR.

01:01:43 If you want any changes.

01:01:44 And if you're teaching MATLAB, Python can do everything.

01:01:47 So, if you're afraid, set up a JupyterHub server and you can start next quarter.

01:01:52 That's awesome.

01:01:52 All right.

01:01:53 Well, thank you so much for sharing your story.

01:01:54 It was a lot of fun.

01:01:56 This has been another episode of Talk Python to Me.

01:01:59 Our guest on this episode was Peter Kazarinov.

01:02:02 And it's been brought to you by Linode and Stellarress.

01:02:04 Linode is your go-to hosting for whatever you're building with Python.

01:02:09 Get four months free at talkpython.fm/linode.

01:02:12 That's L-I-N-O-D-E.

01:02:15 Find the right job for you with Stellarress, the AI-powered talent agent for the top tech talent.

01:02:20 Visit talkpython.fm/Stellarress to get started.

01:02:24 That's talkpython.fm/S-T-E-L-L-A-R-E-S.

01:02:29 Stellarress.

01:02:30 Want to level up your Python?

01:02:32 If you're just getting started, try my Python Jumpstart by Building 10 Apps course.

01:02:37 Or if you're looking for something more advanced, check out our new Async course that digs into all the different types of Async programming you can do in Python.

01:02:45 And of course, if you're interested in more than one of these, be sure to check out our Everything Bundle.

01:02:50 It's like a subscription that never expires.

01:02:52 Be sure to subscribe to the show.

01:02:54 Open your favorite podcatcher and search for Python.

01:02:57 We should be right at the top.

01:02:58 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm.

01:03:07 This is your host, Michael Kennedy.

01:03:09 Thanks so much for listening.

01:03:10 I really appreciate it.

01:03:11 Now get out there and write some Python code.

01:03:13 Thank you.

01:03:13 Thank you.

01:03:14 Thank you.

01:03:14 Thank you.

01:03:14 Thank you.

01:03:15 Thank you.

01:03:16 Thank you.

01:03:16 Thank you.

01:03:16 Thank you.

01:03:16 Thank you.

01:03:17 Thank you.

01:03:17 Thank you.

01:03:17 Thank you.

01:03:17 Thank you.

01:03:17 Thank you.

01:03:17 Thank you.

01:03:17 Thank you.

01:03:17 Thank you.

01:03:17 Thank you.

01:03:17 Thank you.

01:03:18 Thank you.

01:03:18 Thank you.

01:03:18 Thank you.

01:03:18 Thank you.

01:03:18 Thank you.

01:03:19 Thank you.

01:03:19 Thank you.

01:03:20 Thank you.

01:03:21 Thank you.

01:03:21 Thank you.

01:03:21 Thank you.

01:03:21 Thank you.

01:03:21 Thank you.

01:03:22 Thank you.

01:03:23 Thank you.

01:03:24 Thank you.

01:03:25 Thank you.

01:03:26 Thank you.

01:03:27 Thank you.

01:03:28 Thank you.

01:03:29 Thank you.

01:03:30 you you you Thank you.