#173: Coming into Python from another Industry (part 1) Transcript
00:00 Not everyone comes to software development and Python through four-year computer science programs
00:04 at universities. This episode highlights one alternative journey into Python. Over the course
00:10 of two episodes, you will meet people who started in other industries and specializations and now
00:15 make Python part of their daily experience. Some of them have used programming to power up their
00:21 specialization. Others decided they'd rather just be doing programming full-time and made that switch
00:26 over. This episode is part one of a two-part series. Our guests this time are Derek Chambers,
00:31 Jim Taysom, Arash Shoheli, and Rob Ward. This is Talk Python to Me, episode 173, recorded July 25th,
00:40 2018.
00:54 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem,
01:00 and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I'm @mkennedy.
01:05 Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter
01:10 via at talkpython. This episode is brought to you by Linode and brilliant.org. Check out what
01:16 they're offering during their segments. It really helps support the show. Derek, Jim, Arash, and Rob,
01:23 welcome all of you to Talk Python.
01:24 Thanks for having us.
01:25 Thank you.
01:26 Great to be here.
01:26 Thank you.
01:27 You're welcome. It's good to have you all here. I'm really excited about this short series of shows
01:31 that I'm doing on people getting into programming in Python from other disciplines like chemical
01:36 engineering or accounting, things like that. So I guess we'll get started by kind of setting the stage
01:42 and talking about where you're coming from because you're not really full-time developers.
01:47 you're doing other stuff and using development as kind of a superpower. So I guess the first thing I
01:53 want to ask you guys is what is your industry? What is your specialization? What do you study? What
01:57 are you doing day to day? And I guess we'll start with Derek.
02:00 Great. Well, I'm a mining engineer who studies earthquakes and works for the Centers for Disease
02:04 Control. So I usually get a lot of blank stares when I introduce myself that way. Essentially,
02:08 the Centers for Disease Control has a group that does occupational safety and health research,
02:13 and I work within that group studying the mining industry.
02:15 That sounds really interesting. What kind of background do you have to work in this? Did
02:20 you go and get like a bachelor's degree in mining and geology?
02:24 Yeah. So I have a bachelor's degree in mining engineering and then I did my master's work in
02:28 geophysics.
02:29 Very awesome. Jim, how about yourself?
02:31 I work for Radiant Solutions. We're federal contractors in the DC area. My background is as a
02:38 geographer. I got my bachelor's degree in geography with a focus on geographic information systems.
02:43 And more recently, I've been transitioning into a full-time developer role because of picking up
02:50 Python and all the things that's led me to.
02:52 Yeah, that's pretty awesome. You know, I had a similar path. I studied something else and I just
02:56 did more and more programming until I'm just like, why am I doing that other stuff?
02:59 But are you still doing programming around GIS? I know Python and GIS are like a really good fit.
03:06 ArcGIS and the Esri folks do a lot with Python, for example.
03:09 Yeah. So I'm still very involved in the GIS community. The project that I'm on is very GIS
03:15 centric, although I'm doing more of like the web backend for the project right now.
03:20 Okay, cool. Arash, how about you?
03:22 Yeah, I'm working for MarketSmart and we do like software and services for nonprofit organizations
03:28 exclusively. And I was traditionally trained as a organic chemist, but I've pretty much transitioned
03:35 over now. So I'm not really involved in doing really chemistry anymore. Maybe in the future,
03:39 I'll maybe get back to a cross section of doing something with software and chemistry, but I'm
03:44 pretty much doing software engineering now full-time and not really as much doing it, not really doing
03:49 any chemistry anymore.
03:50 That's pretty interesting. How do you feel that studying like a science, like a hard science
03:56 background sort of helped you become a programmer or did it not help at all?
04:01 I think it helped. And what I realized is the skill that I had as a scientist is basically
04:05 the scientific method and the problem solving approach is actually exactly the same as software
04:10 engineering. When I came over, I realized that the approach to solving problems, you know, forming
04:15 a hypothesis, researching what's out there, testing something, feedback loop. That's literally
04:20 what I did in as organic chemist every day. And so the same process is really applied to
04:25 software engineering. It's really the tool set that's changed. Obviously, I'm a guy chemist,
04:30 so you're in the lab messing with chemicals, trying to make stuff. And in software engineering,
04:35 you know, you have a computer and you're trying to build software, but the approach to how you're
04:39 going about solving a problem was actually the same. And it actually really, I think, kind
04:44 of helped me be able to make that transition.
04:46 Yeah. I had it, like I said, a similar background. And I feel like, you know, people will joke like,
04:51 Michael, you almost got your PhD in math, but does that seem like a waste? Right? And I don't
04:55 think it was. I think like those types of thought processes are really, really similar. And like you
05:00 say, the problem solving and stuff like that. So I think it prepares you pretty well. Yeah. So Rob,
05:05 how about yourself?
05:05 Yeah. So I got my bachelor's in accounting and had some accounting jobs early in my career. And I've
05:13 always been technically savvy and kind of along the way, I got exposed to SQL and that kind of led to,
05:20 Hey, this is kind of cool and more interesting than accounting. And then from there decided I wanted
05:25 to learn Python. And I will say like some of the others, I am not doing accounting anymore. I'm a data
05:31 analyst. So working my way towards data science and working on a master's in analytics. So that's cool.
05:38 So you're still going to school for your master's degree right now. How far along are you?
05:42 Right. Right. I'm about a year and a half in and I got a year left. So I'm just taking one class at a
05:46 time. Just keep it simple. Yeah. No reason to go full time. Right. Yeah. Kids and all that. But,
05:52 but yeah, from an accounting standpoint, I did actually use Python when I was still doing accounting
05:57 work a few years ago, which, you know, we can get into more a little later. Yeah. Sounds good.
06:03 So I guess maybe go back in reverse order with you, Rob. If you're in accounting, you know,
06:09 Excel is programmable, right? People do ridiculous stuff. I think I've seen a flight simulator created
06:14 in Excel. Have you seen this? I have not seen that. There's a pretty intense stuff done in Excel.
06:18 So, you know, it's probably like the natural place to do like automation and stuff like that. So why do
06:23 you learn Python and programming? I had years ago, I had, I tried to learn C and that failed miserably.
06:31 I mean, it's hard, man. Yeah. Yeah. I was like, what is this stuff? And so then I kind of decided
06:36 again, you know, data science was starting to become a thing and I knew I wanted more of a
06:41 business intelligence or data analyst type of role. I was doing accounting and, not the name drop,
06:48 but my brother works for GitHub. So he's like, you should learn Python. I'm like, okay, I'll learn Python.
06:53 So, you know, I think it was about, it was five years ago and, I just started on,
06:59 I think it was that learn Python the hard way, which I know that's going to be controversial
07:03 to some people, but, it was a good starter. And then also took some Coursera courses. They have a
07:09 lot of Python, courses, got through about half of it until I got to, object oriented programming and
07:15 was a little dumbfounded at the time. And, I was able to, to see the value and see that there
07:23 were things that could be applied to things I was doing every day in my accounting work,
07:27 which I don't know if you want to get into that right now or a little later.
07:30 Yeah. Let's get into it maybe a little bit later, but I think there's definitely some really cool,
07:33 like automation of, of sort of Excel and processing of those files. And there's lots of
07:39 cool stuff in there. Yeah, for sure. Yeah. We'll dig into that. So Arash, why did you get into
07:44 programming in Python? You were in the lab, your hands are already got like gloves on. It's hard to
07:48 type. Yeah, yeah, yeah. I know. I, I, I mean, honestly, I, I went down a path where I was trying to do
07:54 in chemistry, but I got into pharmaceuticals and, and I started not liking the big companies in the
07:59 pharmaceuticals. I, I thought about, academics and it was not looking good as a very few jobs.
08:05 Funding is kind of bad. And I'd always been interested in technology and I was doing some
08:09 web development, not really much programming, but I was doing that stuff. I was getting more and more.
08:14 And then eventually I kind of just, I don't know. I've always been someone that,
08:19 that kind of takes chances. And so I kind of just was like, well, let me try to see if I can,
08:22 you know, work my way into, into this field. And I kind of, and I got there and I kind of
08:26 pulled myself up, and learned and, and made it through. So it was really kind of been the
08:32 industry of, of chemistry, at least for me, didn't really kind of, kind of failed me.
08:37 At least for me. Yeah. But actually the Excel thing you were talking about, I will say something.
08:40 We actually had an entire application here. We built Marcus Smart out of this, Excel. We had this,
08:46 this guy who was working for one of these nonprofits who had this crazy Excel file
08:51 that he, nonprofits would send him his data and he would just, had this,
08:56 basically these macros he had written and it would take like, some of it would take like a data run.
09:01 And he finally came to us and we talked to him. He's like, you know, this is, this is awesome.
09:05 People are sending me this stuff. They love this thing, but it takes me so long to run. I'm like,
09:09 can you put, make the software as it? We're like, yes, that's what it should be for. So we actually
09:14 built an entire application. it's called the fundraising report card.com that is out of,
09:19 came born out of literally of Excel files. That's awesome. Yeah. I think there's so much
09:24 programming happens in Excel, honestly. So one thing I wanted to ask you coming from on that path,
09:29 do you feel, I know a lot of people talk about imposter syndrome, not coming with a computer
09:34 science degree. Was that a problem for you or were you okay with, did you feel like, do I belong in
09:40 this programming world? Like how do I sort of prove that I earned my chops here? Yeah. Yeah. No,
09:44 I think that's real. I mean, I think people feel it. I've definitely felt that. I think the first
09:48 year was really tough because there was a lot of nomenclature, a lot of words. I just didn't
09:53 understand. And literally there was a moment I, even though it was an accumulation, I literally
09:58 remember a moment after a year where one day it just kind of like clicked and I was like, okay,
10:04 I understand. Like, I don't understand. I don't know everything, but now I can understand. I can read
10:10 an article. I can go through something and comprehend and work my way through it. Where before,
10:15 you know, I would get lost sometimes just completely not understand concepts. So I think
10:20 it's real. And you got to, you know, read, read, read. I read constantly and you got to just always
10:24 train yourself and always be training. So to keep, and I think at some point you get over that to a
10:30 level, you're like, okay, I think I got, you know, through it. So yeah, I think it's, I mean,
10:34 you got, and you got to work through it. Yeah. I agree with that. I feel like once you get to the
10:38 point where you've learned enough new things repeatedly, you're like, there's a lot of stuff I don't know,
10:43 but if I need to know it, I can learn it really quickly. And once you kind of get that skill and
10:46 experience, I think the posture syndrome really fades pretty quickly. But yeah, if you don't like
10:51 continuously learning, this is not the industry that's going to work out so well.
10:54 Yeah, for sure. I mean, I think learning is, is a, and things change so much in software engineering,
11:00 I mean, almost on a weekly basis that you, you better be ready to be learning all the new stuff
11:05 and at least keep up to date with it.
11:07 Yeah, for sure. Jim, I think probably your introduction may be the simplest. If you're
11:12 in GIS, you almost have to learn Python to do some of that stuff, right? But what, what was the
11:16 motivation for you?
11:17 So for me, the first couple of years of doing GIS, I didn't really need to use Python at all. I ended up
11:23 in some mandatory training that I didn't need. And so I would spend hours and hours and hours every day,
11:29 just completely bored out of my mind. So I started going through all the ArcGIS documentation on how
11:35 to use ArcPy and started teaching myself Python enough that within my realm, I was willing to put
11:42 it on my resume. And then there was another position at work that needed someone who knew Python because
11:49 they had this old VBA application that was written for ArcGIS that they needed to replace because
11:56 Esri was...
11:57 Because it's VB6.
11:57 I don't even remember what it was.
12:00 Or VBA or whatever it was, yeah.
12:01 They needed someone that could take what they had and recreate it in Python because Esri had been
12:07 telegraphing for a while that they were going to be dropping VB support. And so it was more than I
12:14 was ready for, but they couldn't find anyone else. And so I just told my boss, I was like,
12:18 I can do the project, you just have to give me at least a little bit of time every week so I can
12:22 do some at-home Python training and figure this out. And I did it and loved it and started doing
12:29 more of it.
12:29 That's really cool. It's cool how you're able to sort of laterally move in your company because
12:33 of it too. It's not great to be stuck in mandatory training that's super boring, but at least you
12:38 made a really good use of it.
12:39 Yes. I use my time well.
12:40 Awesome. Derek, how about you? What was the motivation for getting into programming?
12:45 Yeah. I first got into programming when I started graduate school and I had an advisor
12:48 that was really hands-off. And so his charge was kind of just go figure out something new
12:54 and novel and useful and do it. So I ended up changing my research topic, I think three or
13:01 four times, but I was finding that it was very hard to use some of the existing tools that I'd
13:05 known to build sort of a larger system that would do something interesting. And so I run across this
13:11 Python package called obspy, which implements a lot of format parsers and data classes for handling
13:19 seismology data. And so once I found that, I got into Python and I was able to build this bigger
13:24 system that actually looked at implementing a newer method for detecting very small seismic events.
13:30 Like micro earthquake type things?
13:32 Yeah, exactly. The typical seismicity associated with mining is pretty small, so you have to have some
13:37 clever methods for detecting it in most cases.
13:40 Nice. And you said that you're now a contributor to that project, right?
13:44 That's right. Yeah. I'm a maintainer for obspy. I really enjoy when I can find some spare time,
13:48 you know, making improvements or suggesting different things we can do in the future.
13:53 Yeah, that's awesome. So what are my feelings about sort of people becoming programmers and having
13:59 these skills is it's great to just have like a computer science degree. You can solve problems and
14:04 programming stuff. But I think the real value of many more people having programming skills and Python
14:11 skills is like it's kind of a superpower for the thing that you do do. So I wanted to ask you all
14:17 sort of how you saw that in your industry. I know not all of you are still doing that day to day, but you know,
14:24 it sounds like all of you sort of got started in programming while you were still in that industry.
14:29 So Rob, what do you think about this sort of power up idea?
14:33 No, yes, I can attest to that firsthand because in my last job that was truly an accounting job,
14:41 I was I was the accounting manager for a small company in my hometown. And we had they were a
14:49 property manager and had over 500 homes or mortgages that that I had to manage. So every month I was
14:57 sending out these mortgage payments and I have to put them in the in our accounting package, our software,
15:02 right. So I found out there was an easy way to or there was a way to import the data. But there was
15:09 no easy way to create that in Excel, at least not that I could find I didn't want to manually type
15:15 these out. So I, my first project in Python, you could say was, I figured out how to do all the loan and
15:23 amortization payments and the principal and interest, and make them basically batch load this entire file
15:29 for the month. And that's been my workflow, you know, made made paying the bills so much easier
15:34 from a daily on a daily basis.
15:36 That's great. And there's so many examples, I'm sure that are out there like that, like even in my stuff,
15:43 you know, obviously, I have to write code to run websites and things. And I've done other types of
15:48 programming for many years. But like, one example was, I have all of these, these videos for the courses,
15:55 courses, and I had to import them. It's when I load a new course, and it doesn't sound like a big deal. But
16:01 you've got to set the time and like the duration, there's just tons of stuff. I was even thinking of
16:06 hiring somebody to like do that. And I'm like, wait a minute, I want to just write a program. And now it's
16:09 like, you know, a five second command line type it and boom, it's done. It's, it's just so nice when you have
16:16 those realizations, like, why am I doing this by hand? There's something surely this, this can be automated,
16:21 right? That was the impetus for me, you know, everything was doing, doing in my hand. And I,
16:26 and I had played around with, you know, maybe people have heard of auto hockey, you know, I'd done made
16:32 macros with auto hockey before years ago in windows. And so this is kind of a similar idea, like, just
16:39 automating simple tasks that you have to do anyway, but might as well just speed it up.
16:44 Yeah, I can see somebody actually deciding, I'm going to go do that from the coffee shop,
16:49 taking every other Friday off as their code runs and, you know, does the work they did for half a
16:55 day. All right. So, Jim, how about you with this, this power up idea?
16:59 So it was actually really, really amazing. Some of the things that I was able to do
17:03 like really early on in learning Python. So a lot of the GIS workflows are really intensive and you're
17:10 doing a lot of steps over and over. There's been a couple of things where I've done where you might be
17:15 taking and doing like four things, but you do them a thousand times in a row. And without something
17:20 like Python, that's almost impossible to do. And my favorite example is there was a coworker who was
17:27 working on trying to do some correlations between some various data sets and his manual workflow that
17:34 he was using would take him between three and four weeks. So he was spending 120 to 160 hours,
17:41 like just manually just going through the data.
17:44 It's like, not only is that inefficient, but that's got to be terribly ungratifying type of work.
17:48 Oh, just terrible. Cause it was just like, click, click, click, wait five minutes, click, click,
17:55 click, wait five minutes for weeks on end. And when I heard about that, I was like, all right,
18:01 let's fully explain what you're trying to accomplish. And I'll work something out. And a week later I had a,
18:09 just a little script written in Python that did his entire workflow. But instead of being able to like
18:17 correlate, like, like the top three correlations, I was able to correlate all of the data sets
18:23 and the entire script would run in eight minutes. So I replaced a month's worth of work with an eight
18:29 minute job and it only took me a week to do it.
18:32 That's incredible. What was the reaction of this guy?
18:35 He was just completely blown away because like he went from being able to do like this type of analysis
18:41 12 times a year to being able to do it 12 times before lunch.
18:46 Yeah. That's amazing. I've worked at companies before where there are a lot of like real scientists
18:51 doing scientific work. And it was this kind of thing. It was like really manual involving lots
18:56 of different sort of software pieces like Excel or other MATLAB or stuff glued together. And there's
19:01 they did the tedious little steps thinking like, this is how you do it. And it just, you have to do it
19:06 this way. And me and a couple other folks, we sort of systematically started automating like the
19:11 worst of these things like you're describing here. And every time something amazing like that came
19:16 out, it was like, oh, you're going to automate us out of a job. It was like, oh, yeah, this was my job.
19:22 And now I don't do it anymore. What am I going to do? But of course the next week they were doing
19:25 higher level stuff or doing it more in more depth or whatever. And those people never got automated out
19:30 of a job. As far as I could tell, there's always more problems that can be solved once you free up some
19:35 time. This portion of Talk Python to me is brought to you by Linode. Are you looking for bulletproof
19:42 hosting that's fast, simple, and incredibly affordable? Look past that bookstore.
19:46 And check out Linode at talkpython.fm/Linode. That's L-I-N-O-D-E. Plans start at just $5 a month
19:54 for a dedicated server with a gig of RAM. They have 10 data centers across the globe. So no matter where
19:59 you are, there's a data center near you. Whether you want to run your Python web app, host a private
20:05 Git server or file server, you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network,
20:12 24-7 friendly support, even on holidays, and a seven-day money-back guarantee. Do you need a little
20:17 help with your infrastructure? They even offer professional services to help you get started with
20:23 architecture, migrations, and more. Get a dedicated server for free for the next four months. Just visit
20:28 talkpython.fm/Linode. Derek, power up idea? Yeah, I think there's kind of three components to
20:37 this. The first one is scale. You can usually do a process a lot bigger data sets with Python or some
20:43 other programming language than you would normally with Excel or some other tools that you might be using.
20:48 The other one is maintainability. You know, you can put Python scripts into proper version control like Git,
20:54 see what changes and when it changes. Unlike Excel, where you might have people that are working on a
20:59 common workbook and change different things and you don't know who changed what. And then also really
21:03 increasing the level of automation. Just like Jim had mentioned, I've had a few experiences, nothing
21:08 quite as dramatic, but you know, co-workers have come to me with a really manual process and asked for
21:13 help automating it. And we've been able to do that in Python and save a lot of time. That's cool. Are
21:17 you, are you known as like the guy that you can come with manual stuff and you can fix it? I'm trying
21:23 not to let that reputation spread, but yeah, a little bit. This podcast won't help, I'm sure.
21:28 Arash, how about you? An example we have is, and I think, yeah, it's very important to kind of get
21:34 everybody to understand that, that a little bit of programming knowledge can really make you very
21:39 powerful. We have account managers who, similar story, were doing some manual stuff that, some
21:45 campaign data that wasn't available via an API. So they were literally going to every account and like
21:49 putting copy and pasting it to an Excel. And of course that made it tedious. So eventually they came to us
21:55 and I kind of, with a, with somebody helped them write a web scraper with Selenium to just basically
22:00 scrape that data. And so something that took her basically eight hours or a whole day, or, you know,
22:06 would just bog her down. We were able to do it like basically the same thing in five minutes, which, you know,
22:12 they were very thankful for. And she is actually somebody who is starting to learn Python. And I think that's the
22:17 thing, like for people to get into using programming to make themselves super powerful, they do need somebody or
22:24 somebody to really kind of show them the benefits of it and kind of get them in the direction. Because I think it's,
22:30 that's not the first thing they're going to think about, because it's not natural for them. And also,
22:35 they don't really know how to get into it. So I think it really, to kind of make that happen,
22:39 you kind of do need somebody to kind of show them the way. And then hopefully that gets them excited
22:44 enough and realizes the benefits that they can start to go down that path and realize, yeah,
22:48 I could be an account manager, I could be a product manager, I could be kind of anything, but with a
22:52 little bit of programming, I could be a really powerful account manager or a powerful product manager or
22:57 whatever field you're in.
22:59 I think that's a really great point about sort of seeing the light and realizing that it's not
23:03 all that complicated. Because to me, before I got into software development, programming computers
23:09 seemed really like a deep, complicated thing that took years and years and years of studying. And you
23:15 had to be the right type of person to have that kind of thinking and that just that knowledge,
23:19 right. And while there is some type of like, some types of software that probably is that way,
23:25 like writing a kernel for a new OS that you've invented, probably is like that. But a lot of the
23:30 stuff that we're talking about, and what most people need is actually not that that bad. But
23:34 you kind of have to see the light, right? You're like, wait, you did that in 15 minutes. And now
23:38 my horrible job I did for eight hours once a week is now five minutes in automatic, like,
23:43 maybe I could do that.
23:44 Exactly. I think it's also and I try to tell everybody in our company that it's like,
23:48 you know, you don't need just because you do a little bit of programming, or you're going to
23:52 use some tool doesn't mean you're a programmer doesn't mean you're going to be an engineer.
23:55 That's not the point of it. It's just a tool set that's going to help you do whatever is your job
24:01 better. So you don't need to think that I have to be a programmer or have to be an software engineer
24:06 to do it. It's just a tool set that you can learn just like any other tool set.
24:10 Yeah, maybe you are into psychology, and you don't want to be a programmer, you want to be a
24:14 psychologist. Well, I'm sure there's awesome stuff you could automate and discover and make you a
24:19 better psychologist, right? You don't have to give that up.
24:20 Exactly.
24:21 All right. So let me ask Derek and Jim this one, because you guys are still doing the sort of
24:26 what you studied day to day. How has learning programming changed what you do day to day? Like,
24:34 has it changed your job? Or has it not really?
24:36 So I guess I'll start, I would say that it's completely changed almost everything that I do,
24:41 because ArcGIS, it's very good at what it is. But it's also very expensive. And so a lot of places
24:48 don't necessarily want to use it. And it's also quite slow. And so there's a lot of things I can do in
24:54 Python that are a lot faster. But one of the things that you end up missing is that like that interface
24:59 that ArcGIS provides. And so now I'm doing a whole lot of like, web programming and back end
25:05 development to like, like build up a core system. But then you still need something that like can be
25:11 provided to a UI so that when my teammate makes a really beautiful UI, it can consume everything that
25:18 we've processed on the back end.
25:21 Yeah, that makes a lot of sense. I think this is really great. Derek?
25:24 Yeah, so a lot of people that study seismology are already kind of halfway programmers, you can't
25:30 really do much in seismology without having some programming knowledge. But a lot of tools that were
25:35 common there were things like Fortran and scripting things together, stitching them together with bash,
25:40 that sort of thing. So I think Python has given me maybe an advantage over a lot of those
25:46 old workflows and that I can prototype things faster, maybe try out a new research idea much faster than
25:53 than I would have if I wasn't using Python.
25:55 Yeah, there's probably a lot of MATLAB and those types of things going on as well there.
25:59 Right. Yeah.
26:00 Yeah. So what type of packages and tool sets do you guys use? Is it like Anaconda? Is it Django?
26:08 Derek, I'll keep going with you first, I guess.
26:10 Yeah, I mostly use Anaconda. That's been a really great system for us just to avoid a lot of the
26:16 compilation, especially when we have to work on Windows systems.
26:19 Yeah, a lot of the scientific tools can be a pain to install. So getting them pre-compiled
26:24 and just download.
26:25 Yeah, very much so.
26:26 Yeah.
26:26 VC var all bad or whatever that is.
26:29 Yeah, I've seen that a few times.
26:31 Yeah, for sure. Jim, how about you?
26:33 So at first, when I was just learning, really the only thing I had available was the Python
26:39 installation that came with ArcGIS. So I had the core library and ArcPy and nothing else for
26:45 probably the first like two years of using Python. And then when I got Anaconda, it was fantastic
26:51 because I had all these great extra libraries that I could use. And now at this point, it's
26:56 a whole lot of web programming. So like right now I'm working with Sanic. We're using Postgres
27:02 and the Postgres extension. So because of that, we're using asyncpg to connect to Postgres. We're
27:08 Docker and Kafka and all these other things. I'm using PyKafka to communicate with Kafka.
27:13 So it's gotten to be very, very diverse compared to where I started with Windows and ArcGIS.
27:21 That is a serious change there. That's amazing. How do you like Sanic?
27:27 So for the most part, I really do like Sanic. It's very responsive. Every once in a while when I get
27:33 like down into a rabbit hole and there's just like not nearly as like many things on Stack Overflow of
27:38 like, how do I fix this problem? And I'm still wrapping my head around the whole async thing. So
27:44 it's definitely different, but I do enjoy it. It's fast.
27:48 That's cool. Yeah. I've definitely wanted to do more Sanic, but I actually have a little,
27:52 a little bit of that feeling like, well, not necessarily that there'll be not enough answers
27:58 on Stack Overflow, but there's also all the other support stuff. Like here's an example of how you
28:04 do this, or here's a library you plug in that like fills this gap and things like that. So,
28:08 but it definitely looks like a cool framework.
28:10 There's definitely some of that missing. And so there's been times where it's like,
28:13 I really wish I had just started this project using Flask instead of Sanic, but.
28:18 But now, now you get these flask.
28:19 We had very good reasons for choosing Sanic at the beginning.
28:21 Yeah, absolutely. Arash, how about you?
28:24 Yeah. As far as toolset, I mean, we're mainly doing some of the typical Python web stuff. So we use
28:30 Flask, Frontend, we use Angular. We recently got into like Airflow, which I don't know if everyone's
28:36 familiar, but it's from Airbnb.
28:38 Yeah. It's like a data pipeline thing, right?
28:40 Yeah. It's data pipeline. So we'd originally doing a lot of our data pipeline still is, was with
28:45 Pentaho. And I really hate that software. It's good at some stuff, but it's really annoying. I hate using
28:51 it. And so I started looking at Airflow and that's really nice because it's Python. We know Python,
28:56 it's code. And it's just a lot easier to kind of get it up and running. That also does suffer a little
29:02 bit right now still from, I mean, it has decent documentation, but there's still a lot of stuff
29:06 there that it's kind of hard to find when you run into some issues. So, but it's really growing.
29:11 And I think Google just brought out a product, which the underlying platform of it is using
29:15 Airflow. So I think that, you know, Airflow is probably here to stay and it's going to probably
29:20 grow. So, and then we're also using a Falcon framework for like an API that we've set up.
29:25 So yeah, Falcon, I've had those guys on the show a while ago. It's like a really low level,
29:30 low latency API framework, right?
29:32 Yeah. I mean, I originally, I used to set it up maybe like two years ago when maybe I still wasn't a
29:38 little bit understanding of all the frameworks at the time, but I'm kind of glad I did actually,
29:42 because it was strictly for an API for our company and it is really fast and it's held up. And although
29:49 I probably would have used one of the, if I had to do it again, I'd probably use one of the other
29:52 frameworks that's built on top of Falcon, like is it Hugs or?
29:56 Yeah. Hugs is one of them.
29:57 Yeah. Yeah. I used, we used one of those, but still it's, it's a really good fast framework for what we
30:03 want it.
30:03 Okay. Yeah. That sounds really interesting. I haven't really talked about Airflow on the show.
30:07 maybe give people a quick example of like what problem or what job you're accomplishing with
30:13 Airflow, like describe like why you're using it.
30:15 Yeah. So, I mean, basically it's kind of in the real, it's not technically an ETL, which is
30:21 extract transform layer. So in a lot of companies, office software engineering, you get data,
30:25 but a lot of times you need to get data from one place and kind of transform it and do some
30:30 things to it and then bring it over to another place, usually to another database.
30:33 And so that's where like an ETL comes in. And so those are usually workflows that you want to set
30:39 up that maybe go through many steps. And there are different like sort of softwares out there to
30:44 accomplish this. And Airflow is technically just like a scheduler of these flows, but it's got a nice UI
30:50 to it. You can kind of put tasks, you know, one after the other. It's, it uses directly acyclic graphs.
30:58 So you can, and then the thing is it's code. So because the, you write Python code for those tasks,
31:03 you bring in a lot of programmability where a lot of the other ETL software,
31:07 if you ever use it, they're not like that. They kind of come with prepackaged sort of modules and
31:13 you kind of just have to use those prepackaged modules. And then it gets transported like,
31:17 like Pentaho is, it's really a Java XML. So it's, it's taking XML code and eventually it's,
31:24 you know, it's run by Java. So it's not, you know, that sounds fun. Yeah, it's, it's,
31:30 yeah, I hate it. I love Airflow so much better. Although there, you know, there are differences
31:35 and there are some things that, that sort of like Pentaho is better than an Airflow. I mean,
31:40 right now, just to give a use case, which also brings in async is that one thing we're using
31:45 Airflow mainly for right now is that we're, we're, some of our stuff is done in some other
31:49 software, some email platforms. And so we need a lot of their data through their API. So
31:55 instead of at real time getting that data API, we're like making tasks where we, where we hit
32:00 those APIs, get all that data, transform it the way we want, and then, and then put it in our database.
32:06 So then when we run our application, we don't need to rely on the API anymore. We just pull it from the
32:10 database. And we actually ended up using asyncio and IO HTTP to, to speed up those tasks, which came
32:17 really in handy and was, a great example of using async at that point because we make,
32:22 we're making thousands of API requests. And so instead of like a task original, we were running
32:28 was taking two and a half hours. We got it down to like 10 or 15 minutes.
32:31 That's really awesome. Yeah. Some of that IO HTTP stuff, the client side stuff is really powerful.
32:36 Yeah. So we're basically pulling all this stuff, pre-fetching all this stuff from APIs,
32:40 from our different, softwares that we work with and then, transforming it into the
32:44 forms that we need and dumping it in our database. So then that way we don't have to rely on them when
32:48 we actually run our application.
32:50 It sounds like a real cool use of airflow. Nice.
32:52 This portion of Talk Python to Me is brought to you by brilliant.org. Many of you have come to
32:59 software development and data science through paths that did not include a full-on computer science or
33:04 mathematic degree. Yet in our technical field, you may find you need to learn exactly these topics.
33:09 You could go back to university, but then again, this is the 21st century and we do have the internet.
33:15 Why not take some engaging online courses to quickly get just the skills that you need?
33:20 That's where brilliant.org comes in. They believe that effective learning is active.
33:24 So master the concepts you need by solving fun, challenging problems yourself.
33:29 Get started today. Just visit talkpython.fm/brilliant and sign up for free. And don't wait either.
33:36 If you decide to upgrade to a paid account for guided courses and more practice exercises,
33:41 the first 200 people that sign up from Talk Python will get an extra 20% off an annual premium subscription.
33:47 That's talkpython.fm/brilliant.
33:50 Rob, how about you? Tool set.
33:53 Sounds like I need to learn airflow, but I, yeah, I'm primarily use Anaconda and all the associated or many of the associated
34:03 packages that come with it. So pandas I'm using every day and I love Jupyter Notebooks or JupyterLab now. I have transitioned over to that. That's
34:12 mainly what I'm using at work. And then on a personal level, I, you know, still, of course,
34:17 love requests and beautiful soup, but that's my personal project.
34:21 The whole web is your API. It's sweet.
34:23 Yes, exactly.
34:25 So you switched from Jupyter to JupyterLab. Why was that?
34:29 That's kind of where Jupyter is headed from everything I've read. They're going to eventually,
34:32 I think they're going to sunset Jupyter Notebooks and it's going to move exclusively to JupyterLab. I don't know the timeline. It's probably a few years off still, but they had released the beta.
34:45 I believe it was December or January. I can't remember for sure, but I jumped on board right away and it works. I haven't had any trouble. There's a few features that are missing from JupyterLab still compared to Jupyter Notebooks.
35:00 But the nice thing is if you have the latest version of Jupyter installed, you can run as long as the Jupyter server is running, you can run both. It doesn't care.
35:11 It's just a UI front to the server.
35:13 Yeah, it's just a different UI than the Jupyter Notebooks. I mean, it'll look familiar, but then there's like the file browsers is built in right there. And so is the command palette. So it's all, it's just all built in right there.
35:26 And then you can, the nicest thing about it is you can have, you know, multiple tabs so you can compare notebooks side by side. You can actually copy cells or drag cells from one notebook to the other.
35:39 Oh, wow.
35:40 So, you know, I'm, yeah. So anytime I, which is really nice for reproducibility, because, you know, sometimes I might get a request at work like, hey, can you, you know, run this report, but change this.
35:50 I actually had that happen this morning where it's basically just, you know, I had to drop in like a lot of the same code, but then make a few small changes. So you just literally, I just copy or three or four of the cells and drag them to the new notebook and run it again.
36:06 It's cool. It's a little bit like, like an IDE version of Jupyter, right?
36:10 Yeah, that's actually a really good description for it. Yeah.
36:12 Yeah. We started using that a little bit. Our style, we were, we were also using Pandas and JupyterNova.
36:17 We started using JupyterLab and actually talk about empowering. This is another example. We had one of our colleagues who's working on this app and he would get requests of these files that they would send him that are like over a million.
36:30 And so Excel actually has real trouble when you get to really large files and he had to segment them.
36:34 So it would crash constantly. So he kept sending it to me and be like, hey, can you take care of this?
36:39 So finally I was like, you know what? I'm going to teach you Jupyter. It's not that hard.
36:43 And I did teach him. And once every week he comes to me and says, Arash, thank you so much for teaching me this. Now I can just take care of this stuff and I feel more empowered.
36:54 So that's another example of that empowerment. And then, and JupyterLab notebook, whichever one you use, like really makes that really easy actually.
37:02 I wonder if Jupyter for those types of folks almost is like a Trojan horse for teaching them programming. Like, look, you're not a programmer. You're not writing source code.
37:11 You've got this thing and it does like computation and you type in it. You know what I mean? Like, of course it's still Python and stuff, but like, I wonder if it's perceived differently by people who, who come at it from that perspective fresh.
37:22 Yeah, I think it could because the interface is not, you know, because I guess there's some web versions of it, or if you run it locally, you got that kind of web looking interface.
37:30 So it doesn't, you know, you don't have to pull up a dedicated editor that you have to install and things like that. So yeah, I think it probably is a good, you know, and it deals with data.
37:39 And most people in some area have to deal with manipulating data. And once in Excel, yeah, you can start getting large files or having to do any kind of manipulation of sorting or any kind of complex thing in Excel.
37:52 And it starts taking like two minutes to do that. And you're sitting there with the spinner and you're like, okay, I got to find something, something else.
37:59 It's got to be a better way.
38:00 I think on the flip side too, though, I mean, a lot of people can use Jupyter as kind of a crutch and build some really big systems that really should be put into proper packages and things.
38:10 So it's kind of hard to know once you've made the transition from exploratory data analysis into something that should be, you know, more software engineered.
38:18 That's a really good point.
38:19 I'm probably guilty of that myself.
38:20 There's probably cases where I should just be putting them into a script and not running it in the Jupyter notebook itself.
38:30 It's hard to know, though. It just sort of creeps up on you.
38:32 Yeah, definitely.
38:33 Yeah, that's really interesting that you could take it too far, right?
38:36 Like eventually you might want to break out of it.
38:38 To me, doing traditionally like web development and database stuff and so on, that's usually across lots of files.
38:45 And it's like a different way of programming than people do with exploratory data.
38:50 And it took me for a little while to get my head around like what is the real value?
38:54 Like why do people love Jupyter and these notebooks so much?
38:56 But when I saw people really doing the exploratory data stuff with them, oh, this really is pretty amazing for that type of work.
39:05 I agree.
39:06 I didn't get Jupyter.
39:07 I kept hearing people talking about it.
39:10 And it was really only about probably about a year and a half ago that I saw someone demonstrate it.
39:15 And then it clicked.
39:16 I'm like, oh, that makes a lot of sense now.
39:18 And it really has changed, you know, the way I can do a data analysis.
39:22 I hardly ever open Excel anymore.
39:24 I kind of hate it now.
39:25 Yeah, you've got to be pretty good with pandas for that.
39:28 But that's awesome.
39:29 Yeah, yeah.
39:30 That's, I mean, for sure.
39:32 I mean, panda, but I'll echo, you know, you get with large files.
39:36 You know, Excel just kind of starts crashing if you're trying to do a lot of, especially if you throw, you know, multiple view lookups in, you know, or literally a large data set.
39:46 But pandas, unless you're dealing with gigabytes of data, it is really, really fast.
39:52 Yeah, that's awesome.
39:53 All right.
39:54 So I want to sort of do a little bit of forward looking stuff with you guys here, maybe by starting looking backwards.
40:01 So I guess if you have people who are coming into your industry and they don't have any programming background, what advice would you give them?
40:10 Let's start with you, Derek.
40:11 Okay.
40:11 Well, I think the first one, a lot of people that I've told about programming coming from my background have said things like, well, I haven't had a class in that.
40:19 Or I don't know about that because no one is, I just don't have any experience with it.
40:23 And so I think getting over this mental block that you have to have taken a university or a high school course on something in order to learn it is a real barrier for some people.
40:32 So the first one would be just understand that you don't have to be taught everything formally.
40:37 And the second one would be to be a little bit humble and try to be comfortable being uncomfortable.
40:43 And what I mean by that is when you try something new, especially when there's other people around you that are really good at it, you're kind of putting yourself in a position of inferiority, right?
40:52 It can be kind of intimidating.
40:53 Maybe it's not the best feeling.
40:55 But if you can really just kind of take it step by step and learn what you can in those situations, I think you can really grow professionally and as a person as well.
41:04 Yeah, that's great advice, especially in today's age, you know, like if you could either take a semester long course and disrupt your life Monday, Wednesday, Friday for two hours plus homework and exams.
41:16 Or you could take a weekend and an online course and get pretty close, honestly, you know, even free stuff on YouTube, you may be able to sort of teach yourself.
41:25 So it's definitely a different perspective.
41:27 I think that people probably got to write their minds around.
41:30 Yeah, cool.
41:31 I think, too, if you have a specific problem that you're trying to solve, that really is a good way to learn.
41:36 You just learn enough to solve a problem and then you get a harder problem and you kind of build your knowledge of programming as you solve the problems you have to address anyway.
41:44 Yeah, because you can't learn it all at once.
41:46 It just becomes overwhelming.
41:47 All right.
41:48 So if you can focus it, like I need to do these three things.
41:50 How do I do those three things?
41:52 And it becomes way easier.
41:53 I'll echo that, you know, the statement about solving a specific problem because that was my advice, too.
41:58 I set out to learn programming just because and but I didn't I didn't have a problem to solve at first.
42:03 But then once I did and I didn't really make a lot of progress until I did have a problem to solve.
42:08 You know, once I was doing that accounting job and said, oh, I can automate this step.
42:13 And that's where I made progress because I was figuring out how to do a specific task.
42:18 And then I'm just like he said, you move on from there and solve bigger problems and take it step by step.
42:23 Don't feel like you have to learn everything all at once because you're you're not going to.
42:27 And it takes time to for things to come automatically.
42:30 I mean, there's things that I do now in Python that, you know, just come naturally.
42:34 But a year or two years ago, I would have had to look it up on Stack Overflow again, you know, for the hundredth time.
42:41 It's hard to realize that things like that will become automatic and you sort of go on autopilot like driving a car.
42:47 It's like there's so many inputs and things and stuff you got to juggle.
42:51 But eventually most of it becomes sort of just stuff you do automatically.
42:56 Yeah, even Pandas.
42:57 I mean, I've only been using Pandas for just over a year.
43:00 But because I've been using it every day for about almost a year, it is natural now, you know.
43:06 And and I know, you know, the documentation is there.
43:09 I know how to figure out how to do something, even if I don't know how to do it off the top of my head.
43:13 Yeah, that's excellent.
43:14 Jim, what's your advice?
43:16 So I would say for anyone who's working with ArcGIS, I would start by just looking at what your daily workflow is, whatever you're doing day to day.
43:25 Go look at Arc's help docs because they've got for every function that you can use.
43:31 They've got a Python example of how you do it with Python.
43:35 And it's not just like the one line of how to use the function.
43:38 It's like a short little script of beginning to end how you would use this.
43:43 And then look at your workflow, start grabbing all those like pieces of scripts, customize it to whatever you need.
43:49 And then all of a sudden you've replaced an entire workflow with a Python script that's your own work.
43:54 And after you've done that, learn a little bit more about just like generic Python.
43:59 And then even though this isn't necessarily Python related, I would highly recommend learning PostGIS and SQL because there's a lot of things that you can do in PostGIS that just aren't readily easy to do in ArcGIS.
44:15 Right.
44:16 Have a little bit of database knowledge, possibly.
44:19 Yeah, great.
44:20 Arush?
44:20 I actually probably have a lot to say on this topic, but I'll try to keep it to some main points.
44:25 I think it's a combination.
44:26 I think one of the two main points is a combination of learning and doing.
44:30 And I think you've got to balance it in the sense that some people might realize you don't do anything on titles.
44:37 And some people might just rush in and just do stuff but not really even pick up a book and learn some fundamentals.
44:42 I think you've got to kind of do both.
44:44 You've got to pick up the book and learn the fundamentals.
44:46 But most of the time you won't truly understand all of it until you actually start doing some stuff.
44:52 So you've got to kind of do that back and forth.
44:53 And a lot of times, decide to build the application for Flask while you're reading your Python book.
44:59 And then after you finish the Flask, you probably want to go back and read that Python.
45:02 Because then a lot of the things in there will start to click better for you after you've had the experience.
45:07 And so you've got to kind of go back and forth.
45:10 Almost an iterative learning style, right?
45:12 Yeah.
45:12 It's like, yeah, because I know a lot of times the stuff I read, I thought I understood it.
45:16 Or maybe you think you understand it, but you don't really understand it.
45:19 So you actually are somewhere stuck and you're trying to do something.
45:22 You're like, oh.
45:23 Then you refer to it.
45:25 And then you're like, oh, now I understand what they were talking about in that book.
45:28 And so going back and rereading some of the stuff that you're trying to.
45:32 So try to learn the fundamentals.
45:34 But don't let that from stop you trying to build stuff.
45:37 Because as I said, as everyone said here, especially if you have a specific problem.
45:41 Or just decide on some project.
45:43 And just start working on it.
45:45 And then try to read and work sort of concurrently as best as you can.
45:49 And I think that will set you on a good path to try to learn.
45:53 Yeah, that's great.
45:54 So the last one, I guess, is think about the industry that you came from originally.
45:59 Where do you see programming and things like artificial intelligence?
46:04 And just people having these programming skills increasingly over time.
46:09 Pushing the industry and changing the way that people do their jobs there.
46:13 Derek, let's just start with you.
46:15 Okay.
46:15 I've seen a lot of hype around artificial intelligence and machine learning.
46:19 And there's some bigger companies that are doing some things that are pretty significant with it.
46:23 But I think there has to be a lot of footwork on the ground level.
46:27 Maybe the average engineer is going to have to gain some more data literacy before you can get data into a format or a sensible database in order to be able to do these bigger analyses on them that really can add value.
46:39 And so I see a lot of emphasis on data literacy, maybe simple scripting becoming part of the curriculum for a lot of engineering disciplines.
46:48 And then I'd also like to see a lot more open source being used.
46:51 A lot of companies will pay a lot of money for, I would even say, subpar proprietary packages that a lot of open source libraries can do for free if you just knew that they existed and knew how to install them.
47:05 Yeah, that's a really good point about the open source stuff.
47:07 And I think open source is definitely going to drive a lot of change for a lot of areas.
47:11 Jim?
47:12 In the GIS realm, one of the things that we have always had to deal with is the fact that we almost always have more data than we can actually handle.
47:18 Because the whole world is our subject.
47:23 And so a lot of times we'll scale our data back to a lower resolution so that we can just even begin to do the processing.
47:32 So as new things come out that lets us work with bigger and bigger data sets, we already have those data sets available.
47:38 We've got petabytes of satellite imagery that just, for the most part, goes relatively unprocessed just because there's nothing that can process that much data all that easily.
47:49 Yeah.
47:50 And as the processing power gets better and people are more capable to take advantage of it, you know, larger problems will be solved, right?
47:56 Yep.
47:57 Yeah.
47:57 Very cool.
47:57 Arash?
47:58 Yeah.
47:59 Actually, you know, even though I'm not in organic chemistry anymore, I still read a few articles here and there about what's going on.
48:04 And actually one that I saw, which I was not surprised that things were in this direction, was that MIT, someone at a group at MIT, I come up with an artificial intelligence that was significantly better at predicting medicinal chemistry compounds for clinical trial type of stuff.
48:19 So basically, you know, medicinal chemists, if people don't know chemistry, are the people who sort of investigate the type of compounds that eventually will become drugs.
48:27 And that's a very slow, costly process of iteratively making compounds, thousands of compounds to even see, whereas eventually there is definitely a solution here for software to really step in and make that process faster and cheaper.
48:43 And this MIT team came up with this article about it being vastly improved as far as predicting better compounds.
48:50 And I see that's really where the direction is going.
48:53 And there's already lots of robotic automation happening in organic chemistry.
48:57 When I was there, I'm sure that's expanded.
48:59 So along with artificial intelligence, you kind of see where now computers can fairly accurately design drugs for diseases and push it through the pipeline, which, you know, hopefully should reduce, we would hope to reduce the cost of drugs and be able to discover drugs faster.
49:15 I mean, that's ultimately the goal.
49:16 Right now, the process takes anywhere from 10 to 20 years.
49:19 I mean, if we could discover a drug for disease in two years, that would be a lot better.
49:24 Yeah, that would be great.
49:26 Maybe, maybe even lower the price of.
49:28 Maybe.
49:30 Probably not.
49:31 Just raise the profits, probably.
49:32 But one thing I'm really surprised about is how much programming and automation are affecting medicine, actually, because that seems like a really human hands-on sort of thing.
49:44 But we've got, you know, AI doing the job of radiologists in cancer detection for Google did a thing on studying readmissions and, like, finding out that people are going to have to come back to the hospital sooner.
49:56 I believe this example.
49:58 Like, it's really interesting how these sort of good doctor medical jobs are almost under threat from software.
50:05 Yeah, I think it's a case where a lot of this feels like, if you're talking about medicinal chemists, I mean, the lore was you had the old medicinal chemist guy who, you know, had been making compounds for, you know, 20 years.
50:16 And he just knew which one was going to be the right compound, right?
50:19 So that had been basically an operation for a very long time just because there really wasn't an alternative.
50:25 There wasn't the data, be able to crunch data, analyze it, and do the type of things predictably that we do now that we are starting to be able to do now just wasn't available.
50:34 So you kind of had to rely on this sort of people who had just experience in the field and just sort of their experience guiding it.
50:42 But now I think we have more data.
50:44 We can analyze it.
50:45 We can predict it.
50:46 And that's going to start to replace those, unfortunately, in some level, replace those old guys with the feel of, yeah, I know this is going to be the right thing or that's going to be, you know, the right way to do it.
50:57 And it's like, no, the data is going to drive us now.
50:59 That's really interesting.
51:01 Rob, how about you?
51:02 Yes, I'm not sure from an accounting standpoint, since I'm not in the industry anymore, but I would imagine there are companies already working on, you know, the AI machine learning side of things to automate the more manual sides of doing accounting and bookkeeping and things like that.
51:20 And probably tax wise, too, I would imagine.
51:23 I think it's what H&R Block already advertises how they use IBM Watson for calculating your taxes or something like that.
51:31 Oh, interesting.
51:32 Yeah.
51:33 So I'm sure, you know, down the road, there's going to be a point where, you know, you click two buttons and all your tax data is imported and your return is filed automatically without having to do much of anything.
51:45 But we're probably still a little ways on.
51:47 Yeah.
51:47 So kind of folks can become almost like advisors more than like computers, I guess.
51:54 Yeah, I think it kind of goes back to what some of the other guys talked about where, you know, oh, you're going to take away my job.
52:00 Well, no, not really.
52:01 You're just going to work on higher level, you know, planning type of things versus the actual manual work of calculating taxes or doing keeping.
52:10 Yeah.
52:10 And that sounds like a good thing.
52:11 Yeah, probably.
52:14 All right.
52:14 So I think we have to leave it there.
52:15 We're pretty much out of time, everyone.
52:17 But I wouldn't be Talk Python without the final two questions.
52:20 So just quickly, I'll go down the list.
52:22 You guys can let me know.
52:23 So Derek, favorite editor and notable PyPI package?
52:27 I don't really have a favorite editor.
52:29 I'll use Vim or Spyder or PyCharm just depending on what I'm doing.
52:33 Notable package, I found this package called Sorted Containers, which basically I had run into some software that had used this custom AVL implementation.
52:42 And I was able to basically get rid of a lot of the compiled C code using this package.
52:47 So check that out if you're in the need of something like that.
52:49 Oh, that's sweet.
52:50 Jim?
52:50 So I spend most of my time using PyCharm if I can.
52:53 And my notable package would be Huppr, which is a great development package where if you've got like a web server running and anytime you make a change to your code, you want it to automatically restart.
53:04 Huppr takes care of that for you.
53:06 Yeah, that's really cool because normally you've got to like restart the process for it to redetect and reload the Python modules.
53:12 But I think Flask does this automatically.
53:15 All right.
53:15 You can say like --debug or reload or something and it'll watch those changes.
53:21 And maybe it's even using Huppr inside.
53:23 That's cool.
53:23 Definitely a nice feature.
53:25 Arash?
53:25 I used to be a PyCharm person, but I have switched to VS Code.
53:29 I'm really loving VS Code, so I think I'm pretty set on sticking with it.
53:33 And then for a package, I'll reroute it Airflow.
53:36 I think it's still fairly new.
53:39 I don't know how many people are using it, but if you have data pipeline workflows,
53:43 I need to do some data processing and you want to do some scheduling, check out Airflow.
53:48 It's a really good package and it can solve a lot of those kind of problems.
53:52 Nice.
53:52 It sounds like you guys are doing cool stuff with it.
53:54 Yeah.
53:54 Rob?
53:55 Favorite editor.
53:56 I'm doing, I mean, a lot of my work is, you know, Jupyter now, but if I'm doing just stuff locally,
54:03 it's sublime generally or occasionally then.
54:06 Okay, great.
54:07 And package?
54:07 Yeah, package a little one called PiperClip, which is really simple.
54:12 It can, you just give it a path or anything and it will copy something to your clipboard,
54:17 which is really handy when people send you files that you have to load into Pandas and the name is never consistent.
54:25 So we literally just copy the file name and have the path set up and you concatenate together.
54:31 It's all set to go.
54:33 Oh, that's cool.
54:34 I really like PiperClip.
54:35 I use it for some automation as well.
54:38 And just like the results will pop out and I don't want to save them to a file, but I just,
54:41 I need to paste them somewhere else.
54:43 And so the last bit in that step is just copy to clipboard.
54:46 And, you know, if I want it, it's there.
54:48 If not, like, you know, I just ignore what's in my clipboard.
54:51 Yeah, I just, I usually just use it to grab something and put it in my clipboard and grab it from there without having to paste it.
54:56 So.
54:57 Yeah, great.
54:57 Very nice.
54:58 Yep.
54:58 Very nice.
54:59 Good recommendation.
54:59 All right, everyone.
55:01 Thank you for being on the show.
55:02 This was a really interesting conversation and I'm looking forward to sharing with everyone.
55:06 Thanks.
55:06 Thanks.
55:07 Thank you.
55:08 Thanks.
55:08 You bet.
55:09 Bye.
55:10 This has been another episode of Talk Python to Me.
55:13 Our guests on this episode have been Derek Chambers, Jim Taysom, Arash Shoheli, and Rob Ward.
55:18 It's been brought to you by Linode and Brilliant.org.
55:22 Linode is bulletproof hosting for whatever you're building with Python.
55:26 Get four months free at talkpython.fm/Linode.
55:30 That's L-I-N-O-D-E.
55:32 Brilliant.org wants to help you level up your math and science through fun, guided problem solving.
55:37 Get started for free at talkpython.fm/brilliant.
55:41 Want to level up your Python?
55:42 If you're just getting started, try my Python jumpstart by building 10 apps or our brand new
55:47 100 days of code in Python.
55:49 And if you're interested in more than one course, be sure to check out the everything bundle.
55:53 It's like a subscription that never expires.
55:55 Be sure to subscribe to the show.
55:58 Open your favorite podcatcher and search for Python.
56:00 We should be right at the top.
56:01 You can also find the iTunes feed at /itunes, Google Play feed at /play, and
56:07 direct RSS feed at /rss on talkpython.fm.
56:10 This is your host, Michael Kennedy.
56:12 Thanks so much for listening.
56:14 I really appreciate it.
56:15 Now get out there and write some Python code.
56:17 And I'll see you next time.
56:37 Thank you.