#203: Beginners and Experts in Software Development Transcript

00:00 Michael Kennedy: What's it like to be a beginner in software development? How about learning Python for the first time? This episode is a special panel episode, and is the first of a two-part series we're doing on the podcast, called Beginners and Experts. On this first episode, we have a conversation between the beginners and the experts, and how we can close the gap to help the beginners get up to speed as quickly as possible. Our panelists are Karly Sindy, Joy Danton Ma, TsiTsi Flora Munikwa, and Ned Batchelder. This is Talk Python To Me, Episode 203, recorded January 30th, 2019. Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter via @talkpython. This episode is brought to you by Linode and Ting. Check out what they're offering during their segments. It really helps support the show. Tsitsi, Joy, Karly, and Ned, welcome, all of you to the podcast.

01:15 Panelists: Hey, thank you. Thanks. Glad to be here. Thank you, Mike.

01:18 Michael Kennedy: You're welcome, you're welcome. It's great to have you all here. People love the panel shows with lots of different perspectives, so I'm excited to put this together for everyone. So, let's start with just who you are, and maybe where you are in this beginner/expert side of things, and what you do day-to-day so people have a perspective, and Tsitsi, let's begin with you.

01:38 Panelists: Thank you for having me, Mike. I'm really excited to be doing this with all of you. So, I'm a student in Zimbabwe at Harare Institute of Technology, and I'm studying information security. I started programming in 2016, and I started with C when we were doing a module for the degree program that I'm doing. Eventually, we started learning Python. Eventually, I was very passionate about Python 'cause the one who introduced Python to us. He had changes the curriculum. Instead, we would have learned Java, but he changed the curriculum so that he could introduce Python to us. So, my first out in Python, I did in class, and after that, I just started going on and on learning Python.

02:30 Michael Kennedy: Yeah, that's really great. I'm so glad that person changed it from Java to Python. That's happening in a lot of education spaces.

02:36 Panelists: Yeah, I love that guy. Definitely, 'cause I don't think I would have continued with Java. It's not my favorite for me.

02:44 Michael Kennedy: For sure. Alright, that's wonderful, and what do you do day-to-day? You're still a student? Are you a grad student at the school there?

02:50 Panelists: I'm still a student. I'm going for my senior year next year, but currently, I've been interning at Dimension Data, and I've been with them for a few months now. My main work task at Dimension Data includes automating tasks and developing learning hubs. So, what we do is write Python scripts. All of the work that I've been doing there I've done in Python, so it is great. We also do a lot of integrating with APIs, which is a new technology for me, and I've been learning a lot from Dimension Data, and everything that we do there.

03:29 Michael Kennedy: Yeah, how exciting. Dimension Data's doing amazing stuff with getting people into programming, and that's how I know you. You and I worked together a little bit on some of those projects, so that's great. Joy? Yeah, how about you? Tell us about yourself.

03:41 Panelists: Thanks for having me, and my name is Joy, Joy Ma. I am at a product manager at the Paulson Institute in Chicago. So, we're a think tank focused on international economics, and when you think about think tanks, there are the stereotype of a think tank is usually in DC. As a matter of fact, there are over 400 think tanks in DC. They have a lot of former politicians. They have a lot of former diplomats, and they have amazing experiences. And based on their personal experiences of sometimes 60 years, they make policy recommendations for upper policymakers on the Capitol Hill. But our think tank is pretty young. We were established in 2012 by former Treasury Secretary Hank Paulson. We're trying to think of a new way to establish a name in this already very crowded field, and that's when I started to venture out and look at all of the different options to do policy analysis. Back in 2016, when I first started looking at international trade policy and international investment policy, that was a moment I saw they are just half a decade of data freely, publicly accessible out there, but nobody has ever done any sort of research into it. So, I thought, being a relatively younger person, without a 60 year diplomatic experience on my belt, I would just look into 60 years worth of data and see what I can come up with, and that's how we conducted our research at Paulson Institute, and has been pretty successfully, I would say by institute, by think tank standard. A lot of our research did get published on Wall Street Journal, New York Times, and we certainly also made our fair share of contribution to policy making recommendation.

05:40 Michael Kennedy: That's really great. So, it sounds like a lot of data science. Probably a lot of Jupyter notebooks and analysis like that, yeah?

05:46 Panelists: A lot of Pandas, a lot of cleaning, a lot of JSON, too. Sometimes it's pretty painful.

05:52 Michael Kennedy: I can imagine. Cool, cool. That's excellent. Welcome. Karly, how about you?

05:57 Panelists: Yeah, hey. Thanks again so much for having me on the show. I'm excited to be here. I'm representing part of the beginner side of this conversation. I have my master's in molecular biology, and I worked doing scientific research for about 10 years. I was working in an epigenetics lab, engineering a lot of sequencing data, which are essentially, flat files with a lot of strings or sequences composed of A, T, Cs and Gs, and some metadata. I needed a way to analyze all this data I was generating, and there are public websites, but they can be super slow when you're trying to run these files when they get really large, or a lot of these files, so you really need to run them at the command line, run these tools, and generally, while you're SSHed onto a larger machine that you have on your personal machine. So, that introduced me to a whole lot of Bash, and some scripting, and then we started designing experiments that are a little bit more complex, so it required a lot of modifications to these files before we sent them into the command line tools. And so, that's when I started looking into Python, and I took a class called Python for biologists, and this is what really opened my eyes to how many problems Python could solve that I was faced every day. Just little problems, so I started building tools for the lab where people could drop Excel sheets into it, and it would just parse, and figure out, and do some things that would make our life easier day-to-day. And so, after building some of these tools and doing these analyses at scale, I decided that I wanted to do that full-time. And so, for the past two years I've been working full-time at a company that makes software for life science researchers and for their daily workflow, so it's kind of a perfect fit there. So, in being at a really small startup, I feel like I do a bit of everything day-to-day. I look at code, and improve merge requests, and write code, and modify code, and test code. I also wear a bit of a product/project manager hat, and define projects, and research potential tools, and if they're out there, maybe we can containerize them, or maybe build our own. I mean, I also even talk to customers and see where we can be meeting their needs with the code that we're writing.

08:05 Michael Kennedy: That sounds really interesting. I think this is one of the situations where people have some skill already. Like, really solid skill like you have molecular biology, right? And programming makes that better. It's not like, oh, forget this biology. I'm just going to go become a programmer, right? I want to do React with JavaScript, right? It's just like, how do I do what I'm doing better, yeah?

08:28 Panelists: Yeah, absolutely. I was trying to be affirmative, assertive. Yeah, yeah, yeah. Yeah, I think, I mean the same thing. I think that Joy, like you were coming from the economy side of things where it's like being able to integrate the knowledge you already have and just enhance the knowledge you already have, and just see those problems that you'd seen in the past, however many years you'd been in that field, and then all of a sudden be able to solve them with some code is really exciting. Absolutely. I was trained as an economist in undergrad and graduate level studies for, God, almost a decade. And the most available part of the training is to help you identify questions that needs to be answered. And it's with Python for the first time I feel I have so many tools to answer those questions, but as you said, Karly, sometimes you're wearing the product manager hat, and you need to identify what is the need of your client? And in my case, I need to identify what is the pain point for policy making, and use the domain knowledge we have we can identify those points very specifically, and then use the tools such as Python, we can solve it so much more efficiently.

09:38 Michael Kennedy: Yeah. Absolutely. Ned? You want to introduce yourself, as well?

09:42 Panelists: Yeah, so let me just start off by saying that as a beginning in both molecular biology and policy analysis, I'm a little intimidated to be on this panel with all these experts.

09:52 Michael Kennedy: Absolutely.

09:53 Panelists: So, I'm a software engineer with a long career. I'm deeply embedded in the Python world. I organize the Boston Python User Group. I've given PyCon talks at, I think, 10 different PyCons. I'm a maintainer for coverage.py. I hang out in the Python IRC channel. I really need to find another hobby in my life of some sort. I do everything with Python.

10:16 Michael Kennedy: Yeah, you and me both. I'm not going to introduce myself, but I'm both a beginner and expert, just as you said, Ned.

10:21 Panelists: Mm-hmm. Exactly.

10:23 Michael Kennedy: Certainly in the biology and the policy side of things. So, I want to start this conversation off by talking about something that you wrote, Ned, highlighting the beginner's and expert's perspective from somebody who's been doing it a long time, right? When somebody's a beginner, they are just always bumping into sharp edges of the technology. Oh, why does this not work? Oh, why won't this thing install on this machine? And all that kind of stuff, right? But you talk about how well in some ways those sort of feelings and those challenges don't go away, so maybe you want to quickly summarize your beginners and experts article or essay?

11:01 Panelists: This got started almost two years ago when I was working on a side project, and as I talked about various aspects of that side project, first at a lightning talk at work, and then I wrote a blog post about something I was struggling with about it, and I realized that over the course of that summer as I was working on that project that I was feeling like a beginner, in lots of ways. There are aspects of it that I knew exactly what to do, but there were aspects of it where I was Googling on Stack Overflow to figure out how do I make this thing do a thing? And that beginners often feel like that they are having trouble because they are a beginner, and that their plan is I will learn everything and eventually I will be an expert, and once I am an expert like all these other experts I see, then I won't have those problems anymore. I will know what to do, and I won't have these confusing mornings where everything seems wrong and backwards, and I can't make anything work.

11:56 Michael Kennedy: Just nothing works. Why does this not work?

11:58 Panelists: Why does this not work? Right. Michael, what you just described, right? The sharp edges, and why doesn't this install on that machine, I don't know how you knew what I did this Monday, but yes, that's the way my days often are. So, beginners think experts know everything, and there's no friction in their development tool chain, and cycles, and all that stuff, but what you'll know once you become an expert is that experts don't know everything. Experts are constantly encountering new things. Joy, you just mentioned that you do a lot of Pandas. I have done maybe two things with Pandas, and one of them was literally last night, and I was Googling on Stack Overflow to ask, how do I know what the names of the columns in a Pandas data frame are, right? Which is super basic because I don't know anything about Pandas. Oh, Ned. Countless nights of my youth, spent looking at Stack Overflow. Yes. Exactly. And the thing I was trying to drive home was I eventually wrote this blog post, and I did a talk at Boston Python about it, and the point I was trying to drive home was that, although I stand before you as a sort of acknowledged expert in Python, I often have no idea what I'm doing. And the real difference between beginners and experts isn't that beginners know nothing and experts know everything, but that beginners think that this feeling is either going to last forever because they're dumb, or it's going to eventually go away because they've learned everything. Whereas experts know this is a very familiar feeling. I have been in situations lots of times where I don't understand what's going on, and because I've been in that situation lots of times, I know I'm going to get through it. I have some tools, I have some strategies that I can use to get through this difficult part, and end up back again in a place where I know how to do things. Either because I've put that problem behind me, or because I've learned the solution to that problem and I can now apply that solution. One of the bumper sticker slogans that I'm thinking of using about this is that an expert is just someone who's been a beginner many, many, many times. Over and over again the expert goes back to being a beginner and comes out of it, and so has cycled through that learning curve over and over again.

14:08 Michael Kennedy: That's super interesting, and it's basically you get really good at, alright, how do I interpret this error message? What places was I able to Google, or search, derive what the answer is? Where did I find an example that I could borrow and adapt? Those are the kinds of things you get better at, and then the calmness of like, yeah, this is really frustrating, but it's just like last week, and I know by lunchtime it'll be fine. We'll figure this out, but right? It doesn't go away, yeah?

14:31 Panelists: Right, or I know that this is a thing that I really do have to master, so I'm going to use my good learning strategies for it. Or I know this is a problem that I just need to put some duct tape over right now, and then I can forget about it, so I'll just use my quick hack strategies for this.

14:44 Michael Kennedy: There you go, Stack Overflow for that one.

14:46 Panelists: Yeah, I never use Stack Overflow for the real learning strategies. No, I would never do that.

14:50 Michael Kennedy: Never. Of course not. That would be wrong, Ned. Tsitsi, let me ask you. How do you look at experts? Do you feel like, oh, these people who have been doing this for a long time like Ned, they've just got it all figured out, or do you connect a little bit more with his perspective of what he's writing here?

15:07 Panelists: Yes, I do understand what Ned is talking about. In my perspective, sometimes I feel like experts do know everything because they have been, there are some people who have started coding at like nine years old. It's hard to believe that that person still has those frustrations when you started coding when you were nine years old. In my view, I would be seeing you as someone who knows everything about coding. Every error you come across, you'll be able to fix it in a minute, but as I've been growing in coding, I'm seeing that as I learn more, as I learn new things every day, those frustrations keep coming back. I keep going through the same process again of encountering an error, and then having to fix it, getting the frustration, and everything. So, expert or beginner, we are more of the same on the same level if we try to look at it. Just like Ned said, experts are beginners who have been going through the frustrations over, and over, and over again.

16:15 Michael Kennedy: It's true.

16:17 Panelists: Yeah, I think not unique either just to developers. I feel like scientists make good programmers because we're so familiar with failure. Because when you set up experiments in a lab, a negative result is still a result, too, but you're getting so many negative results. It's so infrequent that you have a eureka moment, and so, but I feel like that kind of helped me a little bit with programming, too. It's like, I try something and it doesn't work. It's like, well that's, you know, just check that off. Move on to the next thing. A negative result is still a result.

16:49 Michael Kennedy: That's right. It's kind of like the Thomas Edison I've learned 3,000 ways not to build a light bulb.

16:53 Panelists: Exactly. Keep going, right? That is really interesting when you say how you know the learning strategy and the tools so that you are confident that you are going to be able to solve the problem either today or tomorrow. And for me, in data science, we talk about this black box. For me, the path to gaining that confidence is almost like a black box to me. As a beginner, we learn very practical skills. We learn all commands in Pandas here and there. But from here, to having the confidence to solve a problem that's never been encountered by us before, gaining that confidence for me, is definitely something very interesting to learning more about. It's hard to get to that point, right? To feel like you will be able to tackle whatever problem you encounter, whether it's a molehill or a mountain. And honestly, I think one of the good strategies is to not try to solve everything. I've seen some beginning learners who they'll put it this way. They want to understand everything. Everything that either mean very, very broad, right? I want to read the entire Python standard library. I want to know all of it, which it's just too big. You can't do it. Or they mean very, very deep. I want to know how Python's implemented. Like what's really down under there? And sometimes, I've seen people get taunted like, so you want to know how transistors are made? How deep down are you really talking about here? Beginners, I love that optimism, and hunger, and voracity that a beginner like that can have, but they're going to have to realize that they're not going to be able to do that, and at some point they're just going to have to say, there's a thing right over there I could learn about, and I'm just not going to learn about it because I have to go and do this other thing over here. That's another kind of strategy.

18:34 Michael Kennedy: I know enough to solve this problem. We're good. Yeah.

18:36 Panelists: What you just said is both very reassuring to a beginner like myself, but also very intimidating. Weirdly, it's the same time. One of the things I said in the blog post is the good news for beginners is that experts aren't significantly different than you. They've just had a lot more experience. But the bad news for beginners is this bad feeling you have where you're confused, and mystified, and frustrated, you're going to have that for the rest of your career. I mean, I've already run into that. I feel like, even I've been doing this, I guess, for two years officially, and I've run into that many times, and each time I hit that where I'm like, I've started coding something and I'm like, oh no. I don't know if I can actually do this. But then somehow, I take a step back, I go back, and then actually getting beyond that feeling like the next time I hit it, I'm like, alright, I already had this terrible feeling before. I know I can get past this. So, I think maybe just the dulling your senses to that feeling of oh my gosh, you know? This is a huge hill to climb. Right.

19:42 Michael Kennedy: Yeah, that's really interesting, Karly. I remember when I was first, the first couple years I was working as a professional developer I didn't have much experience before, so it was both awesome and super frustrating and scary at the same time, and I remember often taking 15 minute walks around the office park. I'm just like, you know what? I just got to step away for a minute. It's sunny outside. I'll be back, and I can clear my mind and do this. I guess I don't really do that, that much anymore.

20:09 Panelists: Oh, wow. That's good to know.

20:10 Michael Kennedy: Yeah, yeah. I think you get less emotionally frustrated, but you still are bumping into these sharp edges. You're just like, why does this not work? I've done this for a year and it's the same. Just like, what is going on today?

20:23 Panelists: Right.

20:24 Michael Kennedy: Yeah, one of the big challenges I think that, Ned, I'd love to hear your thoughts on this, but I'm going to ask the ladies first, one of the really big differences I think between beginners and experts is the beginners have a harder time deciding how to start to solve a problem. As an expert, you can look at something and go, oh, that needs, we got to parse this file type, and we're going to use the database, and probably we're going to use an ORM, and we're going to wire it together like this. And then we go and bump into the sharp edges of those things, but we kind of know roughly how to attack the problem. And I think that a lot of beginners are like, I don't know how to start. I see what you did. This was amazing. I have no idea how I would ever come up with that idea to get there. Does that resonate with you all?

21:07 Panelists: Definitely. I mean, this is just kind of when you're given a problem, and you're just like, oh, okay. Now, where do I start? When you're taking an introductory class, or any of those, it's like, this is what we're going to do and this is how we're going to solve it, you know? I mean, I know that last week I was trying to solve a problem and I ended up coding probably 200 lines of code, and then I was going to set up kind of a sanity check with a colleague. And I was thinking about how to explain this problem to that person, I realized a much simpler solution. So, I basically got rid of all that, and then just was able to solve it with a few lines of code. And I think kind of having that perspective, oh, stepping back, and knowing the tools to solve it, if you've already solved something indirectly that's similar to it, and I think that was my issue last week was I had never solved anything similar to that problem before so I didn't know the tools to be able to solve it in the most efficient way. So, I kind of spent a long time making a really long solution to it, and then realizing that there was a much more direct route. Yeah, I think that familiarity with seeing things done one way, and then just dealing with the little details is definitely tough, being a beginner because you haven't seen a lot of examples of the way a problem is solved.

22:28 Michael Kennedy: This portion of Talk Python To Me is brought to you by Linode. Are you looking for hosting that's fast, simple and incredibly affordable? Well, look past that bookstore and check out Linode at talkpython.fm/linode. That's L-I-N-O-D-E. Plans start at just $5 a month for a dedicated server with a gig of ram. They have 10 data centers across the globe, so no matter where you are or where your users are, there's a data center for you. Whether you want to run a Python web app, host a private git server, or just a file server, you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network, 24/7 friendly support, even on holidays and a seven-day money back guarantee. Need a little help with your infrastructure? They even offer professional services to help you with architecture, migrations, and more. Do you want dedicated server for free for the next four months? Just visit talkpython.fm/linode.

23:20 Panelists: From my experiences, when I first started learning Python about two years ago, it's relatively easy to learn about the specific tools. To learn about Pandas, to learn about after I created programming to be able to write a couple of line of codes, and make it up and running, but the real problem for me, and this might be just beginner's ignorance at this point, I think the tools are relatively easier to acquire, and there are concrete examples out there. And if you run into a very specific problem, there are people out there you can ask them. There is Stack Overflow. But the real problem comes in for me when it comes to the structure. How do I arrange the tools in my toolbox to identify, solve and reassess my own solution to a specific problem? And that's when, I think it was roughly two years into learning Python by myself, I started to realize, okay, right now the real problem for me is not to solve one specific problem to get a name of the column, but to actually identify the problem and put in a framework that I know how to interact the features and how to solve it. And that's when I started for the first time, I was just taking a lot of online classes for the first time. I don't know if I'm supposed to say this, but I joined data science training at general assembly, which gives you a much deeper understanding about just ideologic from structurally speaking, how to think about the problem as a data science problem, as opposed to economics problem, which I'm very comfortable with, but it's not the best way to solve to get answers that we were looking for. Joy, that's exactly, I think being able to become familiar with the tools is something that is really helpful 'cause knowing the tools that can help solve your problems is really important. If you don't know a general idea of what some libraries or tools can do, then you might like me, start writing it from scratch, and then be like, oh, there's something that I can... And the more you're around, and you have those water cooler conversations, and you hear about tools that other people are using, and then you can kind of apply those tools to problems that you have. I think it's kind of exposure to the domain, I think can really help speed up answering problems. But that's, I mean, that's tough as a beginner 'cause you don't have that exposure yet.

25:53 Michael Kennedy: Yeah, and it's super hard to learn it. It's not very well taught anywhere, I would say. Tsitsi, how do you feel about this idea of how do I even get started solving a problem is maybe the hardest problem?

26:02 Panelists: I can relate to what you're saying, Mike. There are times that I come across challenges that I do not know where is it good to start solving the challenge. When I started working at Dimension Data, I came across this site, the PyBites site where they have challenges. Their challenges are also awesome, and I started taking those challenges. I started with the beginner challenges, of course, and when I read the challenges, some of them are so simple that I can think I can do this in like, one minute. But when I go into the edit and try to write some code, my brain freezes, and then I have nothing at all. Yes, that's hard sometimes. But I continue taking the challenges on PyBites. I realized that after some period of time end, I can relate to other challenges that I have done before, and be like, oh, last week I did these combinations. I think I can apply that to this challenge. That's how experts feel like. They have exposure, just like Joy said. They have exposure to more tools to use, and what you can do to solve challenges, which I think the first step of deciding what to use when solving a challenge is the most important when you're facing a challenge, rather than just jumping in to try and solve the challenge without even thinking through what tools you can use to solve the challenge.

27:27 Michael Kennedy: That's really interesting, and I think you doing the PyBites Challenge makes a lot of sense. Bob and Julian are doing great stuff with their co-challenges platform. I think that practice helps. I do want to say I feel I totally agree with Ned that as experts we run into these things where it's like, wow, this is so frustrating. Why doesn't this work? Why can't I make this silly thing just do whatever it's supposed to? And it feels very similar, but what I think is very different is when you approach a problem, you're like, I see. I've solved this kind of before, and we did this major thing, this major thing, and that major thing, and it kind of clicked. And you can sort of break the problem down into these smaller bits and you have this history of I've kind of solved this data problem four or five times like this. It's probably like that. Let me try that first. I think those building blocks are a huge differentiator, and that gets way, way easier. Ned, what do you think about my idea here?

28:17 Panelists: No, definitely. As someone with a long career, I've been in lots of different projects and written lots of different code, and so I have sort of a long tail of patterns that I can match against, right? Look at a new problem, this reminds me of something. I can either just remember what I did before, or I can literally go and find the code. Often I can do something quickly at work because previously on a side project I had to solve that problem and I could do it at my leisure on a side project, but then I've got the code in place that I can just go and grab for a work thing. But I have to say, Tsitsi just said that she would read PyBites, Python bites, think, oh, I can do this, and then she gets in her editor and her brain freezes. That happens to me, too. Exactly that feeling. I can't tell you how many times I'm waking up in the morning and I think, I know what I'm going to hack on before breakfast. This is going to be great. And then I get into Vim and I'm like, you know what? This is actually hard. I don't know what to do right here. Maybe I was just still in kind of that foggy dream state and I thought it was easy, but right now it's actually hard and I'm not going to make the kind of progress over this cereal bowl that I thought I was going to make. That's just programming. It's really hard to be specific enough to tell a computer exactly what to do. It just is. And so, sometimes you think it's going to be a thing, and it turns out to be harder than that. I think a lot of it, the pattern matching is really the way I think of it, in terms of going back into your experience. I was working with an intern last summer and we were debugging a problem, and I said, I think it's going to be this is the solution, and he asked me like, how did you know that was the solution? I felt like, well, in the logs there was a zero, and over here there was a none, and zero and none are kind of the same, and they were both kind of out of place in their each niche, and so I kind of put those two things together, and what would make that happen. And so, there's a lot of pattern matching in solving problems, either to decide how to get started on a thing, or to debug a thing, and just going through it a number of times over the years gives you more of a sense of sort of what's the anomaly in this environment that I should be focusing on to debug the problem? Or how do I break this big problem into smaller chunks, two of which are familiar, and one of which is unfamiliar, so that I can just do the familiar ones quickly and then focus on the unfamiliar? And that kind of cognitive pattern matching I think is a skill that you develop over time.

30:42 Michael Kennedy: You definitely develop it over time. What I've been wondering and kind of searching for the last six months is how can you fast track that? What I've been wanting to do is build a class that basically says, here are a bunch of problems. How as an experienced person do you go through the idea of even starting to attack the problem? You almost forget here's the details of an ORM or an API. It's just like, how do I even begin to put this into parts that then I can attack with code? What do you think, Ned?

31:15 Panelists: For instance, what I just said was, oh, this is the anomaly that I'll focus on. Well, to a beginner, the entire log file is new, and different, and exotic, right? So how do you know?

31:24 Michael Kennedy: It's a scrambled mess. How do you know what's important?

31:25 Panelists: What's the regular, normal part of this gobbledy-gook? I don't know. So, yeah, that is a thorny problem, how to get that kind of facility with the normal versus the weird in our environments.

31:39 Michael Kennedy: Yeah, I think it can be done, but I don't really quite have a great sense on how it is. So, somebody presents a problem to you like, hey, I need you to get this data that we got off of our experiment, and I want you to clean it up, and analyze it, and show it in this way. Something to that effect. Conceptually, that's really easy, but when you try to actually look at what are the actual steps my program has to do? What are the actual big ideas like, okay, we're going to read this file. Okay, we're going to try to transform this data. Just those big steps I feel like are the challenge. Once you identify them, then it kind of becomes an easier problem to solve. But how do you help beginners learn to identify those, sort of break the problem down like that, do you think?

32:20 Panelists: One of the problems I think beginners have, just to jump in, is to think like a computer. We're not used to having to think so specifically about each tiny, little step that we can then write the code that does what we need it to do, right? A lot of beginners, I'll say, well, tell me in English what it is you want this program to do, and they'll give me one sentence of the whole thing. It's got to browse the web. Well, there's little bits and pieces in browse the web, right? So let's break it down. And that's very difficult for some people to do because we're not used to thinking that way, right? I can say to a person, go to the store and get the milk, and they're like, okay, I'll go to the store and get the milk. I didn't need to break it down into all the little pieces. You can find videos online of people playing with their kids. I'm a robot. Tell me how to make a peanut butter and jelly sandwich, and the kid tries to tell him what to do, and the dad just literally follows the instruction and makes a total mess of the thing because the kid's not used to explaining how do you get the lid off of a peanut butter jar? The computer is that dad.

33:21 Michael Kennedy: This portion of Talk Python To Me is brought to you by Ting. Let me tell you about Ting, a new mobile service available in the US that's targeted developers and other technically savvy folks. First of all, their average customer only pays $23 a month, but they're a no discount provider. Their service runs over T-Mobile's and Sprint's fast, nationwide network. If you don't use that much data because you're usually on WiFi, like many of you are, then Ting will save you a ton of cash. But don't worry, you can still use as much data as you'd like for just $10 per gig. One mobile feature I use all the time is tethering, and with Ting, you get unlimited tethering at the same data rate with your account. $6 a month for a phone line, $10 a gig, $3 a month for texts if you usually chat over iMessage or WhatsApp. Think about it. No contracts and super clear and fair billing. Visit python.ting.com. That's python.T-I-N-G.com, and check out their savings calculator. Enter your usage and see exactly what you'd pay. Use that link and you'll get a $25 credit to try them, as well. That's python.ting.com, or just click the link in the show notes.

34:25 Panelists: I mean, that's a really interesting way to put it, where you're like, I'm just going to go to the store, but you're like, get in the car, and do all these things. So, it's kind of when you say that, I kind of picture like a flowchart in my head. So, with a problem generally, is there's some type of data. Is it in a file? Is it a stream? Is it from the web? And how do you access that data? Where is that data going? So, kind of following the trail of the data and what needs to happen to that data at each point really dictates the way you will program something, whether it's in a file, or whether it's in a stream. So, I think kind of following the trail of data, and then kind of making the choices on that can help break things down. Right, and one of the fundamental characteristics of the software world that we've put ourselves in is that there are unlimited levels of complexity, and so we deal with abstractions all the time. And another challenge for beginners is accepting an abstraction, and using it. And in a way, breaking down a problem for a computer is all about setting up those barriers between the two halves of the problem and dealing with them separately, rather than thinking of them as a whole, right? So, when we said go to the store and get some milk, well, step one is get in the car, step two is drive the car to the store. That step two is awfully complicated, right? Ask the people who are building these self-driving cars right now. So, there's an abstraction there that we're willing to deal with, right? You turn the wheel, the car turns, right? Sometimes you have to debug that abstraction. Sometimes you just have to accept that abstraction. And moving up and down those levels of abstractions and just ignoring what's underneath some of them, but debugging what's underneath some other ones, it's a huge maze of distractions and complexity that beginners can get lost in. So, that's just another skill. I think it sort of touches upon the point I was talking about before. The really difficult part for a beginner may come from how to extract structure and abstraction out of a very complex problem. When we're talking about making a peanut butter sandwich, then for a beginner, he or she doesn't necessarily know what are the steps to do so. Once you have the steps to do so, you can Google how to open the peanut butter jar. You can Google how to put the bread into the toaster oven. I think the difficult part is actually building the structure. Having the mental structure to know which step is after another one. And for me, personally, it was about two years getting to Python studies it started to click for me. To some extent, I feel like the building the structure of a solution to a problem is just like any type of analytical thinking. You need to identify the problem, and then you need to have five to seven steps to resolve a problem, and then going into each of every single step to look into the tools, the practical solutions to every single bottleneck that we need to overcome. For example, I'm an economist, and I was looking at US-China trade war, and people keep asking this million dollar question, which is what is going to be the cost of the trade war? Is the US economy going to be impacted? And it's such a great question, but it's so complex, but then once you break it down into how about let's look at a first industry, the second industry, and the service industry, and look at the impact on all three industries by the trade war, and then you have a structure to go into there to find the data, to extract information out of the data. And once you compile them together, you're answering an awfully complex question.

38:05 Michael Kennedy: That's a good way to break it down. Ned, something that came to mind as you were describing these being willing to accept an abstraction, right? You don't actually care how the CPU makes it work. I just know if I type this and I hit run, it runs, right? I'm willing to just not even worry. It's like there's a gray layer that I just am wiling to kind of accept. I think that those layers of abstraction happen a lot in programming at different levels, right? I'm willing to use that ORM and I'm willing to call save, and I don't care how the database works. Or I need to dig in and understand the full third normal form theory of relational data, and get obsessed about that, right? There's totally different levels there and I think one of the challenges when you're first learning is what is the right level? When can I ignore the nuance details of the database, and when is that critical to make that right? So, choosing the level of abstraction I think also tells you what kind of tools, and APIs, and packages you can use, even.

39:08 Panelists: Right, well, and the fact, I mean, you've already chosen Python, right? Which means that there's certain kinds of things other programmers in other languages have to worry about that you don't have to worry about.

39:17 Michael Kennedy: Right. malloc, anyone?

39:20 Panelists: Exactly. Right. You don't have to allocate and deallocate memory. And if you're working in data science like Joy is, then you've got Pandas, and Pandas provides you this really high level abstraction of a data frame that does all sorts of things that five years ago before Pandas was around, if we wanted to do that analysis even in Python, we'd be writing 100 lines of code just to sum and average, or whatever it is we need to do.

39:41 Michael Kennedy: Right, you try to build it with dictionaries, and lists, and other more fundamental things.

39:45 Panelists: Right. Exactly. You have to decide what level you're going to be working at, and sometimes you can go down a level just because you're fascinated by it. Maybe you want to know how transistors work and how a CPU is made out of transistors. But if you're trying to write a Python program and you can't get it done because you're obsessed about transistors, you're not going to be able to finish the Python program. You have to leave that to someone else. You have to assume that's going to work.

40:07 Michael Kennedy: Yeah, it's fine to be obsessed about some nuance detail, but you shouldn't conflate it with trying to solve your problem if it's not relevant. And I think making that distinction is the challenge. Yeah, yeah, yeah. Tsitsi, how about you? How do you see trying to find the right level to think about these problems? You talked about working with APIs, for example. That takes a lot of getting your head around, and you can either say I call this function and the data comes back as JSON, or you could say, well, there's all this HTTP headers, and all this exchange. Like there's a lot of details you could get into. How did you think about working on those problems?

40:44 Panelists: Well, for me, I think that there really is no problem when you're facing a challenge to write what you're supposed to do down. Like what you learn when you're learning the basics, the flowcharts, and how to do algorithms. I think even when you are an expert, it's okay to just jot down what you're supposed to do and write all the simple steps that you're supposed to do, and in those simple steps, decide exactly what you're supposed to do. Like what Joy mentioned, those are the steps that will guide you into the challenge that you're facing. For me, I've been learning to do that. When I face a challenge that I really cannot solve within five or 10 minutes, maybe after some deep thinking I'll write down what I'm supposed to do, and those simple steps, I work on each one by one. If we can all adapt on that concept, I think it'll be easier for everyone. Because when you're a beginner, it's not easy to go and Google your challenges on Stack Overflow, and just get it straight away from Stack Overflow, and a piece of code and cut and paste it into your editor. But then the problem is when you place code from Stack Overflow, that code is going to bring more bugs. Rather than going step-by-step and understanding each step that you're supposed to do, I think it's an easier way, and I've learned that the hard way myself, because I did projects where, I've done projects where most of the times I just need to take code from Stack Overflow, but at the end of the day I would still be having a ton of bugs that I have to debug for the code to work correctly.

42:39 Michael Kennedy: Yeah, the danger with Stack Overflow can be, as you know, it might solve the problem, but then it has all these other issues. And if you take too many of those pieces together and don't understand what they do, you can end up with something that's really hard to take care of 'cause you're like, if I change this, it breaks, but I just don't really know why. That can be super frustrating. This touches a little bit on what Ned talked about earlier, where sometimes you know this part doesn't really matter that much. I just need this to work, and I'm going to never worry about it again, so I'm going to just grab that thing from Stack Overflow, jam it in there, and we're good. And other times, it's really important. Ned, how would you help people make that distinction?

43:18 Panelists: That can be hard, too. I mean, that unfortunately, requires predicting the future, which no one knows how to do. So, I mean, I was dealing with this a few days ago. I was trying to automate creating AWS instances. I was stuck in Googling finding things that didn't work, that didn't seem to go together when I thought they'd go together. I didn't understand. At the time, I knew that I just needed a little thing, and if it could just do the thing, it could be crappy, and I knew exactly who at work I could go and talk to when I wanted to make it nice. You kind of have to decide, what do you want to be an expert in, and what don't you want to be an expert in? I'm personally not, I don't want to be an expert in devops. I don't want to know everything about how to make servers on AWS. I just need sometimes to have one, so I'm not going to dive into that, right? And it doesn't interest me.

44:05 Michael Kennedy: What config setting do I need to put here to make my website run? Okay, I'm putting that. Thank you.

44:10 Panelists: So, one of the things I've learned in my long career is that almost everything when I approach it as a new thing and I've got a mental model of what that thing is going to be like, it's always at least three times more complicated than I thought it was going to be, which it makes sense. Like, why if I know nothing about a thing will I have an accurate mental model of that thing? But I'm constantly surprised at just how much more complex real things can be.

44:32 Michael Kennedy: Things generally tend to be way more complicated than you estimate them to be in so many ways, and you just kind of got to accept that, right?

44:39 Panelists: That's right.

44:39 Michael Kennedy: We're kind of getting near the end, and I want to ask a couple more questions. So, let me start with this one. As people who just learned to program, or are just really getting your feet underneath you as programmers, what advice do you all have for maybe other people who are thinking of getting into programming, or they're also close to the same level as you? Joy, let's start with you.

45:05 Panelists: I think keep going. Keep going is the number one advice I'd give to anyone who is starting, just started to learn programming. It's very easy to feel frustrated, and just give it up after one try, second try, third try. I wasn't the only one started programming two years ago, and I think I'm unfortunately, I'm one of the few who are still doing it. And one thing that really helped me was to join this, it's called ChiPy, Chicago Python User Group. They have a weekly meeting, and sometimes I go just with a bunch of friends who are full-time developers. They work on their set projects in coffee shop, and I just joined them. And not necessarily I'm going to learn, I'm going to bug them a lot, or learn a lot directly from them because we work on completely different set of problems, but just having that schedule. Having that mental space that every Saturday, one o'clock in the afternoon to five o'clock in the afternoon is data science time, is Python time. I think for me, that's at moments of frustration and moments of, why don't I just give it up already, those are really the things that kept me going.

46:20 Michael Kennedy: That's an interesting point. I think having somebody to ask those questions of, even if you don't bug them a lot, just every now and then be able to go, you know, I've been stuck on this for six hours. I know it is easy. Could you just tell me what is the thing I'm doing wrong? And a lot of times, if you have somebody to ask that question of, I think it could make a big difference. 'Cause you could go to Stack Overflow. People are going to be mean that you maybe don't know how to phrase the question quite right. It can be really frustrating. But having just a little bit of a community like you talked about, is nice. Tsitsi, how about you? Advice for people who are getting into programming?

46:55 Panelists: I think the most surprising thing that I learned when I go into programming is how simple it is. Before I started programming, I thought programmers are people who have some level of IQ, or some superpower, but then I just realized that you can just pass commands to the computer, and then it can do exactly what you want it to do. So, for those who'd like to start programming, I think I'd just say you should just go for it because anyone can do it. Anyone can be a programmer as long as you have a passion for it and you really want to achieve something in your life. Mostly to make time for programming, because I think that is the most important factor, to make time to practice your programming. It is difficult for most people I know, and most people struggle with making time to practice their programming. So, making time is very important so that you don't forget what you learned yesterday, and then you have to start over again. And I'd like to support Karly, as well, on joining communities. That has helped me a lot. I have a lot of communities that I am a part of, and I get to ask questions, and people , and that is very helpful. Being a part of a community with like-minded people who are doing the same thing you are doing is awesome.

48:17 Michael Kennedy: Yeah, it makes a huge difference. And I think I really connect with this idea that you just talked about, that before I was a programmer, I looked at people who were programmers, and I'm like, wow. Now, that is a smart person right there. There's no way I can do what they, there's just no way. But as you get into it, what I've realized is to be really good at programming, and Joy, this also touches on what you said, I feel like persistence. How did I know to do this thing to make the program work? Well, because I tried 15 other things, and this was the 16th, and now it worked, and so problem solved, right? If you just go, well, I tried it and it didn't work, so I quit, it just doesn't work that way. Like you said, Karly, it's kind of like an experiment. You have to try a bunch of stuff, and eventually, something will work, but it really requires persistence, and I think that that's more important than being super smart.

49:09 Panelists: Yeah, exactly.

49:09 Michael Kennedy: Yeah, Karly?

49:10 Panelists: Yeah, I definitely agree with all of that, for sure. Just kind of ripping the bandaid off and solving a problem that you're interested in, and just jumping in on it. Also, going to groups, like meet ups. 'Cause I went to a Python meet up a lot, and still do, and just being around people who are taking their evenings, they're going to be really supportive, and excited to share and answer questions. And also, when I kind of related to that, when I started at my job, I know I struggled a little bit feeling like I wasn't contributing, and still just trying to get familiar with the code base. And I ended up just reaching out to one of my colleagues, and just, I think, showing genuine interest in what your colleagues are doing in their domain of expertise, I think that's really, really helpful for a beginner. I specifically worked with our head of security, and helped develop static and dynamic code scans that would run at build time. And this was a win-win because I was able to contribute early on, and my colleague was able to see progress being made in an area that they were interested in. Even if it's just for a short time, having that moral support, and camaraderie, and mentorship with someone at your company can be really, really helpful to kind of get off the ground at a company.

50:24 Michael Kennedy: Yeah, absolutely. So, leverage the relationships you already have to kind of get a mentor, almost. So, maybe we could round this conversation out. We're getting long on time here. We don't have much time. Maybe a closing thought here, Ned, on one of your other articles. You talked about toxic experts, and I don't want to necessarily take all the time to go into too much detail, but your conclusion was a lot of the experts bemoan the fact that there's a lot of people in the industry that don't know that much, and that missing knowledge is one of the problems. But your conclusion was that really empathy is something that we need to make sure we have enough of.

50:59 Panelists: I want to get a chance to answer your question about what beginners should do as they're getting started. I'll quickly give some advice to beginners. The first is, just know yourself, and for instance, how you learn best. I see beginners come in and say, what's the best way to learn Python? And that really depends on how you like to learn. I as an expert might say, oh, you definitely have to read a 600 page book from cover to cover, but maybe you actually like 10 minute videos better. So, just know what works for you, and do it, and if someone gives you advice that you know doesn't work for you, that doesn't mean that you're doing it wrong. That means that they don't know who you are. Which kind of gets into the toxic experts idea, which is that there are people who know a lot, and they can be bad sources of help in a bunch of ways. One of the risks in any walk of life, not just programming, is that you know know your own insides really well, and you can look at someone's outsides and make a conclusion about their insides, and compare, and feel bad. But that's because you're comparing your insides to their outsides. So, for instance, there's a lot of stuff I don't know, but you don't know that I don't know that. All you see is all my answers on Stack Overflow, or all the code I'm maintaining and you think, wow, Ned knows everything. That's because I'm not writing blog posts about all the stuff I don't know. So, don't compare your insides where you know all the stuff you don't know and all your doubts to my outsides where I'm just like, hey, I'm great. Look at all my awesome tweets, or whatever it is I'm doing on social media. So, there was a great keynote by Jacob Kaplan Moss I forget how many PyCons ago. Jacob is one of the benevolent dictators of Django, or at least he was at the time. He helped create Django and ran the Django Project, so he was well-known for that, and he did a keynote at PyCon basically entitled, I Am a Mediocre Programmer. And he talked about all the stuff he doesn't know how to do, and honestly, it was a little eye-opening. I thought, wow, Jacob. You don't know how to do that? Okay. That's surprising to me because you run the Django Project. But it's just a good demonstration of the idea that experts are not the superpowers that beginners might think they are, and we are in a culture where we don't expose our failures. To get back to Karly's scientific world, right? Scientists typically don't publish their failed experiments, and programmers typically don't. Michael didn't publish his 15 failed attempts at getting the database to work, right? So, it's really easy to look at all the successes that people are trumpeting, and conclude that your little failed program indicates that you are bad and they are good, and it's just not true. So, that's my advice to beginners, is know yourself, and don't judge your insides by other people's outsides.

53:39 Michael Kennedy: Yeah, that's great advice. Want to make a quick call for empathy to everyone out there listening?

53:44 Panelists: Yes, so for empathy. Yeah, a lot of experts are really good at writing complex code and really bad at understanding what beginners know and don't know, caring about the beginner's path that they're on. This all got started because one of the things that came out of that side project is I wrote a piece explaining the Big-O notation, and one of the comments on the piece was, you should be ashamed of this post. And this guy thought I should be ashamed of the entire post because he had a quibble about one sentence in it, which in even after looking into it, I still think he was wrong, but he thought I was a bad person because of this one technical detail about algorithmic analysis. Now, if he had a point, he could've said to me, hey, Ned, I love this piece. There's one little place where you could've done a little bit better, but he came in with a huge hammer and hit me over the head with it. That's not a way to teach people. Beginners, you are going to encounter these people. Just walk away from them. They may know more than you, but they're not going to be able to teach you, and it doesn't matter that they know more than you. They're bad people. Just skip them.

54:47 Michael Kennedy: That's pretty good advice for a lot of situations, right?

54:50 Panelists: Yeah, that is. Absolutely. Yeah. Exactly.

54:52 Michael Kennedy: I think we're over time, so let's go ahead and just leave it there. Karly, Joy, Tsitsi and Ned, thank you all for being on the show, and sharing all of your advice from the different parts of the spectrum you're living on. It's great.

55:04 Panelists: Yeah, absolutely. Thank you. Yes, thank you. It was good. Thank you, Mike.

55:08 Michael Kennedy: Yeah, thanks, and bye, everyone.

55:09 Panelists: Bye. Bye. Bye.

55:11 Michael Kennedy: This has been another episode of Talk Python To Me. Our guests in this episode were Karly Sindy, Joy Danton Ma, Tsitsi Flora Munikwa, and Ned Batchelder. It's been brought to you by Linode and Ting. Linode is your go-to hosting for whatever you're building with Python. Get four months free at talkpython.fm/linode. That's L-I-N-O-D-E. Ting is the fast mobile network custom-built for technical folks. Use their savings calculator to see exactly what you'd pay. Visit python.ting.com to get a $25 credit and get started without a contract. Want to level up your Python? If you're just getting started, try my Python Jumpstart by Building 10 Apps course. Or if you're looking for something more advanced, check out our new Async course that digs into all the different types of Async programming you can do in Python. And of course, if you're interested in more than one of these, be sure to check out our everything bundle. It's like a subscription that never expires. Be sure to subscribe to the show. Open your favorite podcatcher and search for Python. We should be right at the top. You could also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. Now, get out there and write some Python code.