#140: Level up your Python with #100DaysOfCode challenge Transcript

00:00 Michael Kennedy: How do you learn libraries or even parts of Python itself that you don't have actual work projects to try them out on? Whether that's SQLAlchemy, Slackbox or map APIs, actually building projects small and large with them is really the only way to gain true competency. Well, you might try one of the 100 days of Python code challenges that have been going around. This week, you'll meet Bob Belderbos And Julian Sequiera who created PyBites. They've done a few 100 days of code challenges and are here to share their experience and even some concrete examples. This is Talk Python to Me, Episode 140 recorded November 28th, 2017. Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem and the personalities. This is your host, Michael Kennedy. Follow me on Twitter where I'm @MKennedy, keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via @talkpython. This episode has been sponsored by Rollbar and GoCD. Thank them both for supporting the podcast by checking out what they're offering during their segments. Bob, Julian, welcome to Talk Python.

01:23 Bob Belderbos: Hey Mike, thanks for having us.

01:24 Julian Sequeira: An honor.

01:25 Michael Kennedy: It's great to have you here and see you at least on Skype. Bob, we had such a great time at PyCon.

01:30 Bob Belderbos: Indeed, it was very nice meeting you.

01:32 Michael Kennedy: Yeah, it was good to hang out and that was really my favorite part of PyCon is just meeting all the people and experiences outside the talks in the conference rooms but it was a good time and Julian, next year maybe you can make it. Who knows?

01:45 Julian Sequeira: Aw man, totally jealous.

01:47 Michael Kennedy: It is a very long way from Australia.

01:49 Julian Sequeira: I'm going to try in 2018.

01:51 Michael Kennedy: That's awesome. Yeah, I've done that flight from Sydney, it's a commitment.

01:54 Julian Sequeira: It's not pleasant, let's just say that.

01:56 Michael Kennedy: No but you know look if you're going to Australia, coming from my direction to yours it's a wonderful place to visit so maybe I'll get down there some day. So we're going to talk about code challenges. Twitter has been afire with this concept of 100 Days of Code, basically people are learning something new or pushing their boundaries in some way like I'm going to do this for 100 days and I'm going to commit publicly to doing it and sort of testify to social media if that is a thing. I've done my day's work right. So we're going to have a good sort of retrospective on this one that you did and maybe talk about another one that you guys are doing as well. Before we get there, though, let's start with your story. How'd you get into programming in Python? Julian, go first.

02:42 Julian Sequeira: Alright. So it's actually kind of funny. If you're familiar with PyBites, I'm the relative newbie compared to Bob. So I actually started with C++ in high school but Python wise I don't actually work with Python every day. It's not what I do for a job which has made it incredibly difficult. It was actually Bob who pushed me. So I was trying to calculate some overtime when I worked in the field for Oracle and yeah, he said, "Why don't you create a Python app to track that?" And that was it. As soon as I started doing it, bang, I was like, this is so cool.

03:18 Michael Kennedy: This is so much easier than smart pointers in C++ or whatever.

03:23 Julian Sequeira: Exactly. Well now I don't have to log in and use the Oracle tool, I can use my own.

03:27 Michael Kennedy: Beautiful.

03:28 Julian Sequeira: It was great. And then from that it just one thing after another and then PyBites.

03:32 Michael Kennedy: Yeah, awesome. And what do you do day to day for work? Do you work at Oracle still?

03:37 Julian Sequeira: No, no. So I resigned about a year and a half ago and I now work for Amazon Web Services.

03:43 Michael Kennedy: Oh, exciting. What do you do there?

03:44 Julian Sequeira: It's a bit hush hush but I work in one of the data centers. It's a lot of fun. So if anything goes down, it's probably me, no I'm just kidding.

03:54 Michael Kennedy: We'll put your Twitter account on this page and soon as AWS Asia goes down, you're going to just hear about.

04:02 Julian Sequeira: Yeah, I'm sure I'll get an email from you as well.

04:04 Michael Kennedy: Yeah, that's right. Sorry, Bob, how 'about yourself?

04:08 Bob Belderbos: I started programming about 10 years ago. I was working in the financial department and we had to close a month every month and the task got a bit boring. So I started to use Excel and Macros to automate the stuff. At the same time, I discovered web design and making web pages which I was very passionate about. So I self taught programming, went back into Sun and later Oracle as a support engineer, started to build web tills for the engineers to solve the cases more rapidly. Well, I was doing that in--

04:43 Michael Kennedy: Give us an example of a tool you might make and the problem that it solves.

04:46 Bob Belderbos: An example of a tool?

04:48 Michael Kennedy: Yeah like what kind of tools did you make? Like what kind of problems and how would you help people solve them?

04:52 Bob Belderbos: Yeah that was cool because there was one day in the chat room, somebody said, "Well, we're getting all these diagnostic bundles from the servers and we don't have any tool to do diagnostic on them." It's like these lucky moments when you see an opportunity and you can build something big so I started to develop a tool to diagnose these bundles and have a web gobe for engineers to help solve the issues and that all became pretty big with hundreds of engineers using it. That got me into--

05:24 Michael Kennedy: And it's still in Python?

05:25 Bob Belderbos: No that was in PHP and later work was in Perl and that got a bit of a mess so in 2012 I looked at other options, stumbled upon Python, found it very clean and nice and I never looked back. So I've been using it for five years now.

05:46 Michael Kennedy: If you look at the graphs of the popularity of Python and the growth of Python, there's an amazing article from Stack Overflow called The Incredible Growth of Python and that's super amazing but if you look at where the graph turns sharply up, that's in 2012. So you're partly to blame or to credit for that, that's awesome. Yeah man. So you just like languages with P's, right, PHP, Perl, Python?

06:09 Bob Belderbos: I do like Java Script a little bit as well.

06:12 Michael Kennedy: We can put a silent P in there. Speaking of P's, let's talk about PyBites. This is so similar to the name of my other podcast. There's all these weird coincidences right, there were no Python podcasts and then I released this one days before Podcast Init comes out and they're like, "We're the first. Wait, no we're not, why?" And then we both released a similar thing, it felt like it was just about the same time, right?

06:38 Bob Belderbos: It was around December, last year I think, right?

06:41 Michael Kennedy: Exactly, I think we were November. It was just like crazy. So PyBit.es, right?

06:48 Bob Belderbos: Yep, it reveals one of the locations in Spain.

06:52 Michael Kennedy: Exactly, that's where you are. Yeah, so you guys, tell everyone about it. It's a pretty cool resource.

06:58 Julian Sequeira: So PyBites is a blog, where pretty much at the bare minimum we share our passion and love for Python, that's it. It started because of that. Bob and I, we always struggled to work on projects together because of the sheer time difference, time zones, different jobs now and it was just ridiculous so we eventually just bit the bullet and said, "You know what? Why don't we start a blog together on Python?" So he knew a lot, I didn't know a huge amount and we felt that that gap between our knowledge levels would be an awesome concept for a blog. He could do the more complex stuff and I could do the really beginner stuff as I'm learning it. So as Bob would be learning some crazy complex concepts, he would write these amazing articles on it and I'd be scratching my head going, "What the hell are you talking about?" And then I'd take something that was quite simple to Bob and I would write that in super layman's terms so that anyone could read it and understand it and we had this sort of double whammy approach.

07:58 Michael Kennedy: I think that's really, that's a super cool idea because I think there's huge value in people who are new to a particular technology like you described yourself talking about their experiences, explaining what they're learning. There's so many people who are learning what you're learning that they're not necessarily looking for the high end expert analysis of meta classes. They want to know how to write a function with default values or something like this, right, and so that's super helpful but then on the other hand, a lot of times that comes up a little bit short in not actually having the right depth or really the best answer and it feels like you guys could team up on that. Did it work that way, Bob?

08:36 Bob Belderbos: Yes, I think it's a nice dynamic between beginner level and Julian making stuff really accessible and me sometimes tackle maybe a bit more advanced topic. So yeah, I think it's a great combination and yeah, finally yesterday, we did a co challenge and we made a new one and Julian sent me the beginning and I then tweaked it and he said, "That's way advanced, like sort with lambda." And that's cool because for some folks, some of the audience, do need that more advanced level. But some of the folks are still with for loops and variables and yeah, you really have to take a step back and Julian's great at that and writes some very accessible articles like on what is a class or how to write a for loop and you notice that a lot of beginning Pythonists just need that.

09:28 Michael Kennedy: That's for sure. It's really easy to sort of zoom past the introductory stuff and get to the exciting internals like here, let's disassemble this and look at the bite code and see what it's doing. It's like, yeah, no, maybe not so much, maybe we got to take a step back. Cool, so how did you guys meet up? Let's talk about some crazy time zone shifting, like the world we live in is so amazing. So Bob, what time is it?

09:53 Bob Belderbos: 11 p.m., 11:15 almost.

09:56 Michael Kennedy: Julian, what time is it?

09:58 Julian Sequeira: It's 9:15 a.m. on Wednesday.

09:59 Michael Kennedy: Oh you guys are all wrong, it's two in the afternoon on Tuesday. Man, you are in the future.

10:04 Julian Sequeira: We made it.

10:05 Michael Kennedy: So that's pretty, yeah exactly, we got at least 12 more hours. So how'd you guys meet up across all this time and space?

10:12 Julian Sequeira: So we actually haven't met, believe it or not, you probably think I'm lying but we haven't met in person. Back in Sun Microsystems where we both worked, Bob actually out of the blue randomly wanted to use some machines that I was looking after and we had this conversation and we just I guess ended up emailing back and forth and now we're practically married, it's crazy.

10:36 Michael Kennedy: Yeah, you guys have like deadlines every single day. You have to agree on code and whatnot, right?

10:42 Bob Belderbos: Yep, daily on the phone.

10:44 Michael Kennedy: That's awesome. So a lot of Skype or other VOIP stuff and GitHub and things like that?

10:50 Bob Belderbos: Yeah, WhatsApp works well for talking and we use Slack also with some more folks in our community and yeah, GitHub for coding. Yeah, we use a lot of tools. So it's a great time to live in.

11:04 Michael Kennedy: It totally is. For Slack, do you guys have like a PyBites community that people can join?

11:09 Bob Belderbos: Yes.

11:10 Michael Kennedy: Alright, tell us about that.

11:11 Bob Belderbos: So that came around code challenges which we will discuss in a bit. We started to ask people to join Slack to ask questions and share their experiences on the code challenges and yeah, it's a great asset to build a community and get inputs and help people.

11:31 Michael Kennedy: Yeah, I think that's really nice. So we can put a link to that in the show notes if people want to join up, it'd be cool.

11:37 Julian Sequeira: It's actually really cool because we invited people as they were doing pool requests and the challenges and they're the most dedicated people, right? And so we're bringing them in and you get one or two people every few days and then we have this community in there where we sort of back away sometimes and these guys and girls, they help each other, they answer their own questions and they have these conversations. We'll come back, I'll be at work all day and my watch is going nuts with hundreds of messages coming through and I'm thinking, "What am I missing out on, what's happening?"

12:06 Michael Kennedy: Yeah, I got to take a lunch break and see what's going on. Nice, that's really awesome. I have a Gitter channel for the podcast and a lot of people drop in there and it's similar. There's probably more people in your channel 'cause I don't talk about it very much but still it's great when people get in there and share ideas and help each other.

12:22 Julian Sequeira: Yeah, totally.

12:23 Michael Kennedy: Alright, this code challenge, so 100 Days of Code. Bob, tell us what the general idea of this is.

12:29 Bob Belderbos: I have to get his name right. I think it was Alex Galloway who invented the challenge for JavaScript and promoted coding in general. It's about 100 days in a row share your coding, share what you're doing, whatever it is. Do an hour a day and be responsible, do a tweet out what you did. So you see a lot of tweets with the #100DaysOfCode and people share scripts or if they did a tutorial or whatever they did but it's one hour a day. We gave a little twist of do actually a script a day which we shot ourselves a little bit in the foot there.

13:04 Michael Kennedy: What happened? Is it a little more than an hour or what happened?

13:07 Bob Belderbos: Lot of scripts were pretty basic and can use some refactoring and they're within the hour but sometimes it was not because typically happens, you start to build something cool and you really want to tweak it and yeah, it can be more. But on the flip side, and we will go into a couple of scripts shortly, yeah, we were picking our favorites for the show and we went through the logs and it's like a couple of months ago now and we're like, did we do all this coding? It was like 5,000 lines of code in 100 days, well, between the two of us, so even and uneven days but still, we did so much.

13:42 Michael Kennedy: That was really cool. Julian, do you feel like, you said you were a little newer at Python, do you feel like this gave you some experience with Python in ways that maybe projects you're working on wouldn't?

13:53 Julian Sequeira: Oh totally. Well first of all, it absolutely destroyed me in a good way. It was just insane, right, we were coding every day as it was for PyBites so to suddenly say well, we're going to do 100 days, it was almost, well we're already doing it so what's the point? So that's when we started and said, "Well, what if we made a script?" And that just took me from one level to maybe five levels above because I was forced to not only learn something. So it's not just watching a video or reading a doc or something, it was learn it, apply it, make it functional and have it actually do something that you can then share out and someone can clone that from GitHub and have a running script that does something.

14:35 Michael Kennedy: Oh that's really awesome. So you kind of put a little twist on it and not just, I wrote some code, I have a code streak for 100 days or whatever you want to call it, it's I created 100 working little baby applications in 100 days, right.

14:50 Julian Sequeira: And that was nuts.

14:51 Michael Kennedy: So we'll talk about the details of them in a minute but Bob, can you just touch on some of the things you got to play with? Did you get to play with say SQLAlchemy or TinyDB or things like this?

15:03 Bob Belderbos: Yeah, SQLAlchemy was there, certainly PyPI, we will have some examples but we did some stats and I think we actually used exactly 100 modules which was like a weird coincidence, 100 days, 100 modules and I think it like 40% or so was from PyPI so it was a lot of external stuff as well. We discovered maps, plot cities on maps and lot of APIs, web development, all kinds of stuff. And we have a lot of snippets there now so we can go back and there's tools we use till this day. There were foundations in there which really API that led to a guest post for the blog. There's slack codes in there which we use on our Slack today. Twitter automation we run today, it's pretty cool.

15:48 Michael Kennedy: Yeah, that's really, really awesome, actually. I feel like you could get a bunch of ideas of like almost products or things that you could create if you were willing to put a few months into one of 'em, right?

15:59 Julian Sequeira: Totally.

16:00 Bob Belderbos: Exactly and it's not about perfectionism, it's about doing these quick prototypes, just get coding. If you think about having the perfect script, you're not going to do it.

16:13 Michael Kennedy: Yeah, I've known a lot of people or some people who are in programming who spend too much time worrying about getting things just right and not just going, I'm going to write this and if I have to I'll refactor it and if it's worst case scenario, I'll just rewrite it knowing a lot more than I did the first time. I feel like some people kind of get frozen deer in the headlights for big projects and just not really wanting to start, being afraid of going the wrong direction. This portion of Talk Python to Me has been brought to you by Rollbar. One of the frustrating things about being a developer is dealing with errors, relying on users to report errors, digging through log files trying to debug issues or getting lawyers just flooding your inbox and ruining your day. With Rollbar's full stack error monitoring, you get the context, insight and control you need to find and fix bugs faster. Adding Rollbar to your Python app is as easy as pip install rollbar. You can start tracking production errors and deployments in eight minutes or less. Are you considering self hosting tools for security or compliance reasons? Then you should really check out Rollbar's Compliant SaSS option. Get advanced features and meet compliance without the hassle of self hosting including HIPAA, ISO 27001, Privacy Shield and more. They'd love to give you a demo. Give Rollbar a try today. Go to talkpython.fm/rollbar and check 'em out. Julian, I guess this probably like helps you just like learn to just like okay, you have to write. It is 11 o'clock at night, I have one hour left, I have to write, yeah.

17:42 Julian Sequeira: That was it. It wasn't sit there and plan it and storyboard it or something, it was okay, what's the first thing we need for this app? Okay, it needs to output this data. Just write your prints, write your for loops, create your databases. And it wasn't perfect, it was tough. There were some nights we'd be up till one in the morning with waking up at 5:30 'cause I have kids and you'd be a wreck and the only positive, the light at the end of the tunnel was seeing that countdown to the end and knowing that you had a gap in between so you almost had two days to write this script but you were writing across both days.

18:20 Bob Belderbos: Yeah the good thing was we probably spent, oh not spent, we probably saved a couple of months of Netflix because I probably canceled it.

18:29 Julian Sequeira: Yeah, I was behind in all my Marvel TV.

18:32 Michael Kennedy: Yeah I sort of view programming as my gaming. I used to play things like CounterStrike games and whatnot and now if I have an extra two hours, I'm like, oh what feature can I add to my training website or what other kind of thing could I create? But I can do that in two hours and it becomes kind of a game as well.

18:49 Bob Belderbos: It's becoming like watching series, it's kind of relaxing. Yeah, the more you build, the more projects you have and the more features or stuff you want to add or fix. So yeah, it's kind of a lost game.

19:02 Michael Kennedy: For sure. Alright, so let's talk about some of these particular challenges. You guys did 100 Days of Code, so 100 scripts as you described it and I asked you to pick 10 that you thought would be fun to talk about. So this is just a 10% sampling, right, this is not exhaustive by any measure? So Julian, want to start on day 51?

19:23 Julian Sequeira: Okay so day 51, I chose this one because it had to do with the request module which was one of the first modules I dealt with when I started learning 'cause I wanted to scrape websites, right, that was the way to get into it. One thing I remember back in the day that I was struggling with was how to deal with a website that had a log in prompt. I was like, well, this is impossible, you can't do it, there's no way.

19:45 Michael Kennedy: Yeah, you can't do a Get against that 'cause it redirects you 301 over to this login page.

19:50 Julian Sequeira: Exactly, screw that, I went, I don't need that data, who needs it? So I was struggling and again for this 100 days, probably one of the complex things was coming up with ideas and just I remembered that pain point and I googled it and I found out oh, you can actually log in with requests, what the hell is this? I can't believe it. So then I started playing with it and next thing you know, I was writing a script to log in to my freecycle.org account which is like an Ebay thing but everything's free and I was able to log into it, scrape my current ads and then save them, report that data and save it back to a database and it was just a simple script but the challenge of logging into something using a script was just mindblowing for me.

20:37 Michael Kennedy: That's really cool. So I know that you can do that with requests. I know you can do posts and deletes and all those kind of things and headers but maybe walk us through what is the process of carrying it over the session? So you go find the login page, you put the body of username, password, you do a post. What, do you get back a cookie and have to carry that over to the next request? Do you remember how that went?

21:01 Julian Sequeira: Yeah, essentially you have this payload as it's called and you have to input your username and password into it and you do a post. The crazy thing here was that you had to investigate the URLs that you need to send this data to because you can't just send it to the page. You have to go into the background and look for their login post request URL and then I was able to send that data, that payload data to them using requests and then you get back your session so you can log in. And it was just crazy.

21:36 Michael Kennedy: That's pretty awesome, nice. Okay that one sounds like a really fun one. And so it's like you took this idea that you kind of already had more or less down, like request, BeautifulSoup, CSS selectors, we're going to do this, oh wait, here's this next level that I need to like jump over and it's really cool that you took one of these challenges as an opportunity to do that.

21:55 Julian Sequeira: Yeah, yeah, it was awesome. And then you learn stuff along the way so then it was quite simple to take that data and chuck it into a SQL-like database.

22:05 Michael Kennedy: Yeah, definitely, that's nice and easy for sure. Bob, how 'about yours? What one do you want to talk about?

22:12 Bob Belderbos: The very first script I did, day number one, when we were still full of ideas. So there's a handle of Python on Twitter, you might've heard of them, they do a daily tweet of a trick or tip on Python, pretty scientific but they have some very good tips and get a great following. The way you submit tips to them is by a Google Doc which goes into a spreadsheet or a CSV downloadable but there was not really a way to see if a tip was already submitted. So you don't want to clutter their database or spreadsheet with duplicate tips. So I made a little script to just request get the CSV which is just a public URL, parse it in some data structure and just from the command line you give it one or more strings and then it's just a little CLI script and it then tells you if the string you're looking for, it shows you the tips that has that string in them and so you can just say, oh, okay, they have already five Ether tools or tips. The one I'm about to submit is pretty similar so I don't do it.

23:22 Michael Kennedy: Right exactly.

23:24 Bob Belderbos: And that was also a nice example of like meeting other developers because then we reached out to them, hey, made this script, this can be helpful for you and then they at the submit form they then link to the script that people could download that script. I even made a one liner wrapper that you can do it with a dash wget or whatever, just do one command even if you don't have the script locally and yeah, they linked to it. So it was a little bit also for the community, if somebody wanted to use that. Yeah, and then we stayed in touch and later we reached out to Python Tip again to do more to actually associate to code challenges, to the great collection of tips they have. So that was kind of a nice script but also like meeting other developers.

24:08 Michael Kennedy: Yeah, I think the ability to reach out to people and form these connections is really powerful. That's sort of because GitHub makes it so easy. It's super easy to connect with people and things like Twitter and so on, it's always been there but it's really great and it's like one afternoon of work or something, you've got this little script and now you've got this connection with someone else doing cool, similar stuff, right?

24:36 Bob Belderbos: Yeah, yeah, that's awesome.

24:37 Michael Kennedy: For sure. Julian, take us back to day 25.

24:40 Julian Sequeira: Alright, so we're going backwards.

24:41 Michael Kennedy: You're talking about SQLite before.

24:43 Julian Sequeira: Yeah, yeah, so this one, one of the first things I learnt with the whole 100 days thing was to use SQLite. So I remember seeing people using SQL databases and going, I have no idea what I'm looking at here.

24:58 Michael Kennedy: Yeah, it's so painful to set up and I need a server and credentials and come on.

25:03 Julian Sequeira: Yeah, exactly, I don't have time for that, no. It was always one thing that I wanted to do. I saw it and I thought, this must be such an amazing skill to have, these people are wizards, I don't know how they do it. And so I learnt SQLite 3 using Python and just starting off nice and simple, database and names, the usual stuff you might start off with. But then as I was making more things where I wanted to store that data, I was getting really annoyed that I had to keep setting up this database manually and this is one of my favorite things about the 100 days is that I ended up creating a whole bunch of little scripts that automate these tiny, little repetitive processes and this is one of them. So day 25, I created a script that generates a test SQLite database. So the sort of Python code, see I've prepared now, I've got it up on the screen, the sort of generic database format that I might like with say three text columns and one integer column, that was just the very generic test one that I was creating and this will create that, you just run this command and it prompts you to ask for the name of your database and that's it, it just creates it.

26:16 Michael Kennedy: Oh nice.

26:17 Julian Sequeira: Yeah, that's how I started it. So I have expanded it. I haven't updated the code on GitHub, shame on me, where it asks you then to name your columns and the name of your table. So it walks you through a little almost wizard to set it up. So that was that one. It was actually a lot of fun because then after that I could just use that every time.

26:36 Michael Kennedy: Yeah, this is one of the harder things about working with SQLite is if you map something like SQLAlchemy to it, then you can just say create all tables on the models you're mapping and it just does it but if you don't want to use that, you're like, oh well, I got to do all the DDL creation, syntax and all that kind of stuff. So this is a nice little thing to have.

26:57 Julian Sequeira: Yeah, it was very useful. Yeah, the SQLAlchemy, that's coming next.

27:02 Michael Kennedy: That's a good one. And by the way, if you're listening and you want to follow along, I'll have the link to your GitHub repository, that's the 100 Days of Code so every one of these is just numbered by day. And so it's just github.com/pybites/100DaysOfCode. So you guys can follow along, it has the numbers right there. Alright, Bob, one of the challenges, if you want to do any sort of testing is external dependencies, right? Especially APIs and if they're rate limited or something it's even worse or if they're going to make modifying changes like post something. So your number 81 has some cool examples on how to get around this, right?

27:41 Bob Belderbos: Exactly, that's the problem with external APIs. They don't want to call them over and over again when you run your tests. So I spend one day looking at unittest.mock patch to mock test the Tweepy API and yeah, I managed to do a little script to populate some fake data and do the patch object decorator kind of fake the call to the Tweepy API and yeah, that worked out pretty well. So it's not a critical script but that's really where you do the coding just for the sake of practice because the original script was just a funny who tweeted what. Because somebody on Twitter asked, okay, who am I talking to? It's PyBites, is this Julian or Bob? So we made a little script that based on the location of the tweet, it would then return Bob or Julian because Julian is in Australia, I'm in Spain.

28:38 Michael Kennedy: Oh yeah, that's awesome. So you can just test by the geolocation, nice.

28:42 Bob Belderbos: Exactly, so I took this example to show that you can just go like random stuff, it can be funny and just get practice but then that who tweeted script actually erased the second need to practice mocking and how to test an external API. So that was then a second script. So one script follows the other, so that's the nice thing, just get started and yeah, you might be afraid that you don't get to day 100 but you do.

29:09 Michael Kennedy: Yeah, that's awesome. This portion of Talk Python to Me was brought to you by GoCD. GoCD is an on premise, open source, continuous delivery tool to help you get better visibility into and control of your team's deployments. With GoCD's comprehensive pipeline modeling, you can model complex workflows for multiple teams with ease and GoCD's value stream map lets you track changes from commit to deploy at a glance. Say goodbye to deployment panic and hello to consistent, predictable deliveries. We all know that continuous integration is super important to the code quality of your applications. Choose the open source, local CI server GoCD. Learn more at talkpython.fm/GoCD, that's talkpython.fm/GoCD. So maybe really quickly tell people about the patch.object decorator. So this is a decorator that you can put on any block of code but typically maybe right on a particular unit test method and it will reach deep down inside and change the implementation of some part of probably that API or really whatever you point it at, right? Like time for example?

30:18 Bob Belderbos: Yes, so it was been awhile, this script but the decorator wraps a unit test method and it calls the API, so the original method but you're kind of replacing the original get status method of that API with you give it a return value which is then the fake data to find in the main body of the script. So it's like I'm faking calling that API return my own data and I'm not making the network call, that's basically what it's doing.

30:47 Michael Kennedy: Yeah, that's super cool. Yeah, very nice. Let's see, next up, Julian, word scramble. Tell us about the word scramble one.

30:54 Julian Sequeira: Alright, so let me just say a quick disclaimer, I'm not a cheater at Words with Friends but I may have used this once--

31:02 Michael Kennedy: But if one were to--

31:03 Julian Sequeira: But if someone was to play Words with Friends, this might be helpful. So I remember back in the day playing Words with Friends a lot, there were people who would use these little apps on their iPhone to put the letters in and it would unscramble that into a whole dictionary list of possibilities. And that was the inspiration for this, not that I ever used one of those, right?

31:28 Michael Kennedy: That would be wrong. You shouldn't do that.

31:30 Julian Sequeira: So if I've ever beaten anyone at Words with Friends, I wasn't using this. So the idea is that it's exactly just that. I just Google the standard dictionary.text, just the whole dictionary text file and I used itertools permutations to then read in the words that you will enter into this app and compare it to the dictionary and then come up with whatever valid words you can make out of that series of letters that you enter. And that was it. It's simple in concept. It actually took me awhile, I ended up calling Bob on this one and saying, "Aw man, what am I doing wrong?" And I showed him the code. So this is the beauty of the whole PyBites relationship.

32:13 Bob Belderbos: And I told you, like itertools permutations, dude you're growing.

32:18 Michael Kennedy: Look where you are, this is awesome. Yeah, this is super cool.

32:24 Julian Sequeira: It would have been an ugly for loop I think when I started but...

32:27 Michael Kennedy: Yeah, I think that's one of the things that you have to learn as you get better in Python is I probably could implement this algorithm or I could probably pip install this algorithm. So let's do that instead. I mean I know itertools is built in but yeah just knowing the stuff that you can use like the pre built things like itertools permutations, rather than going well how do I implement a permutation? Well, let me think about a nested loop, right?

32:52 Julian Sequeira: Yeah, well on that, one of the worst things is you'll spend time on something and then 10 minutes later or even 10 days later, someone like Bob goes, "Why don't you just use permutations?" And you'll be like, "Ah!"

33:04 Michael Kennedy: There's a permutations?

33:05 Julian Sequeira: This existed? Damn it!

33:09 Bob Belderbos: That's neverending because even if you think you have some Python tricks up your sleeves, there's always more, there's so many constructs, I'm still learning.

33:21 Michael Kennedy: Sure, there's a bunch built in. I feel like a lot of them live in itertools and functools and those sorts of things but just also there's 120,000 plus on PyPI if you run out of standard library tricks, there's a few more out there.

33:34 Bob Belderbos: Indeed.

33:35 Michael Kennedy: Alright, so the next one up is using a Pillow to create images in Python. Bob, tell us about that.

33:43 Bob Belderbos: Yeah, I picked this one because it's a tool that we still use today and plus again, a simple prototype, just give it the URL and some text, it downloads the URL image, puts it on a canvas and it puts your text on top of it which makes a reasonable ad banner, not like Photoshop art but it's good enough and yeah, I think for this script or some code challenges morphed into it but we're still using this to make PyBites banners for new code challenges articles.

34:15 Michael Kennedy: Oh nice. So you can like autogenerate like day 31, day 32, day 33 for each one?

34:22 Bob Belderbos: That's the thing. I think later we wrapped it in some Flask GUI. So we just have a form and we give it the text and the background image URL there makes the banner there and you can just click download or it even downloads it directly to your, so I make a banner now in 10 seconds or less.

34:40 Michael Kennedy: That's awesome. That's way better than living in Photoshop all the time.

34:44 Bob Belderbos: Yeah, we don't want to, we want to code.

34:46 Michael Kennedy: Yeah, yeah, for sure. Yeah, I think that's really cool. I had a similar experience recently when I wanted to make all the, I wanted to create audio versions of my online courses. So how do you take mp4s, convert them to mp3s? Then how do you go and put like this is the title, this is the album, this is this track in this album? You find something cool like Pillow or it's not actually Pillow but there's just these little libraries that you wouldn't necessarily think is easy to work with. It didn't seem like it'd be that easy to work with images but here it is, right? Same thing with like IDX tags and mp3s. Very cool. Yeah so Julian, what about this Steam scraper?

35:27 Julian Sequeira: Okay so I love games and that's probably one of the worst things about PyBites is that I have less gaming time than ever. This was actually a fun project for me. It actually spanned a few days and it was made up of a few scripts that I made throughout this entire series and this one I still use. What it does is, and I know someone out there's going to be listening and say, "Why don't you just use the RSS feed?" Well, who cares about the RSS? The whole point is to make this, right? What I did was I scraped the RSS feed using this scraper and this is steam.com, store.steampad, the valve game service, and I scrape their RSS feed for all the latest games. By the way, I was scraping that file, I was parsing that file sorry, using a feedpasser as well to then get all of the games, just the titles and the URLs, dump that into a database. So that was one section that took me a day or two to wrap my head around 'cause I'd never used feedparser at this point. Then there was the whole emailing myself the notification that these games were released. So I'd written an email script at the start of this entire series and then the catch was I was spamming myself with every new game multiple times. So I'm going, well I already know that game's released, I've already bought that game, I don't want to be told again. So then I spoke with Bob and he said, "Oh, I've conquered this problem before. I've encountered it. Just add a zero one flag." And that was it, just mindblown, I was like ah sweet. So then in the database, add a new column, every existing game. Every time something's added, it's given a flag of zero and then once it's been emailed out, it then gets a flag of one and then you only email out the zeros.

37:18 Michael Kennedy: Yeah, that's actually really, really cool. So tell me about feedparser.

37:21 Julian Sequeira: That was actually complex for me because I'd never had to parse XML before. So trying to drill your way down through these nested XML tags just did my head in and I remember crying one night huddled in a corner with a blanket over me and--

37:41 Michael Kennedy: The name spaces, the name spaces.

37:44 Julian Sequeira: Exactly and Bob called me and he said, "What's wrong, my son? How can I help you?" And I talked to him about it and he showed me the namedtuple and at that point, I had never used a namedtuple so this is what I mean, Bob knows these things and I'm still grasping at straws sometimes. So I was like, what's a nametuple? And he walked me through it. I remember this phone call was about half an hour as he's telling me all these things and I'm like, this is amazing. And he already had a script that used namedtuple so he used that as an example. And then I was able to use that, find the correct names and links and then save the data and my code, I was hacking my way down and it just became like five lines as opposed to 20 or 30.

38:32 Michael Kennedy: Yeah, you don't need to know all the XPath expressions to search through it. This is really awesome. So feedparser just understands RSS1, RSS2, Atom, all that and you could just ask it questions after it loads up an XML feed?

38:46 Julian Sequeira: Yeah pretty much, exactly right. I was just looking for those tag names and it just knew it, it just found it. It was able to search that stuff for me so I loved it.

38:56 Michael Kennedy: Oh that's sweet. Alright, so the next one, you know what I enjoyed, I enjoy a lot of things about talking to you guys but one of the things that's really cool that go into this, it's like, I didn't know about feedparser, I'd just import element tree and just go after it with xpath. Apparently I don't need to do that anymore, that's really, really cool. I mean it's not a huge deal for me but if it was just automatic, that would be better. And I think for the next one as well, I've never played with pyGitHub but apparently on day 84, you did, right Bob?

39:24 Bob Belderbos: Yep.

39:25 Michael Kennedy: Alright, what is this one?

39:26 Bob Belderbos: Simple prototype again, just a little script, two functions, to get a user object from the GitHub API and show all the stats for all his or her repos, so the stars and the forks. Yeah, the funny thing is, I should say, you raise awareness to yourself that that library exists and the funny thing is now that developing something bigger for code challenges where I do use a lot of GitHub integration, I went back to that library again.

39:58 Michael Kennedy: Yeah, that's really cool. I mean GitHub does have a straight up HTTP JSON API but having a wrapper over top of it so you just say GetUser or Repo.fork, that is so much nicer than actually parsing the dictionaries.

40:15 Bob Belderbos: I do have to say that recent acquire PDB scales helped me because there was into the GitHub API, the pyGitHub, documentation was a bit limited but that was actually also a blessing to inspect the objects with PDB and really look into it. I mean GitHub APIs is huge and you can do a lot of fun stuff with it so I highly recommend it.

40:41 Michael Kennedy: Yeah, very cool. Alright, let's talk about some Flask apps that you worked on, Julian.

40:48 Julian Sequeira: Alright, so you probably know by now Mike, I'm a bit of a flask addict, I can't get over it. So this was something, so for a long time I wanted to be able to make web apps and I had no idea and I just thought it was magic how people were able to have this script turn into a webpage and I never understood how they did it. It scared me approaching it and Bob pointed me at Flask and he said try this and first of all, I went through the Flask documentation and I just melted. I was like, what is this? This is so detailed and complex.

41:19 Michael Kennedy: This is crazy to sort the route maps and methods and views and templates and template language and models and ah.

41:26 Julian Sequeira: Exactly, exactly. This whole new language, it's like learning a whole new programming language and I ended up going through 20 different sources. I was checking YouTube, I was googling--

41:36 Michael Kennedy: Like three or four new languages, right?

41:38 Julian Sequeira: It was just crazy. So I said, this is something I have to learn and I ended up writing I think three or four Flask apps as part of this 100 days. So I won't go into each one in too much detail but every one of them was something different. It was like a new sort of Flask skill that I hadn't dealt with before. So my three favorites that I wrote, I wrote a BMI calculator, so that's body mass index, just a simple calculation right on the command line but to turn that into a workable form on a website that someone can enter in the data and then it actually shows that result, your BMI, on the same page, that was difficult for me. I was just thinking, how do I do this? So that was pretty cool. So that involved having to have a div below the form. So the form entry field is quite simple but there's a hidden div that doesn't appear unless a certain variable exists. And it was actually really cool. I felt like I was a magician, I felt like a hacker. I was like, man, this is cool.

42:44 Michael Kennedy: I love building web apps like that. It takes awhile to put the pieces together but once you have them you're like, whoa, look at it work, it's mesmerizing.

42:51 Julian Sequeira: Exactly, I still use it even though I don't care about my BMI, I just do it every now and then. Another one was a pay calculator. You can tell I care about my pay, right? No, just that the whole overtime thing that Bob helped me with. A year and a half ago, I wanted too calculate how much money if you work X many hours at this rate, how much it makes, that's a simple use case. The cool part is that it uses Flask sessions. So it saves your hourly rate as a session object and then you can keep using that session object to calculate different hours. You don't have to enter it in every time. And to test it out, I had the first page as a where you enter it in and it's saved as a session object and the second page, so when you browse to the second page, the session object carries over and that was the challenge for this specific app. So that was really cool. And the last one was one of my favorites because it was the first Flask app that I wrote that uses an external API. Actually, was it external API? No, I think it was time zone list. Either way it was, it imports every time zone on the planet and then populates a pull down menu with it. So then you choose that time zone and it tells you what the current time is in that country or in that space compared to your current time. So yeah, it was actually really, really cool, I enjoyed that. That was difficult populating that list with data from some other function. So that was the challenge for this one.

44:24 Michael Kennedy: Yeah, yeah, that sounds really cool, very nice.

44:26 Julian Sequeira: And obviously that love of Flask carried on over after this finished and that's how I got started with the Flask course.

44:33 Michael Kennedy: Yeah, exactly. We'll definitely talk a little more about that at the end, that's awesome. So last one, Bob, let's talk about some automation, AutoGen I don't know, Pelican I know, Slack I know but I've never done anything with it. Tell us about all these things.

44:46 Bob Belderbos: Well I couldn't choose so I put them all under automation.

44:49 Michael Kennedy: I'm going to automate all the things, just all of them.

44:51 Bob Belderbos: Automation, what it's going to start with. It was funny because with the 100 days, you do a tweet of your progress every day. So that was the first common thing to automate, to have it reach out to our log file on the GitHub repo, read in a day and automatically tweet what's in there. Which also put a bit of pressure on us because we had this daily deadline at 22:43 p.m.

45:18 Michael Kennedy: The bot's going to get there, the bot's going to get there, quick.

45:21 Bob Belderbos: The bot is going to get us, it's going to get the text. If it's blank, I have to delete the tweet. It's not a shame but it's a shame. It put actually pressure on us. It saves a lot of time with that. There were Slack bots. So for example, a neat tool that could actually help people marketing. When your domain gets mentioned on Twitter, it gets posted to a Slack channel. So I mean if you get retweeted or stuff you notice it in notification but if somebody just tweets your URL or mentions URL in any sense you don't.

45:53 Michael Kennedy: Right, that happens a lot if somebody just discovers a page or something you've done that's cool and they just share it on Twitter. They might not even know about your Twitter account or forget to put it in there. So yeah, you'll find a lot of stuff if you search for your domain outside of your Twitter name.

46:07 Bob Belderbos: Yep, so we see that on Slack now. Pelican, yeah, we write some articles and we have quite some entries. So I made a little command line tool to just do some inputs and ask some questions about topic and text and then generates a markdown file with the right formatting which again, shaves some time off every time we have to write a post. Yeah, Slack is really cool. We have a challenge going on now to build a chat bot and the most original submission will get, actually we'll buy them a book. We're really fond on chat bots. And one I want to mention that I did at Grove at 100 days was a karma bot. So in Slack you do, when you do like user or subject ++ or --, the karma bot just keeps track of the score. And there was all kind of stuff, prototypes that were born out of the 100 days.

47:03 Michael Kennedy: Wow, that's really cool, yeah, lot of stuff came out of there. Alright so that's one tenth of what you guys learned but I feel like that was super valuable, just talking through that with you. So that's pretty awesome. Julian, do you recommend people try this 100 day code challenge or is it like too intense? Is it like P90X or whatever?

47:20 Julian Sequeira: No, no, I think you need to do this. If you're interested in programming in any way, shape or form, you should totally do it. The only advice I'll give is that it's tough. There are days, where you're going to want to go out with your mates and you're going to come home at 11 p.m. and you'll be probably drunk and you'll be I have to write some code or you'll have to prep, maybe do more the day before. This whole script a day thing was over the top. It was something I think we may do one day again if we're crazy but the whole just building a pattern and the repetition to make you learn something every day. So this we apply to other things as well, it's not just 100 days of code now for us but the main thing is if you are married and you have kids, please take that into account 'cause that was one of the biggest hurdles with us is that it's like you'll have family time and then you'll have to go, I've got to go write a script, they don't understand, it's not their thing, right, so you have to run away. That was the hardest thing to balance, to be honest. But if you can tell your better half, look, here's what I'm doing, work it out, even plan a time of the year where work isn't too crazy but plan it in advance, don't just jump on it because it can be pretty taxing.

48:41 Michael Kennedy: Yeah, of course. You know, one thing that comes to mind while you're speaking is maybe this doesn't have to be outside work, maybe you could get an hour of work at your job saying hey, dear manager, I'm trying to learn this thing and I have this training budget, could you just give me time instead of a class with a bunch of boring slides anyway? I'll just do this?

49:03 Julian Sequeira: That's a really good idea. I'm going to take that to work and try that. Thanks man.

49:10 Michael Kennedy: You should, let us know how it goes.

49:11 Bob Belderbos: It's a good return on investment because you're really practicing and to really learn the program, it requires a lot of hours of deliberate practice and 100 days of code does that.

49:24 Michael Kennedy: Yeah, I feel like you guys must have got a lot of confidence and capability with other libraries that you otherwise wouldn't have touched.

49:32 Bob Belderbos: Exactly.

49:33 Michael Kennedy: Okay cool. So you also have another one and I don't even know where you are in the timing of it but it went around Django specifically, not just Python in general. So tell us about that.

49:45 Bob Belderbos: Yeah, so after 100 Days of Code, we were kind of a bit tired but nevertheless we started a new 100 days. It was a bit looser so we didn't do a daily tweet anymore because Django apps are typically long stretched projects.

50:01 Michael Kennedy: You have to do something like Django page view per day or an API endpoint per day or something.

50:07 Bob Belderbos: We weren't like that strict on it but yeah, we did manage to do two or three apps. For example, we made and there we used feedparser to load in all the articles from Python Planet and make a little front end where we can just go through them and tweet them out and kind of a tool to do a news digest on Twitter. We made a notification app with two way out indications so people could sign up and get an email and then confirm the link and we launched three or four code challenges around Django and we got some good traction from the community building Django apps and submitting progress. So it was not like that structured like 100 tweets, we couldn't do that again but still we got quite some stuff done.

50:57 Michael Kennedy: Yeah, that's awesome. And so this one is also done?

50:59 Bob Belderbos: Yeah, it's done. The deadline is over and now we take a little bit of rest to prepare for another.

51:06 Michael Kennedy: Rest? Come on, you got time for one and a half more this year.

51:09 Bob Belderbos: Basically we're focusing on our projects.

51:12 Julian Sequeira: He meant the REST framework, that's what he meant.

51:15 Michael Kennedy: That's right, right, Django REST Framework, 100 days of it. Nice. So I think looking at the time, we'll probably just leave it here for this topic but I feel like this was super valuable. Looking at the 10 you chose and talking about them I can definitely see the value of doing them and I learned a few things myself. So thanks for walking us through those. Yeah, awesome. So before you guys get out of here, though, you have to face the two questions. So first of all, we'll start with you Bob, I guess, if you're going to write some code, what editor do you use?

51:46 Bob Belderbos: Vim.

51:47 Michael Kennedy: Vim, oh right on and notable PyPI package? You guys got to play with 100 packages or modules in this whole thing so you probably found some cool ones?

51:55 Bob Belderbos: Can we still say Requests?

51:56 Michael Kennedy: You can still say Requests.

51:57 Bob Belderbos: For data analysis, I would say Pandas. We didn't really use it in 100 Days but it's an excellent tool to do any kind of data analysis.

52:07 Michael Kennedy: Yeah, absolutely. Alright Julian.

52:09 Julian Sequeira: Obviously in the command line it's going to be Vim and Bob, don't hate me okay, please don't hate me, on Windows, I actually use Notepad++ and I love it. I know, it's crazy.

52:18 Michael Kennedy: That's cool. I've known some people that love Notepad++ as well, very cool. And notable PyPI package?

52:23 Julian Sequeira: Okay so I'm going to just say two. Obviously Flask. Well okay that's not PyPI.

52:27 Michael Kennedy: Yeah, it's on there, it's on there.

52:29 Julian Sequeira: Is it? Oh okay, sorry I thought you meant like part of standard lib?

52:32 Michael Kennedy: No, no outside thee standard lib, external stuff.

52:34 Julian Sequeira: There are probably two, Flask, obviously, I won't go into that but actually and this is an interesting one because I'm pretty terrible at it, pytest. It was only in the past month that Bob was pushing me on this one going, you've got to try this and it just works.

52:49 Michael Kennedy: It's so easy to just get like boring and go ah, just import unittest and just use that, it's right there like there's nothing to it right.

52:57 Julian Sequeira: And I was not very good at testing my own code and I'm still terrible at it but it was just this whole different way of looking at it so definitely pytest.

53:05 Michael Kennedy: Oh yeah, those are both very, very good ones. Awesome, alright guys, people are interested in this. How can they participate? Can they join your community somehow to like have some support while they go through or what do you think? Bob, go first.

53:19 Bob Belderbos: Yeah, so PyBites, video, articles, contents, news and projects. If you really want to practice, I really recommend to go through the list of code challenges we have, we have 44 now up until now and pick whatever you like and clone the repo or fork the repo and make your branch and start trying to do these challenges. We have seen people growing a lot. They thought, oh, I cannot do it and they did it and they thought they would have a lame submission and they actually came up with solutions we were like wow. So we were learning from them as well.

54:00 Michael Kennedy: But it works, it's incredible.

54:01 Bob Belderbos: So really people and ourselves too, we're really stretching ourselves through these code challenges. So that was really what I would recommend from our side.

54:10 Michael Kennedy: Yeah, that's awesome. Julian, how 'about you?

54:12 Julian Sequeira: Yeah so from a, I guess from the beginner standpoint, for anyone listening, just stop procrastinating and start coding. It's literally learn by doing, that's just the greatest way to do it and again obviously a plug, you've got to try the code challenges. They will force you to just, to learn new things and because you're getting told in the challenges what you have to do and what you have to learn, that's an even better way. Do the 100 Days of Code Challenge but the most important thing is to just start writing code, that's it.

54:42 Michael Kennedy: Awesome. And speaking of writing some code, you started working with Flask, you talked about how you fell in love with it. You have a course on Udemy for Flask, right?

54:52 Julian Sequeira: That's right.

54:53 Michael Kennedy: Yeah, quickly tell people about it before we get out of here.

54:55 Julian Sequeira: Yeah, just super quick, it's a beginner's Flask course, it is totally intended for beginners because as I mentioned before, the Flask documentation, trying to find and understand everything was quite difficult to piece it all together. So I've made this course, Bob and I have worked on it and it's just designed to really sell it in layman terms. So if you don't understand, we walk you through all the steps to write your first Flask app, get the basics down. And we will expand this later as we go and we are working on more courses but this one is the way to get into it and it's actually really fun, the feedback's been great and we've got a link for you, Mike, with a discount code for anyone listening.

55:32 Michael Kennedy: Yeah, I see it. It's got the coupon code in there. So I'll put that in the show notes and people just click it there and they get a discount and you'll know it came from the podcast which is awesome.

55:40 Julian Sequeira: Cool.

55:41 Michael Kennedy: Alright guys, well thank you for doing this project and sharing all the stuff you do at PyBites with the community and now sharing it with everyone listening to the podcast.

55:49 Julian Sequeira: No worries, thanks for having us, Mike.

55:50 Bob Belderbos: You bet, bye.

55:51 Julian Sequeira: Thanks, cheers.

55:51 Michael Kennedy: Bye. This has been another episode of Talk Python to Me. Today's guests have been Bob Belderbos and Julian Sequeira and this episode has been brought to you by Rollbar and GoCD. Rollbar takes the pain out of errors. They give you the context and insight you need to quickly locate and fix errors that might have gone unnoticed until your user's complain of course. As Talk Python to Me listeners, track a ridiculous number of errors for free at rollbar.com/talkpythontome. GoCD is the on premise, open source, continuous delivery server. You want to improve your deployment and workflow but keep your code and builds in house? Check out GoCD at talkpython.fm/gocd and take control over your process. Are you or a colleague trying to learn Python? Have you tried books and videos that just left you bored by covering topics point by point? Well, check out my online course Python Jumpstart by building 10 apps at talkpython.fm/course to experience a more engaging way to learn Python and if you're looking for something a little more advanced, try my Write Python at Code Course at talkpython.fm/pythonic. Be sure to subscribe to the show. Open your favorite podcatcher and search for Python, we should be write at the top. You can also find iTunes feed at /iTunes, Google Play feed at /play and direct RSS feed at /rss on talkpython.fm. This is your host, Michael Kennedy. Thanks so much for listening, I really appreciate it. Now get out there and write some Python code.