How do you learn libraries or even parts of Python itself that you don't have actual work projects to try them out on?
Whether that's SQLAlchemy, Slack bots, or Map APIs, actually building projects, small and large, with them is really the only way to gain true competency.
Well, you might try one of the 100 days of Python code challenges that have been going around.
This week, you'll meet Bob Belderbos and Julian Saccaria, who created PyBytes.
They've done a few 100 days of code challenges and are here to share their experience and even some concrete examples.
This is Talk Python to Me, episode 140, recorded November 28, 2017.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.
This is your host, Michael Kennedy. Follow me on Twitter, where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on Twitter via at Talk Python.
Talk Python to Me is partially supported by our training courses.
Have you heard about the 100 days of code challenge?
It's a challenge where you write code for an hour a day for 100 days.
It's helped many developers finally master programming.
But it can be hard to know what to study or have resources to focus on.
That's why we wrote not one, but two 100 days of code courses.
100 days of code in Python, which covers mostly pure Python.
And 100 days of web in Python, which covers a whole spectrum of web frameworks and concepts.
If you've been thinking about taking the 100 days of code challenge, be sure to visit training.talkpython.fm and check out our courses.
They are the 100 days of projects and lessons with a tidy bow on top just for you.
Bob, Julian, welcome to Talk Python.
Hey, Mike. Thanks for having us.
An honor.
It's great to have you here and see you at least on Skype.
Bob, we had such a great time at PyCon.
Indeed. It was very nice meeting you.
Yeah, it was great to hang out.
And I think that was really my favorite part of PyCon is just meeting all the people and the experiences outside the talks in the conference rooms.
But it was a good time.
And Julian, next year maybe you can make it.
Who knows?
Oh, man. Totally jealous.
It is a very long way from Australia.
I'm going to try for 2018.
Oh, that's awesome.
Yeah, I've done that flight from Sydney.
It's a commitment.
It's not pleasant.
Let's just say that.
No, but look, if you're going to Australia, you come from my direction to yours.
It's a wonderful place to visit.
So maybe I'll get down there someday.
So we're going to talk about code challenges.
You know, Twitter has been afire with this concept of 100 days of code.
Basically, people who are learning something new or pushing their boundaries in some way.
Like, I'm going to do this for 100 days.
And I'm going to commit publicly to doing it.
And, you know, sort of testify to social media, if that is a thing that I've done my days at work.
Right.
And so we're going to have a good sort of retrospective on this one that you did and maybe talk about another one that you guys are doing as well.
Before we get there, though, let's start with your story.
How did you get into programming in Python?
Julian, go first.
All right.
So it's actually kind of funny.
I'm really, if you're familiar with PyBytes, I'm the relative newbie compared to Bob.
So I actually started with C++ in high school.
But Python-wise, I don't actually work with Python every day.
It's not what I do for a job, which has made it incredibly difficult.
Right.
It was actually Bob who pushed me.
So I was trying to calculate some overtime when I worked in the field for Oracle.
And, yeah, he said, why don't you create a Python app to track that?
And that was it.
As soon as I started doing it, bang, I was like, this is so cool.
This is so much easier than smart pointers in C++ or, you know, whatever.
Exactly.
I'm like, well, now I don't have to log in and use the Oracle tool.
I can use my own.
Beautiful.
Yeah, it was great.
And then from that, it's just one thing after another and then PyBytes.
Yeah, awesome.
And what do you do day to day for work?
Do you work at Oracle still?
No, no.
So I resigned about a year and a half ago and I now work for Amazon Web Services.
Oh, exciting.
What do you do there?
It's a bit hush-hush, but I work in one of the data centers.
So it's a lot of fun.
So if anything goes down, it's probably me.
No, I'm just kidding.
We'll put your Twitter account on this page.
And as soon as AWS, like Asia goes down, you're going to just hear about it.
Yeah, sure.
I'll get an email from you as well.
Yeah, that's right.
Nice.
All right, Bob, how about yourself?
I started programming about 10 years ago.
I was working in a financial department and we had to close the month every month.
And tasks got a bit boring.
So I started to use Excel and macros to automate the stuff.
At the same time, I discovered web design and making web pages, which I was very passionate about.
So I self-taught programming, went back into Sun and later Oracle as a support engineer,
started to build web tools for the engineers to solve their cases more rapidly.
Well, I was doing that in Bash.
Give us an example of like a tool you might make and the problem that it solves.
An example of a tool?
Yeah, yeah.
Like what kind of tools would you make?
Like what kind of problems and how would you help people solve them?
Yeah, that was cool because there was one day in the chat room, somebody said like,
well, we're getting all these diagnostic bundles from servers and we don't have any tool to do diagnostic on them.
It's like these lucky moments when you see an opportunity and you can build something big.
So I started to develop a tool to diagnose these bundles and have a web GUI for engineers to help solve the issues.
And that tool became pretty big with hundreds of engineers using it.
And that got me into it.
And it's still in Python?
No, that wasn't PHP.
And later work then was in Perl.
And that got a bit of a mess.
So in 2012, I looked at other options, stumbled upon Python, found it very clean and nice.
And I never looked back.
So I've been using it for five years now.
If you look at the graphs of the popularity of Python and the growth of Python,
there's an amazing article from Stack Overflow called The Incredible Growth of Python.
And that's super amazing.
But if you look at where the graph turns sharply up, that's in 2012.
So you're partly to blame or to credit for that.
That's awesome.
Yeah, man.
So you just like languages with Ps, right?
PHP, Perl, Python?
I do like JavaScript a little bit as well.
Maybe we can put a silent P in there.
Speaking of Ps, let's talk about PyBytes.
This is so similar to the name of my other podcast.
That was so funny.
There's all these weird coincidences, right?
There were no Python podcasts.
And then I released this one like days before podcast init comes out.
And they're like, we're the first one.
Wait, no, we're not.
Why?
And then, you know, we both released like a similar thing.
I feel like it was just about the same time, right?
It was around December last year, I think, right?
Exactly.
I think we were in November.
It was just like crazy.
So P-Y-B-I-T dot E-S, right?
Yep.
It reveals one of the locations, Spain.
Exactly.
That's where you are.
Yeah.
So you guys tell everyone about it.
It's a pretty cool resource.
So PyBytes is a blog where pretty much the bare minimum, we share our passion and love for Python.
That's it.
It started because of that.
Bob and I, we always struggled to work on projects together because of the sheer time difference, time zones, different jobs now.
And it was just ridiculous.
So we eventually just bit the bullet and said, you know what?
Why don't we start a blog together on Python?
So he knew a lot.
I didn't know a huge amount.
And we felt that that gap between our knowledge levels would be an awesome concept for a blog.
He could do the more complex stuff and I could do the really beginner stuff as I'm learning it.
So as Bob would be learning some crazy complex concept, he would write these amazing articles on it and I'd be scratching my head going, what the hell are you talking about?
And then I'd take something that was quite simple to Bob and I would write that in super layman's terms so that anyone could read it and understand it.
And we had this sort of double whammy approach.
I think that's really, yeah, that's a super cool idea because I think there's huge value in people who are new to a particular technology like you described yourself, talking about their experiences, explaining what they're learning.
And there's so many people who are learning what you're learning that they're not necessarily looking for the high end expert analysis of meta classes.
They want to know how to write a function with default values or, you know, something like this, right?
And so that's super helpful.
But then on the other hand, a lot of times that comes up a little bit short and not actually having the right depth, the really the best answer.
And it feels like you guys could team up on that.
Did it work that way, Bob?
Yes.
And I think it's a nice dynamic between beginner level and Julian making stuff really accessible.
And me sometimes tackle maybe a bit more advanced topic.
So, yeah, I think it's a great combination.
And, yeah, funnily, yesterday we did a code challenge and we made a new one.
And Julian sent me the beginning and I then tweaked it.
And he said, well, that's way advanced.
You do like sort with lambda.
And that's cool because for some folks, some of the audience do need that more advanced level.
But some of the folks are still with for loops and variables.
And, yeah, you really have to take a step back.
And I think Julian is great at that.
He made some very accessible articles like on what is a class or how to write a for loop.
And you notice that a lot of beginning Pythonistas need that.
That's for sure.
It's really easy to sort of zoom past the introductory stuff and get to like the excited internals.
Like, here, let's disassemble this and look at the bytecode and see what it's doing.
It's like, no, maybe not so much.
And we got to take a step back.
Cool.
So how did you guys meet up?
You guys are let's talk about some crazy time zone shifting.
Like the world we live in is so amazing.
So, Bob, what time is it?
11 p.m.
11.15 almost.
Yeah.
Julian, what time is it?
It's 9.15 a.m. on Wednesday.
Oh, you guys are all wrong.
It's 2 in the afternoon on Tuesday.
And you are in the future.
We made it.
So that's pretty.
Yeah, exactly.
Yes, we got at least 12 more hours.
So how did you guys meet up across all this time and space?
So we actually haven't met, believe it or not.
Probably think I'm lying, but we haven't met in person.
Back in Sun Microsystems where we both worked, Bob actually, out of the blue,
randomly wanted to use some machines that I was looking after.
And we had this conversation and we just, I guess, ended up emailing back and forth.
And now we're practically married.
It's crazy.
Yeah.
You guys have like deadlines every single day.
You have to agree on code and whatnot, right?
Yeah.
Daily on the phone.
That's awesome.
So a lot of Skype or other VoIP stuff and GitHub and things like that.
Yeah.
WhatsApp works well for talking.
And we use Slack also with some more folks of our community.
And yeah, GitHub for coding.
Yeah.
We use a lot of tools.
It's a great time to live in.
It totally is.
For Slack, do you guys have like a PyBytes community?
Yes.
That people can join?
Yes.
All right.
Tell us about that.
So that came around code challenges, which we will discuss in a bit.
We started to ask people to join Slack to ask questions and share their experiences on the code challenges.
And yeah, it's a great asset to build a community and get inputs and help people.
Yeah.
I think that's really nice.
So we can put a link to that in the show notes if people want to join up.
Yeah.
It'd be cool.
It's actually really cool because we invited people as they were doing pull requests and the challenges.
And they're the most dedicated people, right?
And so we're bringing them in and you get one or two people every few days.
And now we have this community in there where we sort of back away sometimes.
And these guys and girls, they just, they help each other.
They answer their own questions and they have these conversations.
We'll come back.
I'll be at work all day and my watch is going nuts with hundreds of messages coming through.
And I'm thinking, what am I missing out on?
What's happening?
Yeah.
I got to take a lunch break and go see what's going on.
Nice.
That's really awesome.
I mean, I have a Gitter channel for the podcast and a lot of people drop in there and it's similar.
There's probably more people in your channel because I don't talk about it very much, but
still it's great when people get in there and share ideas and help each other.
Yeah, totally.
All right.
It's code challenge.
So a hundred days of code.
Bob, tell us what the general idea of this is.
I have to get his name right.
I think it was Alex Calloway who invented the challenge for JavaScript and promoting coding
in general.
It's about a hundred days in a row.
Share your coding, share what you're doing, whatever it is.
Do an hour a day and be responsible.
Do a tweet out what you did.
So you see a lot of tweets with a hashtag, a hundred days of code, and people share scripts
or if they did a tutorial or whatever they did, but it's one hour a day.
And we gave it a little twist of do actually a script a day, which we shot ourselves a little
bit in the foot there.
Tell us what happened.
Is it like a little more than an hour or what happened?
A lot of scripts were pretty basic and yeah, can use some refactoring and were within the
hour, but sometimes it does not because typically happens.
You start to build something cool and you really want to tweak it.
And yeah, it can be more.
But on the flip side, and we will go into a couple of scripts shortly.
Yeah, we were picking our favorites for the show and we went through the logs and it's like
a couple of months ago now.
And we were like, did we do all this coding?
It was like 5,000 lines of code in a hundred days.
Well, between the two of us.
So yeah, even and uneven days, but still we did so much.
That was really cool.
Julian, do you feel like you said you were a little newer at Python?
Do you feel like this gave you some experience with Python in ways that maybe projects you're
working on wouldn't?
Oh, totally.
Well, first of all, it absolutely destroyed me in a good way.
It was just insane, right?
We were coding every day as it was for PyBytes.
So to suddenly say, well, we can do a hundred days.
It was almost, well, we're already doing it.
So what's the point?
So that's when we started, said, well, what if we made a script?
And that just took me from one level to maybe five levels above, you know, because I was
forced to not only learn something.
So it's not just watching a video or reading a doc or something.
It was learn it, apply it, make it functional and have it actually do something that you
can then share out.
And someone can, you know, clone that from GitHub and have a running script that does something.
That's really awesome.
So you kind of put a little twist on it and just not just, I wrote some code.
I have a code streak for a hundred days or whatever you want to call it.
It's, I created 100 working little baby applications in a hundred days, right?
And that was nuts.
So we'll talk about the details of them in a minute.
But Bob, can you just touch on like some of the things you got to play with?
Like, did you get to play with a SQLAlchemy or TinyDB or, you know, things like this?
Yeah.
Oh, goodness.
Yeah.
SQLAlchemy was there.
We will have some examples shortly.
But we did some stats and I think we actually used exactly 100 modules, which was like a
weird coincidence, 100 days, 100 modules.
And I think like 40% or so was from PyPI.
So it was a lot of external stuff as well.
We discovered maps, plot cities on maps and a lot of APIs, web development, all kinds of
stuff.
And, you know, we have a lot of snippets there now, so we can go back and there's tools we
use till this day.
There were foundations in there with Twilio API that led to a guest post for the Twilio
blog.
There are Slack bots in there we use on our Slack today.
Twitter automation we run today.
It's pretty cool.
Yeah, that's really, really awesome, actually.
I feel like you could get a bunch of ideas of like almost products or things that you could
create if you were willing to like put a few months into one of them, right?
Totally.
Exactly.
And it's not about perfectionism.
It's about doing these quick prototypes.
Just get coding.
You know, I mean, if you think about having the perfect script, you don't get it.
You're not getting to do it.
Yeah.
I've known a lot of people who are in some people who are in programming who spend too much
time worrying about getting things just right and not just going, I'm going to write this.
And if I have to, I'll refactor it.
And if like it's the worst case scenario, I'll just rewrite it knowing a lot more than I did
the first time.
You know, I feel like some people kind of get frozen during the headlights for big projects
and just not really wanting to start being afraid of going the wrong direction.
This portion of Talk Python to Me has been brought to you by Rollbar.
One of the frustrating things about being a developer is dealing with errors.
Ugh.
Relying on users to report errors, digging through log files, trying to debug issues, or getting
millions of alerts just flooding your inbox and ruining your day.
With Rollbar's full stack error monitoring, you get the context, insight, and control you
need to find and fix bugs faster.
Adding Rollbar to your Python app is as easy as pip install Rollbar.
You can start tracking production errors and deployments in eight minutes or less.
Are you considering self-hosting tools for security or compliance reasons?
Then you should really check out Rollbar's compliant SaaS option.
Get advanced security features and meet compliance without the hassle of self-hosting, including
HIPAA, ISO 27001, Privacy Shield, and more.
They'd love to give you a demo.
Give Rollbar a try today.
Go to talkpython.fm/Rollbar and check them out.
Julian, I guess this probably helps you just learn to just like, okay, you have to write.
It is 11 o'clock at night.
I have one hour left.
I have to write, yeah?
Yeah, that was it.
It wasn't sit there and plan it and storyboard it or something.
It was just, okay, what's the first thing we need for this app?
Okay, it needs to output this data.
Just write your prints.
Write your for loops.
Create your databases.
And it wasn't perfect.
It was tough.
You know, there were some nights we'd be up till, you know, one in the morning with waking
up at 530 because I have kids.
And you'd be a wreck.
And the only positive, the light at the end of the tunnel was seeing that countdown to the
end.
And knowing that you had a gap in between.
So you could, you almost had two days to write the script, but you were writing across both
days.
And it was just...
The other good thing is we probably spent, not spent, we probably save a couple of months
of Netflix because I probably canceled it.
Yeah, I was behind on all my Marvel TV.
Yeah.
Yeah, I sort of view programming as my gaming.
Like I used to play things like Counter-Strike and games and whatnot.
And now if I have an extra two hours, I'm like, oh, what feature can I add to my like
training website?
Or what other kind of thing could I create?
Like, I bet I can do that in two hours, you know, and it becomes kind of a game as well.
It's becoming like watching series.
It's kind of relaxing.
And yeah, the more you build, the more projects you have and the more features or stuff you
want to add or fix.
So yeah, it's kind of a lost game.
For sure.
All right.
So let's talk about some of these particular challenges.
You guys did 100 days of code.
So 100 scripts, as you described it.
And I asked you to pick 10 that you thought would be fun to talk about.
So this is just a 10% sampling, right?
This is not exhaustive by any measure.
So Julian, want to start on day 51?
Okay.
So day 51, I chose this one because it had to do with the request module, which was one
of the first modules I dealt with when I started learning.
Because I wanted to scrape websites, right?
And that was the way to get into it.
One thing I remember back in the day that I was struggling with was how to deal with a
website that had a login prompt.
I was like, well, this is impossible.
You can't do it.
There's no way.
Yeah.
You can't do a get against that because it redirects you 301 over to this login page.
Forget it.
Exactly.
Screw that.
I don't need that data.
Who needs it?
So I was struggling.
And again, for this 100 days, probably one of the complex things was coming up with ideas.
And just I remembered that pain point and I Googled it and I found out, oh, you can actually
log in with requests.
What the hell is this?
You know, so I can't believe it.
So then I started playing with it.
And next thing you know, I was writing a script to log into my freecycle.org account, which
is like an eBay thing, but everything's free.
And I was able to log into it, scrape my current ads and then save them, you know, report that
data and save it back to a database, you know, and it was just a simple script.
But the challenge of logging into something using a script was just mind blowing for me.
That's really cool.
So I, you know, I know that you can do that with requests.
I know you can do posts and puts and deletes and all those kinds of things and headers, but
maybe walk us through what is the process of like carrying over the session, right?
So you go find the logging page, you put the body of like username, password, you do a
post.
Would you get back a cookie and have to carry that over to the next request?
Do you remember how that went?
Yeah.
Essentially you have this payload as it's called, and you have to input your username and password
into it and you do a post.
The crazy thing here was that you had to investigate the URLs that you need to send this data to because
you can't just send it to the page.
You have to go into the background and look for their, I guess their login post request
URL.
And then I was able to send that data, that payload data to that using requests.
And then you get back your session so you can log in.
And it was, it's just crazy.
That's pretty awesome.
Nice.
Okay.
That one sounds like a really fun one.
And it's like, you took this idea that you kind of already had more or less down like request,
beautiful soup, CSS selectors.
We're going to do this.
Oh, wait, here's this next level that I need to like jump over.
And, you know, it's really cool.
He took one of these challenges and opportunity to do that.
Yeah.
It was, it was awesome.
And then, you know, you, you learn stuff along the way.
So then it was quite simple to take that data and chuck it into a SQL light database, you
know, nice and easy.
Yeah, definitely.
That's nice and easy for sure.
Bob, how about yours?
What one do you want to talk about?
The very first script I did and day number one, and we were still full of ideas.
So there's a handle at Python underscore tip.
You might have heard of them.
They do a daily tweet of a trick or tip in Python, pretty scientific, but they have some very good
tips and get a great following.
The way you submit tips to them is via a Google doc, which goes into a spreadsheet or CSV downloadable.
But there was not really a way to see if, if a tip was already submitted.
So you don't want to clutter their database or spreadsheet with duplicate tips.
So I made a little script to just probably request, get the CSV, which is just a public URL, parse
it in some data structure and just from the command line, you give it one or more strings.
And then it's just a little CLI script.
And it then tells you if the string you're looking for, it shows you the tips that has that string in them.
So you can just say, oh, OK, they have like already five Ether tools tips.
The one I'm about to submit is pretty similar, so I don't do it.
And the fun thing is, right, exactly.
That was also a nice example of like meeting other developers because then we reached out
to them, hey, make this script.
This can be helpful for you.
And then they, at the submit form, they then linked to the script that people could download
that script.
I even made a one-liner wrapper that you can do it with a bash, Wget or whatever.
Just do one command, even if you don't have the script locally.
And yeah, they linked to it.
So it was a little bit also for the community if somebody wanted to use that.
Yeah.
And then we stayed in touch.
And later we reached out to Python tip again to do more, to actually associate two code challenges
to the great collection of tips they have.
So that was kind of a nice script, but also like meeting other developers.
Yeah.
I think the ability to reach out to people and form these connections is really powerful.
You know, that's sort of because GitHub makes it so easy, right?
Like it's super easy to connect with people and things like Twitter and so on.
It's always been there, but it's really great.
And it's like one afternoon of work or something.
You've got this little script and now you've got this connection with someone else doing cool,
similar stuff, right?
Yeah.
Yeah, that's awesome.
Yeah, for sure.
Julian, take us back to day 25.
All right.
So we're going backwards.
You're talking about SQLite before, yeah?
Yeah, yeah.
So this one, one of the first things I learned with the whole 100 days thing was to use SQLite.
So over the years, I remember seeing people using SQL databases.
I'm going, I have no idea what I'm looking at here.
You guys must be.
Yeah, it's so painful to set up and I need a server and the credentials and come on.
Yeah, exactly.
I don't have time for that.
No.
It was always one thing that I wanted to do.
You know, I saw it and I thought this must be such an amazing skill to have.
These people are wizards.
I don't know how they do it, you know?
And so I learned SQLite 3 using Python and just starting off nice and simple database and
names, you know, the usual stuff you might start off with.
But then as I was making more things where I wanted to store that data, I was getting really
annoyed that I had to keep setting up this database manually.
And this is one of my favorite things about the 100 days is that I ended up creating a whole
bunch of little scripts that automate these tiny little repetitive processes.
And this is one of them.
So day 25, I created a script that generates a test SQLite database.
So the sort of Python code.
See, I've prepared now.
I've got it up here on the screen.
The Python for the sort of generic database format that I might like with, say, three text
columns and one integer column.
That was just a very generic test one that I was creating.
And this will create that.
You just run this command and it prompts you to ask for the name of your database.
And that's it.
Just creates it.
Oh, nice.
Yeah, that's how I started it.
So I have expanded it.
I haven't updated the code on GitHub.
Shame on me.
Where it asks you then to name your columns and the name of your table.
So it walks you through a little almost wizard to set it up.
So that was that one.
It was actually a lot of fun because then after that, I could just use that every time.
Yeah, this is one of the harder things to work in with SQLite is if you map something
like SQLAlchemy to it, then you can just say, you know, create all tables right on the
models you're mapping and it just does it.
But if you don't want to use that, you're like, oh, well, I got to do all the DDL like
creation syntax and all that kind of stuff.
So this is a nice little thing to have.
Yeah, it was very useful.
The SQLAlchemy that's coming next.
That's a good one.
And by the way, if you're listening and you want to follow along, I'll have the link to
your GitHub repository.
That's the 100 days of code.
So every one of these is just numbered by day.
So it's github.com/pybytes, P-Y-B-I-T-S, 100 days of slash 100 days of code.
So you guys can follow along.
It has the numbers right there.
All right, Bob.
One of the challenges, if you want to do any sort of testing is external dependencies, right?
Especially APIs.
And if they're like rate limited or something, it's even worse.
Or if they're going to make modifying changes like post something.
So your number 81 has some cool examples of how to get around this, right?
Exactly.
That's the problem with external APIs.
You don't want to call them over and over again when you run your test.
So I spent one day looking at unit test mock patch to mock test the T-PY API.
And yeah, I managed to do a little script to populate some fake data and do the patch object
decorator.
Kind of fake the call to the T-PY API.
And yeah, that worked out pretty well.
It was not a critical script, but that's really where you do the coding just for the sake of
practice.
Because the original script was just a funny who tweeted what.
Because somebody on Twitter asked like, okay, so whom I'm talking to?
It's PyBites.
Is this Julian or Bob?
So we made a little script that based on the location of the tweet, it would then return
Bob or Julian because Julian is in Australia and Spain.
Oh, yeah, that's awesome.
So you can just test by the geolocation.
Nice.
Exactly.
So I took this example to show that you can just go like random stuff.
It can be funny and just get practice.
But then who tweeted script then actually raised this second need to practice mocking and how
to test an external API.
So that was then a second script.
So one script follows the other.
So that's the nice thing.
Just get started.
And yeah, you might be afraid that you don't get to the 100, but you will.
Yeah, that's awesome.
This portion of Talk Python to me was brought to you by GoCD.
GoCD is an on-premise, open source, continuous delivery tool to help you get better visibility
into and control of your team's deployments.
With GoCD's comprehensive pipeline modeling, you can model complex workflows for multiple
teams with ease.
And GoCD's value stream map lets you track changes from commit to deploy at a glance.
Say goodbye to deployment panic and hello to consistent, predictable deliveries.
We all know that continuous integration is super important to the code quality of your applications.
Choose the open source local CI server, GoCD.
Learn more at talkpython.fm/GoCD.
That's talkpython.fm/GoCD.
So maybe really quickly tell people about the patch.object decorator.
So this is a decorator that you can put on any block of code, but typically maybe write
on a particular unit test method.
Yes.
And it will reach deep down inside and change the implementation of some part of probably
that API or really whatever you pointed out, right?
Like time, for example.
Yes.
So it has been a while this script, but the decorator wraps a unit test method and it calls
the CPI API, so the original method.
But you're kind of replacing the original get status method of that API with, you give it a
return value, which is then the fake data defined in the main body of the script.
So it's like I'm faking calling that API, returning my own data, and I'm not making the network
call.
That's basically what it's doing.
Yeah, that's super cool.
Yeah.
Very nice.
Let's see.
Next up, Julian, Word Scramble.
Tell us about the Word Scramble one.
All right.
So let me just say a quick disclaimer.
I'm not a cheater at Words with Friends, but I may have used this one.
But if one were to.
But if someone was to play Words with Friends, this might be helpful.
So I remember, you know, back in the day, playing Words with Friends a lot.
There were people who would use these little apps on their iPhone to put the letters in
and it would unscramble that into a whole dictionary list of possibilities.
Right.
And that was the inspiration for this.
Not that I ever used one of those.
Right.
That would be wrong.
You shouldn't do that.
So if I've ever beaten anyone with Words with Friends, I wasn't using this.
So the idea is that it's exactly just that.
I just Googled a standard dictionary.txt, just the whole dictionary and text file.
And yeah, I used itatools permutations to then read in the words that you will enter into this app and compare it to the dictionary and then come up with whatever valid words you could make out of that series of letters that you enter.
And that was it.
It's simple in concept.
It actually took me a while.
I ended up calling Bob on this one and saying, oh, man, what am I doing wrong?
You know, and I showed him the code.
And this is the beauty of the whole PyBytes relationship, right?
And I told you, like, AetherTools, permutations.
Dude, you're growing.
I mean, look where you are.
This is awesome.
Yeah, this is super cool.
It would have been an ugly for loop, I think, when I started.
Yeah, I think that's one of the things that you have to learn as you get better in Python is I probably could implement this algorithm or I could probably pip install this algorithm.
So let's do that instead.
I mean, I know AetherTools is built in.
But yeah, just knowing the stuff that you can use, like the pre-built things like AetherTools permutations, rather than going, well, how do I implement a permutation?
Well, let me think about like a nested loop, right?
Yeah, well, on that, one of the worst things is you'll spend time on something.
And then 10 minutes later or even 10 days later, someone like Bob goes, why don't you just use permutations?
And you'll be like, oh, there's a permutations.
This existed?
Damn it.
Yeah, exactly.
That's never ending because even if you think you have some Pythonic tricks up your sleeves, there's always more.
There's so many constructs still learning.
Sure, there's a bunch built in.
I feel like a lot of them live in AetherTools and Functools and those sorts of things.
But just also, there's 120,000 plus on PyPI, right?
Like if you run out of standard library tricks, there's a few more out there.
Indeed.
Yeah.
That's awesome.
All right.
So the next one up is using a pillow to create images in Python.
Bob, tell us about that.
Yeah, I picked this one because it's a tool that we still use today.
Just, again, a simple prototype.
Just give it a URL and some text.
Downloads the URL image.
It puts it on a canvas.
And it puts your text on top of it, which makes a reasonable banner.
It's not like Photoshop art, but it's good enough.
And, yeah, I think, or this script or some code channels morphed into it.
But we start, we're still using this to make PyBytes banners for new code challenges articles.
Oh, nice.
So you can like auto generate like day 31, day 32, day 33 for each one.
That's the thing.
I think later we wrapped it in some Flask GUI.
So we just have a form.
And we give it the text and the background image URL there.
And it makes the banner there.
And you can just click download or even download it directly to your.
So I make a banner in 10 seconds now or less.
That's awesome.
That's way better than like living in Photoshop all the time.
Yeah, we don't want that.
We want to go.
Yeah, yeah, for sure.
Yeah, I think that's really cool.
I mean, I had a similar experience recently when I wanted to make all the, I wanted to
create audio versions of my online courses.
So how do you take MP4s, convert them to MP3s?
Then how do you go and put like, this is the title.
This is the album.
This is this track in this album.
And, you know, you find something cool like Pillow or, you know, it's not actually Pillow,
but like there's just these like little libraries that you wouldn't necessarily think is easy
to work with.
Like it doesn't seem like it'd be that easy to work with images, but you know, here
it is.
Right.
Same thing for like IDX tags and MP3s.
Very cool.
Yeah.
So Julian, what about this steam scraper?
Okay.
So I love games.
And that's probably one of the worst things about PyBytes is that I have less gaming time
than ever.
This was actually a fun project for me.
It actually spanned a few days and it was made up of a few scripts that I made throughout
this entire series.
This one I still use.
What it does is, and I know someone out there is going to be listening and say, why don't
you just use the RSS feed?
Well, who cares about the RSS?
The whole point is to make this, right?
What I did was I scraped the RSS feed using this scraper.
And this is Steam.com, store.steampad, you know, the Valve game service.
And I scraped the RSS feed for all the latest games.
Either way, I was scraping that file.
I was passing that file, sorry, using a feed parser as well to then get all of the games,
just the titles and the URLs, dump that into a database.
Okay, so that was like one section that took me a day or two to wrap my head around because
I'd never used feed parser at this point.
Then there was the whole emailing myself the notification that these games were released,
you know.
So I'd written an email script at the start of this entire series.
And then the catch was I was spamming myself with every new game multiple times.
So I'm going, well, I already know that game's released.
I've already bought that game.
I don't want to be told again.
So then I spoke with Bob and he said, oh, you know, I've conquered this problem before.
I've encountered it just at a zero one flag.
And that was it.
I just mind blown, you know, just I was like, oh, sweet.
So then I, in the database, added a new column.
Every existing game, every time something's added, it's given a flag of zero.
And then once it's been emailed out, it then gets a flag of one.
And then you only email out the zeros.
Yeah, that's actually really, really cool.
So tell me about feed parser.
That was actually complex for me because I'd never had to pass XML before.
And so trying to drill your way down through these nested XML tags just did my head in.
And I remember crying one night, huddled in a corner, you know, with a blanket over me.
The namespaces.
Exactly.
And, you know, Bob called me and he said, what's wrong, my son?
How can I help you?
You know, and I talked to him about it.
And he showed me the name Tuple.
And at that point, I'd never used the name Tuple.
So this is what I mean.
You know, Bob knows these things and I'm still grasping at straws sometimes.
So I was like, what's the name Tuple?
And he walked me through it.
You know, I remember this phone call was about half an hour as he's telling me all these things.
And I'm like, this is amazing.
And he'd already had a script that used name Tuple.
So he used that as an example.
And then I was able to use that, find the correct names and links and then save the data.
And, you know, my code, I was hacking my way down.
And it just became like five lines as opposed to, you know, 20 or 30.
Yeah.
And you don't need to know all the XPath expressions to search through it.
This is really awesome.
So FeedParser just understands RSS1, RSS2, Atom, all that.
And you could just ask it questions after it loads up an XML feed.
Yeah, pretty much.
Exactly right.
I was just looking for those tag names and it just knew it.
It just found it.
It was able to search that stuff for me.
So I loved it.
Oh, that's sweet.
All right.
So the next one, you know what I enjoyed talking?
I mean, I enjoy a lot of things about talking to you guys.
But one of the things that's really cool about going through this is like, I didn't know about FeastParser.
I just, you know, import elementary and just go after with XPath.
Like, apparently I don't need to do that anymore.
That's really, really cool.
And so, I mean, it's not a huge deal for me, but it could, if it was just automatic, that would be better.
And I think for the next one as well, I've never played with PyGitHub, but apparently on day 84, you did, right, Bob?
Yeah.
All right.
What is this one?
Simple prototype again, just a little script to functions to get a user object from the GitHub API and show stats, all the stats for all his or her repos.
So the stars and the forks.
Yeah.
The funny thing is, as you say, you raise awareness to yourself that that library exists.
And the funny thing is, now that developing something bigger for our code challenges, where I do use a lot of GitHub integration, I went back to that library again.
Yeah, that's really cool.
I mean, GitHub does have like a straight up HTTP JSON API, but having a wrapper over top of it.
So you just say get user or, you know, repo.fork.
That is so much nicer than actually parsing the dictionaries.
I do have to say that recent acquire PDB skills helped me because there was, now that I'm digging deeper into the GitHub API, the PyGitHub, there was not, documentation was a bit limited.
But that was actually also a blessing to inspect objects with PDB set trace and really look into it.
I mean, GitHub APIs is huge.
Yeah, that's pretty massive.
And you can do a lot of fun stuff with it.
So highly recommend it.
Yeah, very cool.
All right, let's talk about some Flask apps that you worked on, Julian.
All right, so you probably know by now, Mike, I'm a bit of a Flask addict.
I can't get over it.
So this was something, so for a long time, I wanted to be able to make web apps.
And I had no idea.
And I thought it was magic how people were able to have this script turn into a web page.
And I never understood how they did it.
It scared me approaching it.
And Bob pointed me at Flask and he said, try this.
And first of all, I went through the Flask documentation and I just melted.
I was like, what is this?
This is so detailed and complex.
This is crazy.
This is sort of route maps and methods and views and templates and template language and models.
And ah.
Exactly.
This whole new language.
It's like learning a whole new programming language.
And I ended up going through 20 different sources.
You know, I was checking YouTube.
I was Googling.
Like three or four new languages, right?
It was just crazy.
So I said, this is something I have to learn.
And I ended up writing, I think, three or four Flask apps as part of this hundred days.
So I won't go into each one in too much detail.
But every one of them was something different.
It was like a new sort of Flask skill that I hadn't dealt with before.
So my three favorite that I wrote, I wrote a BMI calculator.
So that's body mass index.
Just a simple calculation, right?
On the command line.
But to turn that into a workable form on a website that someone can enter in the data.
And then it actually shows that result, your BMI, on the same page.
Like that, that was difficult for me.
You know, I was just thinking, how do I do this?
So that was pretty cool.
So that involved having to have a div below the form.
So the form entry field is quite simple.
But there's a hidden div that doesn't appear unless a certain variable exists.
And it was actually really cool.
I felt like I was a magician.
I felt like a hacker.
I was like, man, this is cool.
I love building web apps like that.
It takes a while to put the pieces together.
But once you have them, you're like, whoa, look at it work.
Exactly.
It's mesmerizing.
I still use it, even though I don't care about my BMI.
I just do it every now and then.
Another one was a pay calculator.
You can tell I care about my pay, right?
No, just that the whole overtime thing that Bob helped me with a year and a half ago.
I wanted to calculate how much money, if you work X many hours at this rate, how much it makes.
I thought that's a simple use case.
The cool part is that it uses Flask sessions.
So it saves your hourly rate as a session object.
And then you can keep using that session object to calculate different hours.
You don't have to enter it in every time.
And yeah, and to test it out, I had the first page as a way you enter it in and it's saved as a session object.
And the second page.
So when you browse to the second page, the session object carries over.
And that was the challenge for this specific app.
So that was really cool.
And the last one was one of my favorites because it was the first Flask app that I wrote that uses an external API.
Actually, was it an external API?
No, I think it was a time zone list.
Either way, it was it imports every time zone on the planet and then populates a pull down menu with it.
So then you choose that time zone and it tells you what the current time is in that country or in that space compared to your current time.
So, yeah, it was actually really, really cool.
I enjoyed that.
That was difficult populating that list with data from some other function.
So that was the challenge for this one.
Yeah, yeah.
That sounds really cool.
Very, very nice.
And obviously that love of Flask carried on over after this finished and that's how I got started with the Flask course.
Yeah, exactly.
We'll definitely talk a little more about that at the end.
That's awesome.
So last one, Bob.
Let's talk about some automation.
Autogen, I don't know.
Pelican, I know.
Slack, I know, but I've never done anything with it.
Tell us about all these things.
Yeah, I couldn't choose.
So I put them all under the automation.
I'm going to automate all the things, just all of them.
Automation buckets.
So to start with, so it was funny because with the 100 days, you do a tweet of your progress every day.
So that was the first common thing to automate, to have it reach out to our log file on the GitHub repo, read in a day, and automatically tweet what's in there.
Which also put a bit of pressure on us because we had a bit of pressure on us because we had this daily deadline at 22.43 p.m.
The bot's going to get there.
The bot's going to get there quick.
The bot is going to get us.
It's going to get the text.
If it's blank, I have to delete the tweet.
It's not a shame, but it's a shame.
So it put actually pressure on us.
But yeah, we saved a lot of time with that.
There were Slack bots.
So for example, a neat tool that could actually help people in marketing.
When your domain gets mentioned on Twitter, it gets posted to a Slack channel.
So I mean, if you get retweeted or stuff, you notice it in notification.
But if somebody just tweets your URL or mentions your URL in any sense, you don't.
Right.
And that happens a lot if somebody just discovers like a page or something you've done that's cool.
And they just shared on Twitter.
They might not even know about your Twitter account or forget to put it in there.
So yeah, you'll find a lot of stuff if you search for your domain outside of your Twitter name.
Yeah.
So we see that all on Slack now.
Pelican.
Yeah, we write some articles and we have quite some entries.
So I made a little command line tool to just do some inputs and ask some questions about topic and tags.
And then generates a markdown file with the right formatting, which again,
saves some time off every time we have to write a post.
Yeah, Slack is really cool.
We have a challenge going on now to build a chatbot.
And the most original submission we'll get.
Actually, we buy them a book.
We're really fond of chatbots.
And one I wanted to mention, that idea that grew from the 100 days was a karma bot.
So when you're in Slack, you do like, HipChat uses it.
When you do like user or subject plus plus or minus minus, a karma bot just keeps track of the score.
And that was all kind of stuff, prototypes that were born out of the 100 days.
Wow.
That's really cool.
Yeah.
A lot of stuff came out of there.
All right.
So that's one tenth of what you guys learned.
And I feel like that was super valuable just talking through that with you.
So that's pretty awesome.
Julian, do you recommend people try this 100 day code challenge?
Or is it like too intense?
Is it like a P90X or whatever?
No, no.
I totally think you need to do this.
If you're interested in programming at any way, shape, or form, you should totally do it.
The only advice I'll give is that it's tough.
There are days where you're going to want to go out with your mates.
And you're going to come home at 11 p.m.
And you'll be probably drunk.
And you'll be like, I have to write some code.
And you'll have to prep.
Maybe do more the day before.
This whole script a day thing was over the top.
So it was something I think we may do one day, again, if we're crazy.
But the whole just building a pattern and the repetition to make you learn something every day.
So this we apply to other things as well.
It's not just 100 days of code now for us.
But the main thing is if you are married and you have kids, please take that into account.
Because that was one of the biggest hurdles with us is that it's like you'll have family time and then you'll have to go, I've got to go write a script.
They don't understand.
It's not their thing, right?
So you have to run away.
You have to.
That was the hardest thing to balance, to be honest.
So if you can plan, tell your better half, look, here's what I'm doing.
Work it out.
Even plan a time of the year where work isn't too crazy.
But plan it in advance.
Don't just jump on it because it can be pretty taxing.
Yeah, of course.
You know, one thing that comes to mind while you're speaking is maybe this doesn't have to be outside work, right?
Maybe you could get an hour of work at your job saying, hey, dear manager, I'm trying to learn this thing.
And, you know, I have this training budget.
Could you just give me time instead of, you know, a class with a bunch of boring slides anyway?
Like, I'll just do this.
That's an amazing.
Or something along those lines, right?
That's a really good idea.
I'm going to take that to work and try that.
Nice.
Yeah.
That's a good.
Thanks, man.
You should.
Let us know how it goes.
Yeah.
It's a good return on investment because you're really practicing.
And to really to learn the program, it requires a lot of hours of deliberate practice.
And 100 Days of Code does that.
Yeah.
I feel like you guys must have got a lot of confidence and, like, capability with other libraries that you otherwise wouldn't have touched.
Exactly.
Two, right?
Yeah.
Very cool.
Okay.
So, you also have another one.
And I don't even know where you are in the timing of it.
But it went around Django specifically, not just Python in general.
Tell us about that.
Yeah.
So, after 100 Days of Code, we were kind of a bit tired.
But, nevertheless, we started a new 100 Days.
It was a bit looser.
So, we didn't do, like, a daily tweet anymore because, I mean, Django apps are typically, like, long, stretched projects.
You have to do something like Django page view per day or an API endpoint per day or something.
We weren't, like, that strict on it.
But, yeah, we did manage to do two or three apps.
For example, we made a lot of things like that.
So, we made a lot of things like that.
We made a lot of things like that.
So, we made a lot of things like that.
Yeah.
Still, we got quite some stuff done.
Yeah.
That's awesome.
And so, this one is also done?
Yeah.
It's done.
Yeah.
I mean, the deadline is over.
And now, we take a little bit of rest to prepare for another.
Rest.
Come on.
You got time for one and a half more this year.
Right.
Basically, we're focusing on our project.
Yeah.
He meant the rest framework.
That's what he meant.
That's right.
Right.
Django rest framework.
100 days of it.
Perfect.
Awesome framework.
Nice.
So, I think, you know, get a look at the time.
We'll probably just leave it here for this topic.
But I feel like this was super valuable.
Looking at the 10 you chose and talking about them, I can definitely see the value of doing
them.
And I've learned a few things myself.
So, you know, thanks for walking us through this.
Nice.
No worries.
Yeah.
Awesome.
So, before you guys get out of here, though, you have to face the two questions.
So, first of all, we'll start with you, Bob, I guess.
If you're going to write some Python code, what editor do you use?
Vim.
All right.
On.
And notable PyPI package.
Play with 100 packages or modules in this whole thing.
So, probably found some cool ones.
Can we still say request?
You can still say request.
That's awesome.
Maybe for data analysis, I would say pandas.
We didn't really use it in 100 days, but it's an excellent tool to do any kind of data
analysis.
Yeah, absolutely.
All right, Julian.
Obviously, on the command line, it's going to be Vim.
And Bob, don't hate me, okay?
Please don't hate me.
On Windows, I actually use Notepad++.
And I love it.
I know.
It's crazy.
I've known some people that love Notepad++ as well.
Very cool.
And notable PyPI package.
Okay, so I'm going to just say two.
Obviously, Flask.
Well, okay, that's not PyPI.
All right.
Yeah, yeah, it's on there.
It's on there.
Is it?
Oh, okay.
Sorry, I thought you meant like a part of standard lib.
No, no.
Outside the standard lib.
External stuff.
There are probably two.
I won't go into that.
But actually, this is an interesting one because I'm pretty terrible at it.
pytest.
It was only in the past month that Bob was pushing me on this one, going, "You've got
to try this," you know?
And it just worked.
It's so easy to just get like boring and go, "Ah, just import, you know, unit test and just
use that it's right there.
Like, there's nothing to it, right?"
And I was not very good at testing my own code and I'm still terrible at it.
But it was just this whole different way of looking at it.
So definitely pytest.
All right.
Oh, yeah.
Those are both very, very good ones.
Awesome.
All right, guys.
People are interested in this.
Like, how can they participate?
Maybe can they join like your community somehow to like sort of have some support while they
go through?
Or what do you think?
Bob, go first.
Yeah.
So PyBytes, video articles, co-channels, news, and projects.
If you really want to practice, I really recommend to go through the list of code challenges
we have.
We have 44 now, up until now.
And pick whatever you like and clone the repo or fork the repo and make your branch and start
trying to do these challenges.
We have seen people growing a lot.
They thought like, "Ah, I cannot do it."
Then they did it.
Then they thought they would have some lame submission and they actually came up with solutions.
We were like, "Wow."
First off, we were like, "Wow, I'm not even sure what it is."
You know?
So we were learning from them as well.
But it works.
It's incredible.
So really, people and ourselves too, we're really stretching ourselves through these code
challenges.
So that's really what I would recommend from our site.
Yeah, that's awesome.
Julian, how about you?
Yeah, so from a, I guess, from the beginner standpoint, for anyone listening, just stop
procrastinating and start coding.
It's literally learn by doing.
That's just the greatest way to do it.
And again, obviously a plug, you've got to try the code challenges.
They will force you to just to learn new things.
And because you're getting told in the challenges what you have to do and what you have to learn,
that's an even better way.
So do the 100 days of code challenge.
But the most important thing is to just start writing code.
That's it.
Awesome.
And speaking of writing some code, you started working with Flask.
You talked about how you fell in love with it.
You have a course on Udemy for Flask, right?
That's right.
Yeah, quickly tell people about it before we get out of here.
Yeah, just super quick.
It's a beginner's Flask course.
It is totally intended for beginners because, as I mentioned before, the Flask documentation,
trying to find and understand everything was quite difficult to piece it all together.
So I've made this course.
Bob and I have worked on it.
And it's just designed to really sell it in layman terms.
So if you don't understand, we walk you through all the steps to write your first Flask app,
get the basics down.
And we will expand this later as we go.
And we are working on more courses.
But this one is the way to get into it.
And it's actually really fun.
Feedback's been great.
And we've got a link for you, Mike, with a discount code for anyone listening.
Yeah, I see.
Right.
It's got the coupon code in there.
So I'll put that in the show notes and people just click it there and they get a discount
and you'll know it came from the podcast, which is awesome.
Cool.
All right, guys.
Well, thank you for doing this project and sharing all the stuff you do at PyBytes with
the community and now sharing it with everyone listening to the podcast.
No worries.
Thanks for having us, Mike.
You bet.
Bye.
This has been another episode of Talk Python to Me.
Today's guests have been Bob Beldrbos and Julian Securia.
And this episode has been brought to you by Rollbar and GoCD.
Rollbar takes the pain out of errors.
They give you the context and insight you need to quickly locate and fix errors that might
have gone unnoticed until your users complain, of course.
As Talk Python to Me listeners, track a ridiculous number of errors for free at rollbar.com/talkpythontome.
GoCD is the on-premise open source continuous delivery server.
Want to improve your deployment workflow, but keep your code and builds in house?
Check out GoCD at talkpython.fm/gocd and take control over your process.
Are you or a colleague trying to learn Python?
Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course Python Jumpstart by Building 10 Apps at talkpython.fm/course
to experience a more engaging way to learn Python.
And if you're looking for something a little more advanced, try my Write Pythonic Code course
at talkpython.fm/pythonic.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed
at /rss on talkpython.fm.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
Thank you.