How many Python developers do you know that learned Python quickly, but then plateaued
pretty quickly as well? Maybe this is someone you worked with, or maybe it's even you.
Python's clean and simple syntax can mean it's easy to learn, but hard to master. After all,
if you learned it in a week, what else is there to this language? How much more do you need to
dig into it? Well, plenty. And Dan Bader is here to share his very popular Python tricks with us.
You'll learn to look deeper for more than just the how, but the why and when of many of Python's
more subtle features. This is Talk Python to Me, episode 141, recorded December 4th, 2017.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem,
and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm at
mkennedy. Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on
Twitter via at Talk Python. This episode is brought to you by Linode and Rollbar. Thank them both for
supporting the show by checking out what they have to offer during their segments. Talk Python to Me is
partially supported by our training courses. Python's async and parallel programming support is highly
underrated. Have you shied away from the amazing new async and await keywords because you've heard
it's way too complicated or that it's just not worth the effort? With the right workloads, a hundred times
speed up is totally possible with minor changes to your code. But you do need to understand the internals,
and that's why our course, Async Techniques and Examples in Python, show you how to write async code
successfully as well as how it works. Get started with async and await today with our course at
talkpython.fm/async. Hey, everyone. Before we get to the conversation with Dan about his Python
tricks, I want to talk about Python bytes for a second. I know many of you who listen to Talk Python
also listen to my other podcast, Python bytes, but many of you don't, and it's been a while since I've
talked about it on the show. So if you're looking for almost the opposite of Talk Python, instead of
long form, deep conversations on one topic, you want to just catch up on the news, think newsletter and audio
forum for the Python space, head over to
pythonbytes.fm and subscribe to the short
news focused 15 minute weekly podcast
I do with Brian Okken. I hope you guys check it out and I hope you're enjoying both
of the podcasts. Now let's chat with Dan. Dan, welcome back
to Talk Python. Hey, thanks, Mike. Thanks for having
me on the show again. Yeah, I love to talk with you
about technology and programming, so I'm really
excited to do it once again.
You were previously on the show
for our panel with
Anthony and Ronald about
Contributed Open Source, and that was really a well-received
episode. That was 132. Cool, yeah.
That was a really fun experience, and
I really like the panel format. You know, I
listen to your show, and those are
some of my favorite episodes when you bring on a bunch of
people, and it's just, you know, the amount of information
you can soak up so quickly. It's, yeah, it's just great.
Awesome, thanks. The next one that I have scheduled,
I don't know when it's actually going to air, like
four or five episodes out, is
a catch-up for
Python and machine learning at the Large Hadron
Collider and in particle physics,
so that should be really fun. That sounds intense.
It's going to be intense. That means lots of research
for me, so that'll be awesome. All right,
so let's talk about
not contributing to Open Source so much.
Let's talk about making our
Python better with Python
tricks. So that's what we're going to talk about this
week. You know, I know you talked a little
bit about it in 132, episode
132, but it was the panel format,
so you didn't get to go into it too much.
So let's talk just quickly, kind of like
how you got into programming and what you
do day-to-day before we get into your tricks.
Sounds good, yeah. So, well, I guess how I got
into programming was when I
finally managed to convince my parents to buy
a used, old, dingy
Commodore 64 for me.
And, you know, that was the end of the
home computing era, I guess, and those machines
all booted into some kind of
interpreter directly, so you wouldn't, you know,
you don't get, like, a graphical desktop
or anything. You just get a basic, the
programming language.
Yeah, that was crazy because you would,
like, you couldn't just interact with the
computer. You had to, like,
converse with it to get it to do things,
right? It was interesting.
Exactly. It was, like, a glorified
calculator and you just
turned it on and you booted
instantly and then you were just
sitting there like, ah, okay, I'm
getting this blue screen.
What am I going to do now?
And it really forced you to
pick up some basic programming skills
to do anything. You know, if you wanted
to play a game, well, tough luck.
You had to actually figure out, like,
how to load the game from disk and
then run it or, you know, actually
type it in.
Yeah, it was, like, a load command and
stuff to actually do it, right?
Exactly. Yeah, the load 8,1 or
something. It's like the drive ID.
And I think I'm kind of fortunate that
I caught the tail end of that because
it was already, like, the PC era and,
like, Windows 3.11 or something.
But because my parents were really,
really against computers and technology
in some way, the only thing I managed
to convince him was to get this, you
know, used old Commodore 64.
And I got it from this guy and it came
with, like, stacks of disks and
notes this guy took.
And so it was just a treasure trove for
me.
And, you know, it hooked up directly to
your TV screen and it was just, yeah,
just this amazing, like, miracle
machine.
And that's how I got into programming.
Nice. And then how'd you get over to
Python?
Oh, yeah, that's a long,
that's a long jump, a big leap there.
Yeah.
Right.
I think that the basic programming and
basic on the Commodore 64, I think it
just kindled some, some kind of passion
in me.
And eventually I went on and got
bachelor's and master's degree in
computer science and just, you know,
wanted to become a professional
programmer.
At some point in my university days, I
think it was on a ski trip with some
friends and one person, he brought a
book, a Python programming book, one of
these, like, learn Python in 48 hours.
One of those that never really work out
that way. And this was the first time
I really.
Was that the Sam's Learn Python on a
Ski Vacation or Your Money Back
Guaranteed book?
Yeah, I should ask for my money back.
Remember the Sam's book?
It's awesome. So you had the book and
you had some, like, kind of downtime
after you're skiing to chill and flip
through it?
Yeah, it was a very geeky skiing trip.
But this is when, the first time when
I saw the actual language, you know,
just learned a little bit more about
it because my friend, he actually
couldn't, hadn't used Python either,
but it just seemed like an
interesting new language or, you know,
I guess it wasn't new at the time, but
for us it was new.
And so I just really fell in love with
the way the language looked, you know,
just seemed like the perfect blend of,
well, this looks really appealing, you
know, like almost like pseudo code and
like in some way, like really poetic
and nice, but also really, really
powerful where it's not just your
basic or basic programming language, but
it's actually something where you can
take it to places and you can build
full-blown applications.
And so, yeah, that's how it started.
That's awesome.
There's plenty of simple
programming languages, but they often
have like a ceiling where it's like,
okay, you would build this and this
with it, but you wouldn't really build
anything bigger than that, right?
Where it's pretty sweet that Python
generally doesn't fall into that category.
Yeah, totally.
And I think it was you, you called it
a full-spectrum language at some
point or like on that interview that
we did on my blog.
And this is, I love this, you know, I
bring this up all the time when
people ask me about Python because
yeah, it is really a full-spectrum
language where someone can dive in and
they can learn the basics of Python and
know enough to be dangerous.
And then you have giant, very, very
complex systems built in this language
and it can kind of scale that whole
gamut.
And I think that's just really cool.
Yeah, it's amazing.
What I really like about that spectrum is
a lot of languages make you take the
stuff you need for the advanced type of
large applications.
you got to use that syntax and that
structure and those design patterns
right from the start, like Java or C
sharp static main void inside of a
program class.
You're like, whoa, I just want to like
do a few things.
Like what's all this about, right?
Whereas Python, you like as you need the
features, you layer them in, but they're
not required until you need them, which
is, I think, a part of the magic.
That's a huge part of the appeal of
Python that I see, you know, when people
use Python or learn about Python, but
it's also kind of the dirty little
secret where it's very easy to kind of
get stuck at that beginner level.
And you're like, well, okay, so how
would I actually go to the point where
I could build, I don't know, something
like the Instagram backend or even, you
know, think about it.
Yeah, absolutely.
And that is one of the curses of its
sort of success or its features is that
it's a lot of people quickly get into
it and they just get comfortable and
they're just like, well, I just sort of,
I knew C or I knew JavaScript.
So now I know Python.
I just don't do semicolons or curly
braces, right?
That's okay.
And it works.
But, you know, I think our topic today
is really about understanding like where
people get stuck and then going, okay,
these are actually how you should go
farther and do it correctly or
Pythonically or whatever, right?
Yeah, pretty, pretty much.
I mean, this is, I feel like this is the
biggest challenge in taking the next
step with Python.
Like how, you know, what does it even
mean to write Pythonic code and how,
how do you get there?
because it's such an opaque concept and
yeah, and whatever can be done there,
I think it's going to make people's
lives easier and it's going to make
them more successful as developers.
Yeah, absolutely.
So let's talk about your book, Python
Tricks.
It's a buffet of awesome Python features,
which is really nice.
And you released it a little while ago
in a digital only version on your site,
debater.org, right?
Right URL?
That's right.
Yeah.
And you also released it on Amazon and
it kind of went a little bit crazy on
Amazon.
Well done, man.
Thank you.
How did it do?
Like right now, if I'm looking here,
it's number 21 and all the books in
Python, like that's amazing for such a
new book, but it was much higher, right?
Yeah.
So crazy enough, it actually hit the
number one spot in Python programming
and in programming languages in general.
At some point it was even, it was book,
it ranked number 250 in Amazon sales
rank across all books on Amazon.
And, you know, I actually, I took a
screenshot of that and this is like.
Printed out, put it on the wall in frame.
This was like visible proof to my in-laws
that I was just, you know, I was not,
not unemployed, not just unemployed and
addicted to the internet.
It's like, here, I'm an author on the
internet.
This is what I've been doing in my
office.
I promise you I have a job.
Yeah, it was a cool experience.
That's really cool.
Congratulations.
Thank you.
Yeah.
And so it's, it's still going strong.
Like it's number 29 in web programming
and stuff.
So yeah, right now.
Very cool.
So we'll definitely link to your book
from the show.
So maybe give us the quick elevator
pitch.
I mean, we've kind of been building up
to it, but what's the, what's the
idea behind the book and what are you
talking about in there?
Well, like you said, the, the
subtitle for a book is a buffet of
awesome Python features.
And so basically the book is a
collection of actionable tips that
will help you write clean,
professional and developer style,
quote unquote, Python code.
And, you know, the feeling that I
want to create for my readers is not
like they're sitting through an
advanced computer science lecture
where I'm throwing a bunch of
jargon at you, but more like we're
on the same development team.
You know, we're sitting down
together, you know, maybe know the
basics of Python.
And I want to help you get up to
speed as quickly as possible and
help you grasp some of the
intermediate and more advanced
Python topics that often seem
weird and opaque, you know, things
like decorators, first class
functions.
What's the difference between an
iterator and a generator and
generator expressions and list
comprehensions.
And just some of the really useful
things in Python that border on the
arcane, but they're really not.
If you, right.
If you approach them from the right
perspective.
Sure.
It's a lot of these things that,
you know, I feel like Python is, you
can learn the language really quick,
but it takes a long time to really
master it.
And part of that mastery is like
discovering these little trade-offs
like I could use a tuple, but oh, by
the way, did you know there's a name
tuple?
And did you know in Python three, six,
there's a new version of a name
tuple that is more flexible, but
similar in performance and like, oh,
wait, there's a new name tuple.
What is that?
Right.
You don't really easily fall into
that discovery or things like, yeah,
you can use list as a queue, but it's
like a thousand times slower than if
you use this other proper thing for
queuing.
Oh, but don't use that for multi-thread
and use this other one for, for
parallelism and things like that.
Right.
Those types of things are somewhat
arcane, but they're, they're not
easily discoverable.
I don't feel.
Exactly.
you know, where it's a lot of
times like when someone has some
experience with other programming
languages and they're switching to
Python, it can be really hard to
understand how these different
concepts map to Python, because I
guess Python uses a very like human
friendly naming style for a lot of
things, you know, like you just
mentioned the queue and list
example, but if you're coming from,
let's say a Java background, how
does a list map to what I know from,
from the Java world, you know, what,
what kind of time complexity
guarantees does it make and, and
kind of, you know, touching on some
of these things as well.
Like how can you take some existing
algorithm and actually bring it over
to Python and to make sure, you
know, it doesn't just look pretty,
but it actually also really works
and is fast enough.
Yeah, absolutely.
And one of the, you know, one of
the things even more than like list
versus say array list in Java is
array, right?
Like there's not really like the
traditional C style array in Python
in the built-in language sense.
There's the, you know, array dot
array class, but there's not the like,
square bracket means something
different than list, right?
Like those are actually the same
thing.
And that just gets confusing, but I
think we should probably just start
going through some of your individual
tricks that you thought are
particularly like noteworthy in at
least a radio format is.
That'll be interesting.
Yeah.
It's so we'll try to not talk too
much about code, but, you know,
cover some of the ideas behind it.
All right.
So actually, you know, the first one
you said, let's start with something
simple.
And I totally agree that it's a
good place to start.
You talk about comma placement for
dictionaries and lists and just like
structuring those for basically for both
avoiding errors and making them source
code control friendly.
What's the story there?
It started with sort of this being a
pet peeve of mine.
Like if you use lists or any of the other
built-in collection classes like a
dictionary or set, and you're trying to
define a constant in your code, like,
let's say, you know, a list of names
or a set of some other objects.
If those get too long and they don't
fit into a line, like usually, you
know, according to pep8, we're spacing
them out and we're kind of spreading
them out across line breaks.
Now, when you do that, this usually
looks pretty good.
But then when you add an element to
the end of the list, you've got to
make sure you have your trailing
comma right in that list, because if
there's no trailing comma, then and
you're working with strings, for
example, cpython, the interpreter or
the parser is actually going to mash
two consecutive strings together into
one.
So you're going to end up with a
completely different value.
And it's not a it's not a interpreter
error or parser error or syntax error.
But this you actually get the wrong
output.
And so this if that makes sense so far.
So if the first if you had a list of
four for names, name one, two, three,
and four, and you forget the comma
because you added a new line, but you
forgot the comma, it's what's in the
list is name one, name two, name three
and four as one string, like as if you
would put a plus there.
And that drives me crazy about Python.
Like, I understand, OK, there's a few
conveniences, but the ability to
introduce bugs by having two string
variables next to each other and
forgetting a comma or something is
really frustrating.
It is totally frustrating.
Yeah.
And so this is something that I saw come
up in code reviews.
you know, as I was, we were at the
time like onboarding two new developers
and this actually came up in the code
review and we're like, what's going on?
Like, I hate this language, right?
Like it was one of these things where
you run into it and you're like, well,
why did this just happen?
And it's like a really nasty error or
bug that you can introduce into your
programs.
And so basically we came up with this
rule of like just ending every single
item with a comma because you can in
Python, you can actually have a
trailing comma, even in the last and
the final item in one of these
collection constants.
Right.
The dictionary, the list, whatever,
right.
It can always have that comma there.
The other benefit is if you don't put
the trailing comma and you're going to
create new items in that list and you
look at a diff in source control, it
looks like the previous item has
changed as well because that line
changed with a comma, whereas it only
changed so you could extend it.
But if you always put the comma
there, like literally the lines that
changed are the lines that changed.
Yeah, that's right.
You can keep your your diffs, your
get diffs nice and clean.
And I mean, you know, I don't know,
like this probably makes us sound like
super nitpicky.
You know, if you're like, oh, you're
you're like the senior developer
reviewing someone else's code, you're
pointing out stuff like that.
But honestly, I think a lot of times
it's a little things, right?
Like if you see like it's a little
bit hard to talk about this on, I
guess, on a podcast where we can show
people code examples.
But if you see this in front of you,
usually people go like, oh, yeah,
that makes sense.
I'm just going to start doing that
because you can just remove that
class of errors or that potential
for errors there.
You can just remove that completely
just by making one tiny, tiny,
small change to your coding habits
or how you write your code.
It's super simple to do and
understand how it works is great.
The other place that that's really
helpful is for code generation.
So imagine you've got like a cookie
cutter template and you're cookie.
you enter some values and that's
powered by Jinja, too.
So you put some very like give it a
list and it'll generate like it could
generate a bunch of code that's going
to generate your starter project.
And because you can trail everything
with a comma, you don't need some weird
if case to like filter out the last
comma and just throw that puppy in
there and it's all good.
Oh, that's nice.
Yeah, I see.
That would make sense.
Yeah.
This portion of Talk Python to me is
brought to you by Linode.
Are you looking for bulletproof
hosting that's fast, simple
and incredibly affordable?
Look past that bookstore and check
out Linode at talkpython.fm
slash Linode, L-I-N-O-D-E.
Plans start at just $5 a month for
a dedicated server with a gig of RAM.
They have 10 data centers across the
globe.
So no matter where you are,
there's a data center near you.
Whether you want to run your Python
web app, host a private Git server
or a file server, you'll get native
SSDs on all machines, a newly
upgraded 200 gigabit network and
24-7 friendly support, even on
holidays and a seven day money
back guarantee.
Want a dedicated server for free
for the next four months?
Use the coupon code Python17 at
talkpython.fm/Linode.
Let's move on to sort of a more
protective, defensive way of coding
by covering our assets with asserts.
Yeah, the assert statement.
Yeah.
So you talked about the assert
statement, which I don't see being used
all that much in the code that I run
across.
Maybe it's just what I've been looking
at, but I don't see it that often.
Yeah.
And I think that's a shame.
I don't know.
Like I always felt the assert statement
in Python was really, really useful.
I feel like it's underutilized.
It can really help people write while
making their code easier to debug.
First and foremost, you know, it's
like a defensive programming technique.
And basically what an assert statement does,
it's like a shorthand for testing a condition.
And then if the condition is false, it will
automatically raise an exception.
It will just throw an assertion error.
So you can use assert statements in your code
just to quickly test conditions.
If you don't want to write, you know, like a
longer if statement and really, you know, maybe
define a custom exception or anything like
that.
You just want to make sure that, for example, a
product price, it shouldn't be negative or
something like that, where you don't really
expect this to happen, but you kind of want
to make sure that it can never actually really
happen in production because crazy things
sometimes happen in production.
Then this is just a great little tool or little
feature in Python that you can use.
Yeah. And I really like the little examples
that you give.
So like when you introduce assertions, you talk
about like, let's imagine you have a store, say
for buying online classes or something, and you
want to set up a discount coupon type of
mechanism.
And so you might assert that after applying the
discount, the price is no less than zero and no
more than the actual maximum price.
So you don't get like a negative, negative coupon
or something.
Charge them more than if they didn't enter it.
The worst coupon in the world.
Please don't put that in.
Yeah. So that's, those are all these little
examples that you have in the book are really
great. And so like asserting after you apply the
discount code that the price is zero to the full
price is pretty straightforward.
So you do talk about some really interesting things
in here. One is that the idea behind assertions is
not the same as proper error handling.
And that these are really meant to handle
conditions that are like bugs, not just invalid
input.
That's right. Yeah. You want to basically use an
assertion. If you want to inform yourself or fellow
developers about an unrecoverable error in your
program. So I like to think of them more as a
debugging aid rather than just, you know, a glorifier,
like a fancy or weird way to throw an exception.
So this is really, you know, your app is probably still
going to come down crashing, but at least you will know
exactly what cost your application to crash.
So it's usually going to make it easier to have your
application, to have it fail fast, right?
When it hits that, that condition that it should not, or
when it goes into a state that it should not be in rather than
having that carry on.
And then at a later time, you know, cause trouble further down
the road. And so using an assert statement or using multiple
assert statements, that's a great way to just have these little
checkpoints in your code that you can just sprinkle around these
little self checks and they can really save you a ton of time
because they give you all the context you need.
They make it easier for you to have that context to debug your
programs because of course, nothing is a silver bullet here.
So yeah, sure.
But I do think it's good to have, like you said, that context,
because a lot of times maybe the website would start crashing
because Stripe is rejecting the price that you're trying to
charge when the price is negative.
And you're like, what is going on here?
Why is this happening?
Right.
Whereas if you had an assertion that said, well, like when we
applied the discount code, the price went negative.
Like, oh, I see.
I know what's going on here.
So it's really nice.
A final warning that you had in here is you should not use
assertions for regular checks, right?
Like if you ask a person to enter a number between zero and
10 and they enter 11, you shouldn't like assert that that's
between zero to 10, right?
Because they can be turned off the dunder debug setting that you
talk about.
And also they're, they don't, don't tell you anything about it
from a perspective of the exception, right?
It's always an assertion error, not like a negative number custom
exception thing you made or something.
Yeah.
You can add a message to the assertion that will also be part of
the exception stack trace.
But, but yeah, like you said, it's, it's pretty dangerous to use
assertions for actually handling conditions or checking for
conditions that you expect to happen as part of the normal program
flow.
So for example, if you do your user validation with an assert
statement, you know, if you go assert user is admin or something
like that, well, that's going to work great.
But as soon as these assertions are turned off and they can be
globally disabled at the interpreter level, then all of a sudden, you know,
everyone can log into your app and do all kinds of crazy things because
all of a sudden everyone is an admin.
And so you don't want to do that.
Yeah, that's for sure.
Yeah.
You would basically be able to pass a flag that turns off all air
handling and preconditioned checking.
That might be bad.
Do you know why they are statements and not functions?
I remember reading about that.
I hope I didn't actually mention this in the book and now I forgot about it.
I don't think you did.
It seems to me like maybe one of the reasons is if you do turn them
off, they can like literally vanish from the code and not just become
operations that don't do anything.
So maybe they're easier to like factor out.
I actually have no idea, but it drives me crazy.
Like there's like weird conditions about the tuples versus parameters and
all sorts of funkiness because they're not just like actual calls,
function calls.
That's right.
It's very easy to write assertion statements that never fail because you
would somehow intuitively think this feels more like a function call,
sort of like the print statement versus the print function.
But if you write a syntax or if you use this like a function and you put
parentheses around it, Python can actually interpret it as this as a tuple.
So it's like the assert statement plus a tuple expression.
This will always be so this will also be true or truthy.
And so, yeah, you can you can shoot yourself in the foot that way.
But I think this is now actually a warning in Python 3.
Yeah.
You do get a warning in at least 3.6.
I can tell you that for sure.
So that's good.
Yeah.
But I'd rather have just the language, actually, you know,
just not let that be possible in the first place.
All right.
So let's move on to the next one.
The shocking truth about string formatting.
It's not very Zen-like, is it?
Yeah.
No, that's true.
I mean, this is one of the realizations I had when the format strings or F
strings landed in Python 3.6.
I was like, hold on a minute.
Like, there's like four different ways to do string formatting in Python now.
And variations on each four of those.
So which one should I actually use?
And it seemed to go against the Zen of Python as well.
And, you know, I guess it's one of these things where, well, languages, they evolve over time
and we can't just throw out, you know, all of the legacy code that we have.
So it's like a trade-off that we need to make or people need to make.
But still, I just thought it was interesting.
And yeah, so let's chat about that.
Yeah, for sure.
So we've got the old style formatting, the like string with a percent, a format character.
So percent S, percent D, and so on, percent F.
Then mod the values or a tuple of the values to be inserted.
And that's kind of more or less not the way these days, right?
Yeah.
I mean, it's called old style formatting for a reason, I guess.
But yeah, so we have a newer way of doing string formatting.
And it's called, it's referred to as new style string formatting using the string dot format function.
But it's not actually the newest style, but it's the middle age.
And the old style isn't actually...
It's not deprecated, yeah.
It's not going to go away, right?
So I guess it's just a hint that we got from the CPython team that, you know, really you should be using...
Maybe you shouldn't be using the old style of string formatting.
On the other hand, it is kind of nice to use.
Yeah.
And it's, you know, the new style is still in Python 2.7.
So new style would be like, hello, curly, curly, all that in quotes, dot format name or something like that.
And that's pretty standard.
That's really nice.
Takes a lot of sort of its own formatting mini language to convert the values like numbers to digit grouping and things like that.
And then we've got the f-strings, which are the new hotness in 3.6, right?
Yeah, the f-strings are just awesome.
They were added in Python 3.6.
And I think it's a really, really great feature.
The f-strings are also called formatted string literals.
And people also refer to this as string interpolation.
So really what this means is it's basically a new type of string.
So you just put a lowercase f in front of your string.
And then you can use curly braces to just have a Python expressions right in your string.
And when Python evaluates that string, it's actually going to evaluate those expressions, convert them to a string, and then merge them with the surrounding pieces or parts of that string.
So this is a really, really nice and concise way to define strings in Python that are dynamic.
And it does the same job than all of the other formatting tools that we talked about or formatting styles.
But I find this to be very readable and just very, very nice to work with.
Yeah, I really like it too.
The more I use it, the more I really want to use it everywhere.
It's great.
And so you would say F quote and then have a regular text and then curly curly.
Then, like you said, you put these expressions and they don't.
It looks very similar to the dictionary version or the keyword value version of the old format or the new old format or middle-aged format.
But you can do things that are unexpected.
Like you could actually call functions inside there.
You can add up things.
You can, what I didn't realize is you can actually put format descriptors like some variable colon hash X for hexadecimal representation.
And that's pretty cool.
It's pretty neat.
And I think it's also nice and readable because you just have all of the context you want.
Like one of the challenges I've felt with, you know, at least old style string formatting format is that a lot of times you have to jump back and forth.
And then, you know, if you have a complicated format string, you have to actually count out like, okay, this percent S, a placeholder, it refers to this other variable.
And you just have to do all of that, that mental gymnastics to figure out what's actually going on.
And with f-strings, well, you see it right in front of you and it's kind of nice.
Yeah.
It eliminates a whole class of bugs.
Like it's totally possible to have too few arguments in like a format expression or too many arguments or get them in the wrong order.
But when the actual variable goes in the spot, like that's what's in the spot, right?
There's no more debate in it.
Yeah.
I guess the one thing that is kind of nice sometimes is in the string, you have to repeat something that's like sort of an expression that got evaluated.
Like that might be in the middle.
And so I'm going to propose some nomenclature here and we'll see if the world will take it.
I don't know.
So let's call the old style.
Old style is finer.
It could be legacy style formatting.
I think that's even a better word for it.
The new style, what should we call that?
What do you think?
The traditional string formatting style.
And then the interpolation, we could call that the new style.
The new style.
Yeah.
Let's do that.
Let's go and submit a pull request on the Python, CPython docs.
Exactly.
Let's do a PEP to change the naming.
Exactly.
And so really quick.
The last one is one that I really hadn't played with, which is it's sort of the little Bobby tables equivalent of string.
So it's something called string templates.
And it's really good for when you're like taking an untrusted input and jamming it into a string format.
Yeah, this is great when you're dealing with user input.
You know, when you have something else that comes in externally where users can actually define those templates.
Because the problem is if you just feed that into one of the other formatting techniques or methods, people can actually execute code in your interpreter environment.
So there's a way to reach out and look at global variables.
I mean, I'm in Ronaker's blog where he demonstrated that technique.
And basically you can go in and there's a way to reach out from a format string and to reach into the Dunder globals dictionary and just to look up any variable in the global variable environment.
So, you know, if you have your secret API key in there and this is a web app, then, well, bye bye API key.
It's just going to, you know, people can leak it out.
What's my name?
My name is global square bracket AWS secret key.
Oh, look, there it is.
Yeah, that's bad.
Yeah.
Okay.
Yeah.
So you can use this template thing.
And so the template strings, they prevent that.
Yeah, nice.
They're kind of like parametrize queries in SQL expressions.
Like they don't really get evaluated.
They just get put in there as raw strings, but they kind of signal that this part is input and it doesn't run.
And this, it just goes here as text.
This portion of Talk Python to Me has been brought to you by Rollbar.
One of the frustrating things about being a developer is dealing with errors.
Relying on users to report errors, digging through log files, trying to debug issues, or getting millions of alerts just flooding your inbox and ruining your day.
With Rollbar's full stack error monitoring, you get the context, insight, and control you need to find and fix bugs faster.
Adding Rollbar to your Python app is as easy as pip install Rollbar.
You can start tracking production errors and deployments in eight minutes or less.
Are you considering self-hosting tools for security or compliance reasons?
Then you should really check out Rollbar's compliant SaaS option.
Get advanced security features and meet compliance without the hassle of self-hosting, including HIPAA, ISO 27001, Privacy Shield, and more.
They'd love to give you a demo.
Give Rollbar a try today.
Go to talkpython.fm/Rollbar and check them out.
So the next one that you talked about is exceptions.
And there's all sorts of stuff around exceptions.
I mean, there's the whole embracing the look before you leap, rather disregarding the look before you leap style from C and embracing that it's easier to ask for forgiveness than permission.
And just really using exceptions as, like, for exceptional cases, right?
But then there's also making sense of the consuming the APIs that have been really embracing that format.
So Crete, your recommendation is to define your own exception classes to make your code more readable, self-documenting, and so on, right?
Under some circumstances, I think that makes a lot of sense.
I feel like a lot of times you're totally fine if you just, let's say, raise a value error or a key error.
If it's just perfectly clear, you know, if you're working with a small function and this is not some perfectly polished library that you're releasing to the world, you know, just go ahead and raise a value error.
Don't bother with defining your own exception hierarchy.
But I think as soon as you have a larger piece of code that you want to share, either across multiple applications or just, you know, kind of have it nicely served in a well-defined package where it's its own package, its own module, then I think it makes sense.
What do you think about this?
What if you have your rule as if it has a setup dpy, it has its own exception hierarchy?
I think that makes a lot of sense.
Yeah.
I think that's a package.
That's a good idea.
People consume it externally.
Maybe it should be more formalized in this way.
Yeah.
At the very least then.
And also if you're working in a bigger application, you know, where you have a lot of code, like I don't do this in any little script that I write, but if you have a larger body of code, I feel like these are the things that really make it easier for you also to work with your code in the future, because it's just nicer or it's easier for you to see, you know, what went wrong and where.
Essentially, a value error can come from many, many places.
But if it's a password validator exception that you defined, then, well, you know, this has something to do with password validation or, you know, the next person working on your code knows that it has something to do with that.
Another thing that I think is really nice.
Another thing that I think is really nice that you talked about is actually having a base exception that defines, like, these are all the errors that come out of this particular package.
So that makes me crazy.
Sometimes I'll be like handling errors out of different packages.
I'm like, what is this thing throwing?
Is it throwing one of its own errors?
Is it throwing like a system socket error?
Like, what is this?
And how do I find a way to catch it?
You know what I mean?
It'd be really nice to go, okay, I'm working with the SQLAlchemy library.
So I'm going to catch, I'm going to start by catching SQLAlchemy errors and like dealing with that.
And then I'll start to add on special cases of like, oh yeah, this is when the network is down.
This is when the database doesn't exist or, you know, whatever.
Yeah, exactly.
And some people start using value error or something like that, like a built-in exception for that.
And then just put that information in the exception message.
And I mean, for debugging purposes, I think that's okay because, well, you're just going to look at the stack trace and you know, sort of know what's what happened.
But you really don't have the granularity when you're catching these exceptions, right?
When you have a try except block and like actually exception is part of your structuring the control flow of your program.
Then that becomes very dangerous because then, well, you kind of have to parse out that string.
And I don't know, I don't like that.
Yeah.
Well, and also if you're trying to say, well, I'm trying to deal with the case when there's a value error, but is it from this library or that library?
Because that means really different things.
One is the network is down.
The other is you don't have permission.
And I don't really know what to do here, right?
So you can totally help that.
And, you know, another thing that I'm not sure people think very much about, but I think would be really helpful is like discovering the solution to some common problems.
So if it's a generic standard library error and you go search for that on like Google on Stack Overflow, I use Rollbar and Rollbar has this sort of mechanism where you go into the error reporting for your web app.
And it says here are the common solutions or issues that are found associated with this error.
But if it's super general, it's like, well, nothing, right?
But if it's pretty specific errors, like you could probably get help more easily on lots of levels that way.
I really like your thinking there.
It's like this whole meta game of programming, right?
Where you're not just fending on your own, like you're fighting the machine there.
But a lot of times it's just great if you can punch something into Google and just at least find some other people who have the same problem.
And yeah, I think it really increases the Google ability of your exceptions if you do that.
Yeah, absolutely.
Totally agree.
All right.
So let's talk about naming a little bit.
I know that you've talked about this on your blog and some articles.
I've talked about this, Brian Okken on Python Bytes.
There's a special meaning for underscore and double underscores and prefix underscores and suffix underscores in Python.
And maybe we should just touch on those a little bit.
There's like five or so that you cover in here, right?
So many underscores.
Yeah, Python does love the underscore.
I'm getting undersource from all these underscores.
So the first thing that we can start off with is like a semi-private variable, underscore var.
Yeah.
So the underscore var is just a naming convention.
And I like that you called it semi-private because a lot of times when people come from, let's say, a Java or C# background,
they assume that private really means private in this context.
Like you cannot, you know, reach in and modify this variable.
But in Python, if you do underscore var, it's a naming convention.
You know, it's just something that the community agrees on.
It's like a hint to the programmer, but it's generally not enforced.
So that's something you got to keep in mind.
Like we're all consenting adults or whatever Guido said about that.
Exactly.
I've heard him describe it that way as well.
But then there's the, yeah, but I still don't trust you.
So I'll put another underscore in front of it.
Yeah.
And that's where it gets interesting because that enables or that kicks, that makes the name mangling kick in.
So if you do a double underscore in front of your variable name, the Python interpreter will actually mangle that name and prepend the name of the surrounding class to that variable.
So this sort of kind of makes it like a private variable, but not really, because if you do the mangling yourself, you know, if you just do that string concatenation in your head or in your program, you can still reach in and modify that variable.
So it's really more a mechanism for preventing naming collisions, but not, you know, not a way to implement private variables.
Not true private.
I mean, it's not private in the sense that it's private, but there's still really not that much private in all the other languages.
So if you think of say C#, it has a private protected internal, all these different mechanisms and scoping rules.
But then on the other hand, you have reflection and you could just go ask for it for the private variables.
Java has introspection C++ they have private, but you know, how many bytes offset is that?
Okay.
We just go to the front of the object and like read that form over.
And we, I mean, like in all of these languages, there's a backdoor to these privates.
Right.
And so I feel like this is really no more or less than that.
It's just maybe slightly easier if you know the convention, but it certainly screams to you.
Don't do this.
Yeah.
It's like you're, you're digging a moat around your, around your fortress.
And, you know, of course people can still overcome it and destroy your beautiful programming fortress,
but you're just making it a little bit harder for people to hurt themselves.
So yeah.
I mean, the whole point is like people shouldn't mess with the internal implementation.
They shouldn't depend on the internal implementation.
And if they do, they're going to suffer.
Not you probably unless they fire out, file a bug and you got to deal with it.
Yeah.
You would hope so.
Right.
That it's not you as the maintainer who gets the flack for that.
Yeah, exactly.
But I do think like that's kind of far enough.
I use that every now and then.
I also use it for when I want clean autocomplete.
If there's some library that I'm like, I want to give this to somebody.
I want them to say thing dot and the list of the stuff I want them to work with really clearly.
I'll put that either the underscore, the double underscore, not so much to make people stay away,
but to like make the API more discoverable in the tooling.
Oh, that's nice.
That's a good idea.
Yeah.
And the double underscore, like it takes it entirely out of the list.
So I'm kind of like, ah, you know, if it's not going to bother me too much, I might do that.
But that's one of my motivations.
I did want to ask you with the single underscore, that means like internal private.
But do you think that means protected or truly private, like only in the class or classes and
its derivatives?
That'll come down to the individual developers and their teams agreement.
Personally, I use it more like a protected variable, I think, where it's like, yeah, you
know, if it makes sense to reach in and modify this.
And we're part of the same, you know, if it's related code, very, very closely related code,
then I feel like it's okay.
But it's definitely one of these things where it's it's a it's a slippery slope, right?
Or it's like, oh, yeah, it'll be fine.
And it's like one of these famous last word things.
It's this convention.
And it's not super explicit, right?
Like, yeah.
But I think it's, I think it's good to know about it.
So just really quickly, maybe touch on the other underscores.
There's var underscore the suffix.
Yeah, so that's mostly just used to get around.
So, you know, you can use a reserved word, like a Python keyword, like class, you can use
that as a variable name in your programs.
But a lot of times, it actually makes sense to have a variable name, name class, if you're
passing in some kind of class to a function.
Yeah, the place where I've seen it is in Beautiful Soup.
You can do a find and specify a class keyword.
And then you say the name of the class you're looking for in the HTML.
But you can't say class, right?
So they have class underscore equals as the keyword argument.
Yeah, there you go.
And then people sometimes do, they just kind of mangle the name themselves.
So they maybe call it class with a K or with a Z.
Yeah, with a Z.
I actually went and dug through PEP 8.
And so what people recommend or what like the official recommendation is to just end
your variable name with an underscore.
And that disambiguates it and make sure it doesn't conflict or there's no conflict with
the keyword or the built-in reserved word.
And I kind of like that because I feel like, yeah, it's pretty clear, pretty easy to remember.
And we have more underscores that way.
Isn't that wonderful?
Yeah, we have more.
All these languages have ways of dealing with them and none of them are really amazing.
This seems just totally, totally decent.
And then we have the Dunder methods, which is really the Python data model magic methods,
which I know you don't like that term, but.
Do you like that?
I was going to ask you, do you like the word magic methods?
To me, they feel like operators.
I don't know.
I don't really like the term either because magic seems like they're inaccessible and
they're like these internal black box things.
They're not really right.
And then the final thing around underscores, because Python loves underscores, is to use
it as the, I don't care about this variable.
Don't lint it.
Don't make me name it.
I don't care about it.
Yeah.
It almost has like a Haskell kind of feel to it where you're like, oh yeah, I'm so cool.
I'm ignoring all these variables and I'm deconstructing tuples and whatnot.
Yes, exactly.
Yeah.
So if you want to like loop over something like 10 times, you could say for n in range
one to 11 or zero to 10 or whatever.
But if you don't care about that n, you know, you might get warnings like that variable is
not used and whatnot.
And if you want to be explicit and say, I don't care about the loop variable.
I just want to do this 10 times.
Just put an underscore instead of n and everything's good.
Same thing for like parameters.
Yeah.
And methods.
Yeah.
The only thing you got to be careful about is when you do that in a Python REPL session
where the underscore, like a single standalone underscore is usually represents the result
of the last expression that the interpreter evaluated, which is kind of nice if you're using,
you know, if you're working with the REPL, but if you redefine it, that can cause all kinds
of trouble.
So, you know, that's kind of the one caveat with using the single underscore.
That's for sure.
And it's nice because if you type an expression like, oh, geez, I would love to use that as
a variable.
You can say variable name equals underscore and then roll on from there.
That's nice in the REPL.
Yeah.
All right.
So let's move on to data structures.
And this is like a pretty significant chapter that you talk about.
And I kind of hinted at this at the opening.
Like you can come in and find the really basic structures really easily, but there's actually
a rich set of data structures that are special made for certain circumstances and situations.
So let's dig into those.
Maybe, yeah, start with the dictionaries because that's a pretty central thing in Python.
Yeah.
It's like the central data structure in definitely, I think, in Python and in, I feel like probably
in programming in general, it's like just a super versatile tool.
Yeah.
And it's not just something you use in Python, right?
Like the variables and attributes in a class are defined by those.
You know, the stuff in module, like global variables, all sorts of stuff, right?
Like it's the structure that stores, that holds the types of Python together, I guess.
Like the stuff the universe is really made off.
It's the dark matter of Python.
I like that.
Yeah.
And it's, I mean, it's everywhere, you know, but on the other hand, when we started, you
know, the last place I worked at, we were interviewing pretty aggressively and by aggressively mean,
you know, interviewing a lot of people and trying to hire Python developers.
And so one of the things that came up over and over again in those interviews that was that
people were really struggling with when you're talking about like basic, like hash table or
hash map questions, people were like, yeah, I'm not sure if that's available in Python.
And I don't really know how this concept would work in Python, but at the same time, they knew
about dictionaries.
So that was just, I don't know.
I mean, obviously, you know, I understand where this is coming from and this is not necessarily
like a terrible sign.
But on the other hand, I was curious or just confused.
Like, why, why was this happening?
I mean, you know, like, why, why is there this disconnect?
Because in Java, well, if you want like a mapping structure or a dictionary, you know that you're
dealing with some kind of map or a hash map, right?
Like, you know, what kind of concrete data structure you instantiate in Python is just like, yeah,
it's a dictionary.
It does its job.
I don't really care how it works.
And that's good.
But it also means you're kind of, you're disconnected from a lot of the specific implementation details
as a developer.
Yeah, that's for sure.
And until recently, dictionaries and still under some circumstances, they're unordered,
which can cause all sorts of issues.
If you read a file in, make some changes, save it back out in Python, you know, and you had
that file in source control, it could look like 95% change where you just change like something
from true to false or something silly like that.
Right.
And so there's actually, depending on the situations that you're in, there's all these
other types of dictionaries that are pretty awesome.
So we have ordered dict.
I've seen the people that do data science and like really chain together generator expressions
and list comprehensions and make massive use of default dict.
There's chain map, which I don't think I had heard of chain map before.
Maybe I heard of it and forgot about it.
There's even mapping proxy type.
So maybe take us through like when you might want to use those other things, like when dict
or curly curly is not enough.
I was also really, really amazed or, you know, really enjoyed learning more about chain map.
So chain map is really cool.
Basically, you can chain together multiple dictionaries and treat them like a single dictionary, like
a single mapping.
So you can group several dictionaries into one.
And then when you do a lookup, actually the chain map is going to go and it's going to
search all of these underlying dictionaries one by one until it finds a match.
Yeah, that's really cool.
To me, it felt like what I had previously done, if I wanted this behaviors, I would create a
new dictionary and I would merge all of them into it.
Then I would ask that question.
But this lets you basically create the same behavior without actually building that other
dictionary.
And potentially you could create the chain map and then one of the underlying dictionaries
could change.
And then the answer coming back from the chain map would adapt to that, which is pretty awesome,
right?
That's right.
And you don't incur that memory overhead because, you know, it's just like an abstract way to
merge all these dictionaries together.
And it's just the magic is in this search algorithm.
But on the other hand, of course, you know, it can potentially be slower because the search
actually needs to happen.
So depending on, you know, if you change together thousands of dictionaries, maybe that's not
a good idea.
But if you, if you only have a couple of them and maybe you're combining a bunch of
configuration dictionaries, then this could be really handy and you can just make sure,
you know, okay, we have like a default setting dictionary with default settings, and then we
can override them with more specific user settings.
Right.
Put the user settings and stuff at the beginning of the chain map.
Yeah.
That's nice.
Yeah.
The place I see this happening all the time on the web is because you've got to go, you've
got a dictionary for like the route URL data.
You've got one for the post, you've got one for the query string, you've got one for or else,
somewhere else, headers, all sorts of stuff.
And you can like combine them together into one place and ask like, did they give me this
data somehow?
I don't really care where it came from.
Did they tell me their username or something?
Right.
Yeah.
Where's the API key?
Yeah, exactly.
And then where's that print statement I'm looking for?
All right.
So another major class of data structures that I've found to be pretty interesting that you
covered were arrays.
So, you know, sort of the most common one of this is the list, but there's more than that,
right?
Oh, yeah.
Yeah.
And it's really similar to, you know, what we just discussed with the dictionaries where,
well, if you're looking for an array, it's typically people will just tell you,
well, use a Python list.
It does everything you want.
And, you know, behind the scenes, Python lists are actually dynamic arrays.
So mutable dynamic arrays, they will grow and shrink with the data you put in.
But there's actually other options as well.
You know, you can, in a lot of ways, you can view a tuple as an immutable array, which means
you can't change it.
Right.
You index into it.
It has a length.
You can say, give me the third, give me the fifth one, but it's totally static.
Even the individual elements can't be changed, right?
Exactly.
Yeah.
And I mean, the only problem you run into if you have like, you know, a nested data structure.
So if you have a dictionary in your tuple, you can't put something else at that position in the
tuple and replace the dictionary, but you could still reach in and modify the dictionary because
the dictionary itself is not immutable.
It's, you can modify it.
And so sometimes, you know, that causes trouble for people.
But in general, you know, yeah, you can absolutely treat a tuple like as an immutable array.
Let's go back just for a second.
Like one thing we kind of moved on from without talking much about it is mapping proxy type
in the dictionary space.
And so if you really wanted like this truly immutable tuple, you want to put a dictionary
there, you could wrap it in this mapping proxy type, right?
Oh yeah.
That's a perfect segue.
Yeah, exactly.
So you, you could use this mapping proxy type class to make, take any standard dictionary and
then to provide a read only view into that dictionary.
So you can still access all of the variables, all the values, all the keys, but you cannot
reach in and modify the dictionary.
I mean, sure.
You could just kind of pry that mapping proxy apart and then reach in and modify it.
And this is probably not a good idea, but.
That goes back to where the, like, you, you know, you shouldn't be doing it.
And if you, if you mess it up, it's kind of your fault, not the APIs.
It's like, yeah, you could install a kernel driver and patch the memory and.
I rooted my machine and then it crashed.
Oh, this thing is a crappy OS.
Yeah.
So that would, that, I mean, you could combine that for like a greater functional programming,
immutable pass by value type of behavior, right?
Yeah, you could do that.
I mean, personally, I'm always thinking like, okay, at what point do we reach the level where
we're really fighting against what the language wants us to do, but it is entirely possible,
you know, to do what we just talked about, you know, you have a tuple and then instead
of putting like dictionaries in there, I mean, you'd probably just go with a frozen dict class
for that, but you could totally also wrap it in this mapping proxy type thing.
And that way you really enforce that immutability as best as we can.
And I think that's so nice about Python.
The frozen dict is a little more obvious, but yeah, that is super nice.
One of the things that traditional, let's say C, C++ arrays to a lesser degree, like
some category of say C# arrays have is they have the same type in them and then they're
tightly packed.
And the tightly packed thing that I'm really thinking about, right?
So if I say list and Python and I put number one, number two, number three in there, that's
really a tightly packed set of pointers that points out to three things that are not anywhere
near each other.
Potentially.
I mean, those are probably like pre-allocated and put next to you.
Like I know one to 20255 or pre-allocated, but you know, in general, things in the arrays
are not nearby each other, which means you're following a lot of pointers and there's a lot of GC type
behavior, but there are structures that if you really want homogeneous types, tightly
packed, you can get them, right?
You can also get that in Python because like you said, with your standard list, there's really
no guarantees.
If your list of a couple of elements, it could actually be much larger than just those like
five or six elements because it's pre-allocated to a bigger size and kind of shrinking and growing
dynamically.
And so if you want absolute control over how your data is laid out in memory, then you
could use something like the array class from the array module that's in the Python standard
library.
And with this, you can actually say, Hey, I want an array off C style floats or ints, and
you can just put in your values there and they will be laid out in memory exactly the way you
request it.
But it won't accept types that don't fit the declared data type, right?
So if you say it takes floats, it won't take a string, right?
It enforces that as well as the packing.
Yeah, that's right.
So you get some type safety there.
So it's going to make sure that you're not putting in, I don't know, something crazy needs
to be consistent.
Let's skip ahead a little bit.
We're actually, I thought we would get through more of these.
I think we're going to run out of time pretty quick.
But there's some stuff I really want to cover while we're here, because I think it's, it's
quite interesting.
You talk about stacks and you talk about queues.
And I think this list that we were just talking about, it can be many things to many people.
It can act as a queue.
You can take the thing off the front.
It has a pop to take things off the end.
You can append that's like a synonym for push sort of.
But there's also other queues.
And there's also some really terrible performance issues if you try to use, say, like a list for
a queue.
So let's maybe touch on those things while we're on this topic.
Yeah, for sure.
So I mean, the list, again, it's such a versatile data structure in Python, right?
And it's, I feel like they made exactly the right performance trade-offs when they implemented
how list is actually implemented in the CPython interpreter.
But on the other hand, it leads to those situations where you're like, oh yeah, you know, I can push
and pop from a list.
So I can totally make a stack or I can totally use it to make a queue.
And I mean, you can, but the problem is, for example, if you use a list to make a queue,
you can run into a situation where you'll just have terribly slow performance.
Because if you enqueue an element in a list and you added, like you insert it at the front,
this will actually shifting all of the other elements by one.
So you're looking at a linear time operation just to insert one element.
And that's really, really bad for a queue implementation.
That's super bad.
Yeah.
If you're doing like the first in, first out behavior, right?
So append, that's fine.
It goes on the end.
But every time you take an element off the front, it has to entirely copy every element.
If there's a million items in your queue, that's a heavyweight DQ operation right there.
So ironically, you should use this thing called DQ to fix it.
Yeah.
So there's this great, great little class in the collections module.
And the collections module is just so awesome.
If you're listening to this, for anyone who's listening to this show and they haven't heard
or played with the collections module, check out the collections module.
There's, it's just a treasure trove.
And it's also fun reading, you know, just, just seeing what's available in there and the
reasons for why it exists.
And, but yeah, the collections module has this thing called a DQ or deck.
I think some people pronounce it.
And this is actually implemented as a linked list.
So you can append and remove from both ends really, really fast because, you know, it's no,
there's no shuffling around.
It's just, okay, cut off the head or add something to the end.
It's a single pointer operation.
It's super, super fast.
That's right.
And the DE at the beginning is not the operation of removing the item from the Q, but that's
the double ended Q, DEQ, right?
Yeah, that's right.
It's fast from, from the front and fast from the back as well.
Right.
And we have things like Q.Q and multiprocessing.Q and some other ones, special cases one.
So I think it's really interesting to think about these because a lot of them, you know,
like we said, you don't really discover them on your own.
You have to kind of study them and seek them out.
And you're like, oh, that's, I thought just Python was slow.
List turns out to just be a really bad Q.
So maybe I'll do something different, right?
Yeah.
So I think these are all really great, but I don't want to give people the impression that
your book is all about just data structures, right?
There's actually, you know, just one chapter.
It just happens to be a super rich chapter.
So maybe like we have some other stuff that was on deck that we're not probably going
to be able to talk about in terms of time, but maybe just run us super quick through like
what else you wanted to highlight.
When I wrote this book, really the angle it's written from is that I don't want to create
another technical reference, but I want to walk people through some actionable stuff and
explain things step-by-step with examples that make sense.
So I'm also covering kind of the classic features in Python that are often difficult to wrap your
head around.
And, you know, I don't know how much more time we have here, but for example, I think if
people understand decorators and like really understand how they work internally, that is
a huge leap from a beginner level understanding of Python, because you need to understand, you
know, a couple of rather advanced features, like you need to understand, you know, what
first class functions are that all functions can be treated are real full blown objects.
And, you know, you can not just call them, but also look like a hand them around to other
functions.
Right.
And closures, the concept of like variable capture and closures is front and center there.
And that is all super important stuff.
And just so by, you know, focusing on under really understanding how this feature works,
you can actually learn a lot about Python and you can pick up a lot of other features and
abilities that are going to help you in programming Python in general.
And so a big part of the book is, you know, I really spent a lot of time on this and it's,
it's been iterated over.
I tested this with people and just walking them through understanding how decorators,
for example, work, or, you know, walking them through iterators and then going from there
to generators and then later generate expressions and really just explaining the logical connection
between all of those and kind of structuring it away in a way where it's not just like,
oh yeah, this is how it is, but kind of approaching the topic step by step.
I think those are really, really nice things.
You certainly building a simple decorator, like come up with some concept where it makes sense.
And go through the motions of creating it will really, like you said, expose you to a lot.
Some other things you talk about, you talk about the sushi operator, and I'm not going to give any
more details.
People have to check out your book to learn about that Python has a sushi operator.
Design patterns like emulating a switch statement with dictionary dispatch, all sorts of stuff like
that.
So really just tons and tons of these little tricks.
And you said the, it's like a buffet, right?
So you don't have to take it in order.
Yeah, that's right.
So most of these chunks in this book or the sections in this book, you can actually,
you know, jump around and just kind of look for things that interest you.
They're usually pretty short and they all have a key takeaway section at the end.
So you can quickly review this.
And for example, I heard feedback on the data structures chapter where it's, you know,
it's a great way to prepare yourself for a Python interview and just kind of, you know,
get updated or get a refresher on how these standard data structures map to Python.
I feel like people kind of, they take the easy way out and they ask these algorithm questions
and these data structure questions in interviews.
And yes, it's important, but I don't necessarily feel like if I don't know the most efficient
particular queue for this type of operation, that doesn't necessarily mean I couldn't write
an awesome web app that has, you know, a hundred thousand users a month and has no real performance
pressure on it.
Right.
I agree with you.
I mean, unfortunately, some people will really bust out the trick questions where it's like,
ha ha, you didn't know about this like special case.
That means you're a fool and we're never going to hire you.
But the reason I don't like those is I feel like those tend to advantage the people that
have CS degrees, but not necessarily better programming skills than people who are self-taught.
So I really like that your book kind of backfields that information for people.
Thanks.
I mean, that was one of the goals, right?
Where it's like, yeah, a lot of this stuff, you don't need to spend like four years in
university just to understand what a hash table is.
Right.
And it's, I don't like it when people create that impression.
And so, you know, I feel like if you have a good practical understanding or practical programming
skill, then it will be in most cases, like pretty easy for someone to fill in those gaps in
our knowledge and, you know, to satisfy the people who ask the more computer science motivated
questions, I guess.
Yeah, absolutely.
All right.
Well, let's leave it there for the book because we are definitely going to run out of time.
So I do want to talk about a few other projects that you have going.
First of all, you have this thing called Pythonista Cafe.
Did you open like some kind of like coffee shop or something?
That's my lifelong dream, you know, open a coffee shop, write books.
Answer a Python question before you're permitted in.
Yes.
So Pythonista Cafe, it all started when occasionally I read Reddit or Hacker News and it can be great.
And it can also be like people have some weird personas on these forums sometimes, you know,
and people kind of lash out at each other and get really worked up.
And I remember this one time, like somebody made a post on Reddit, one of the Python forums
on Reddit and said, hey, I'm this 14 year old guy.
I really love Python.
I made these tutorial videos on my YouTube channel about, you know, building a Flask app.
What do you think about this?
And like the first person to reply is like, oh, you're teaching it all wrong and you should
have never created these videos and you'll basically, you'll never be a good programmer
and blah, blah, blah.
Who do you think you are?
He's probably like in the top, like 5% of all like 14 year old programmers.
I mean, come on.
And that is nuts, right?
Like, how can you, how can you, I was just almost like sitting in front of a computer
shaking and getting angry at this person.
And it just seems so crazy.
But the problem is, I mean, this is, this is real, right?
Like this happens.
I mean, most of the time it's a great forum.
It's a great community, but some people, they are just crazy when it, the knee jerk reactions
they get.
And I really liked that.
And so I reached out to a bunch of people and said, hey, so here's an idea.
We're going to do a Python forum, an online forum where people can discuss Python, you
know, you can bring their code and get feedback on their code.
And we're going to talk about all things Python, but we're going to do it in private.
So it won't be a public forum and there will be an application process to get in.
And I launched that at the beginning of this year.
So 2017, I don't know when the show is going to come out, but so far it's been a really,
really cool experience.
We have about 120 people in the forum right now.
And it is, I don't like the word safe space, but in a way it is a safe space where you can
just go and post some code and you're going to get like really, really positive and encouraging,
helpful comments.
And there's no danger of someone going in and be like, oh, you're the worst like 14
year old programmer in the world.
And you should go back to where you came from.
That's Pythonista Cafe in a nutshell.
Nice.
That's awesome.
So yeah, you've got that going.
Did you do a book or videos or something for Sublime for Python developers?
Yeah, I also did that.
So I have a couple of courses that I offer on my website.
And one of them is also on Talk Python training.
And so one of them is a complete course on setting up Sublime Text for Python development.
So like an end-to-end, step-by-step course.
And that consists of a couple of modules and, you know, like workbooks and video lessons on
how to get a really, really awesome and productive Python environment with the Sublime Text editor,
because that's my favorite editor.
I know you really enjoy PyCharm and I think that's a great option too.
But for me, Sublime Text has just been, it's just been a great setup.
And you can script it with Python, which is nice too.
It's definitely a very Pythonic place to be.
That's awesome.
So yeah, that's cool.
If people want to learn more about like getting the most out of Sublime, they can check that out.
You've got like the pip dependencies course available both on your website and Talk Python training,
which I thought that was really awesome.
I mean, people are like, well, I know PIP.
Like now this is like really, really the right way to sort of manage dependencies in your app
or other packages that you're using.
It's really great.
So I definitely enjoyed checking that out as well.
All right.
So I think you've given us a spoiler for this.
The last two questions here, at least the first one and the last two.
Favorite editor?
No, it's Sublime.
Bye, John.
Yeah, that's awesome.
Yeah, I think I like Sublime as well.
I definitely use it some of the time.
Very cool.
And notable PyPI package this time around?
Yeah.
So this time around, it'll be Django because Django 2.0 just came out.
That's big news.
What's the biggest piece of news about Django 2.0?
I think the new URL routing syntax is pretty cool.
It looks really nice.
And for me, Django, you know, it just represents so well what Python stands for.
It's really, you know, this is a piece of software that you can depend on.
It's mature.
And it's for when you actually want to get work done and not be worried about having
to rewrite your whole stack in six or eight months down the road.
And I feel like, you know, the Django team is so awesome.
And it's a huge achievement about what they've done over the last couple of years.
And yeah, so, you know, my pick is Django.
Awesome.
New 2.0.
Yeah, the shiny Django.
And the other major news around there is that they've dropped legacy Python support.
No more Python 2 in Django 2.
Oh, yeah.
Yeah, ironically, given the numbers there.
I think that's a super positive move.
Just the fact that Django switched their default behavior in their documentation from Python
2 to Python 3 made a huge dent in the consumption of Python 3 on PyPI.
And now I imagine this is just going to take it up a notch.
That's great.
Yeah, totally.
I mean, I saw that.
Like, I've been, you know, I have a couple of videos on YouTube on should I be using Python
2 or Python 3?
And just a couple of months ago, you know, people were like, oh, yeah, this is a valid question.
And now it's like every time I tweet this video, someone is like, there's no question
about it.
Like Python 3, like, don't go with Python 2.
So that's great news, I think.
I totally agree.
I feel like the last six months or so it really has become like, why would you even ask this
question?
Like, this is sort of a deprecated video.
You should just take it down.
But that's really positive for the whole community.
Awesome.
All right.
So final call to action.
People can get your book.
They can check out Python East Cafe.
There's all sorts of stuff they can do to learn more about this, right?
If you're listening to the show and you found interesting what Michael and I talked about,
and you want to learn more and kind of go from beginner to intermediate Python skills and
beyond without complex jargon and other crazy stuff, then you should download the free sample
chapter for my book.
You can just go to pythontricksbook.com or also just go to Amazon and search for Python
tricks.
And the book also comes with a free bonus toolkit where people can get more than two
hours worth of video material to help reinforce some of the main lessons in the book.
So yeah, go check it out at pythontricksbook.com.
Awesome.
All right, Dan, thanks for being here.
It's fun, as always, to chat with you.
Hey, thanks for having me on the show again.
It was great.
You bet.
Bye.
This has been another episode of Talk Python to Me.
Today's guest has been Dan Bader, and this episode has been brought to you by Linode and
Rollbar.
Linode is bulletproof hosting for whatever you're building with Python.
Get your four months free at talkpython.fm/linode.
Just use the code Python17.
Rollbar takes the pain out of errors.
They give you the context and insight you need to quickly locate and fix errors that might have
gone unnoticed until your users complain, of course.
As Talk Python to Me listeners, track a ridiculous number of errors for free at
rollbar.com slash talkpythontome.
Are you or a colleague trying to learn Python?
Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course, Python Jumpstart by Building 10 Apps at
talkpython.fm/course to experience a more engaging way to learn Python.
And if you're looking for something a little more advanced,
try my Write Pythonic Code course at talkpython.fm/pythonic.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes,
Google Play feed at /play,
and direct RSS feed at /rss on talkpython.fm.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
See you soon.
I'll see you next time.