Django is a very popular Python web framework.
One of the reasons is you have these many building blocks to drop in for large sections of your application.
Need a full-on admin editor backend?
There's a few lines of code and boom, you have a basic table editor.
This appeals to many people.
But for those of us, myself included, who appreciate lightweight frameworks where you choose just what's included
and piece together our apps from best-of-breed components, can find this a turnoff.
This week you'll meet Julie Ellman and Mark Lavin, authors of Lightweight Django,
who are here to dispel the myth that Django apps have to be built out of large building blocks.
This is Talk Python to Me, episode 88, recorded November 21st, 2016.
Anyone can get the job done.
It's the execution that matters.
I have many interests.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities.
This is your host, Michael Kennedy.
Follow me on Twitter where I'm @mkennedy.
Keep up with the show and listen to past episodes at talkpython.fm.
And follow the show on Twitter via at Talk Python.
This episode has been sponsored by GoCD and PyUp.
That's right.
PyUp.io has joined the show as a sponsor this week.
Thank them both for supporting the show by checking out what they have to offer during their segments.
Julia, Mark, welcome to the show.
Thanks for having us.
Yes, thank you.
Yeah, you're welcome.
I'm super excited to talk about your book.
You guys are doing some cool stuff.
I'm really looking forward to talking about this combination of technologies.
I mean, we're going to talk about Django.
We're also going to talk about Backbone.
We're going to talk about WebSockets, Redis, Tornado, all sorts of cool things.
And I really like the way that you put them together.
So we'll have a good time talking about those.
But before we get to that, why don't you tell us your story?
Like, how did you get into programming?
I think that my love of programming really started when I was pretty young.
And I was very fortunate to be able to learn how to use Logo on Apple 2e in about first grade.
And just kind of like being able to just play around, right?
So having the ability to do that, really started to get into it there.
But I ended up going into art.
And slowly after graduating from college, realized that I can't really survive on making art.
So I decided to go back to this thing that I was really interested in since I was a kid and started doing web design and development,
which back then there wasn't really programs for it.
So I ended up getting a job at an internal marketing agency at a small company and really learned there on the job.
And shortly after that, got a job at Hallmark Cards, where I was an interaction designer.
So doing a lot of e-cards, things like that.
Oh, cool.
So, yeah, yeah.
But then I met this guy at South by Southwest Interactive a bajillion years ago.
And he was like, so have you heard of Django?
And I said, no.
And he's like, oh, you live in Kansas City and you live so close to Lawrence, Kansas, and you don't know what Django is?
And I said, no.
And he's like, you've got to go work at the Lawrence Journal World.
I'm sorry.
And that person was Simon Willison.
Nice.
And he introduced me to a few people.
And I ended up working there for about a year and just learned all about Django and contributed to the actual project.
And that's when I really started working in Django was then.
It's a really cool way to get introduced to it.
It's basically go to the birthplace, right?
Yeah.
It was a really, really amazing time for sure.
Because that was like 2007?
2008 when I started working there.
So, yeah.
Like three years after it had become an official open source project.
Right.
It was just some kind of internal thing.
It was actually recognized as this big project.
Yeah.
Yep.
Yeah.
By then.
Yeah.
Nice.
I love Lawrence, Kansas.
I went to university there and I enjoyed my time in that town.
Oh, I rock chalk Jayhawk.
I'm a Jayhawk.
Yeah.
Right on.
How about you, Mark?
How'd you get into programming?
My father, programmer, a long time.
Maybe he worked at IBM.
My older brother always loves programming.
So, I resisted being a programmer, being a stubborn little child, wanting to do something
on my own.
I always kept coming back towards it.
I studied math in college and I dreamed about being a professor.
I don't know why anyone would dream about being a math professor.
But that's what I had.
There's a lot of cool stuff about being a math professor.
Job opportunities, not necessarily one of them.
I guess I just really love like tweed jackets and elbow patches.
And that's what I envisioned for my future.
And so I continued doing math in graduate school.
And there's a lot of programming that comes into doing some applied mathematics.
And as much as I resisted, again, I just kept coming back to it.
And when I left graduate school, I started looking for what could I do with this math degree that I had.
I was in New York City at the time.
And I got into finance.
I just got sucked right in to doing crazy math on Wall Street.
And it was fun and exciting for a time.
We were doing some really cool stuff.
Over-the-counter derivative pricing.
It wasn't as flashy as Michael Lewis sort of draws in his books.
But we were, it felt like we were on the cutting edge of financial math to some extent.
But there was a little bit of a financial crisis that you may have heard of, I don't know, 2008, 2009.
And while my company wasn't really negatively impacted by that, many of our clients were.
And it was a really stressful time to be pricing such a volatile market.
We were having a lot of questions about our work.
A lot of really bad market data that we were sifting through.
In that process, a lot of my work was about automating what we were doing.
Collecting data.
Parsing really messy files.
And trying to turn it into a reasonable data we could use.
And we were doing weird things in VBA.
Visual Basic.
Excel.
Yeah.
How much of the world runs on Excel as the, quote, programming back end?
Nothing you'll ever believe.
And then we also had applications that were written in C++.
And we had nothing in the middle.
And I started exploring sort of scripting languages is what I started looking at.
I started learning, I started learning, I started learning Perl.
And I had a friend from graduate school who was pursuing a PhD.
And he said, forget everything you know about Perl and learn Python.
And I took his advice and I did.
It didn't stick at work.
My boss wasn't really interested in change.
But it stuck with me.
And I kept playing with it for different, both scripting tasks and automating foreign things and solving math problems with it.
And when I decided to sort of leave New York with my wife and started pursuing careers outside of finance, I started looking at what else I could do with this sort of hobby.
And...
What's this Python good for anyway, huh?
Yeah, what's this Python good for?
And I had all this free time when I just, we just quit.
We just left.
Came and stayed with my family down in North Carolina, where I'm from.
Built myself a website in Django.
And I was having a blast.
I built a little route tracking thing for running.
And yeah, I was still searching for jobs in finance.
And then eventually I thought, maybe someone would just pay me to write stuff in Django.
And strangely enough, I found a contractor hosting at the Cactus Group.
It was like two guys working in an office in Carrboro at the time.
And started working with them.
Took on one contract.
Finished that.
Took on another.
And eventually got hired full-time with them.
And I've been doing full-time Django development with them for almost seven years now.
Yeah, that's really cool.
I like how you kind of just kept going and looking for that Python opportunity to work in the space you wanted to.
And eventually it worked out.
That's great.
Changed my life.
I mean, the advice to forget Perl and learn Python was the best advice someone's ever given me.
Now we come to about a year and a half ago.
You guys created this book called Lightweight Django.
Julia, what's the story with your book?
It wasn't at a year and a half ago.
No way.
Was it?
I think it was like two years ago.
Mark?
Yeah, it was since it got published.
Yeah.
Yeah, yeah.
It's almost two years to the day.
Yeah.
Sorry.
No, don't be sorry.
I just was kind of like, no, it couldn't have been that long.
It couldn't have been that long.
I thought it was like 10 years ago.
So, yeah, come up for the idea for the book.
So, the thing with Django is that a lot of people look at it and they're like, oh, it's so big and it's like a dinosaur and can't use it because there's too much and I don't need everything that's in there.
And, I don't know, I would just talk to Mark about that when I, like, I would hear this and Mark and I were both working at the same place at that time.
It's like, I don't know that I agree with this.
There's got to be a way that we can show people that they can use Django.
And this is also the time of Flask sort of coming on the scene and, you know, people started making all these little Flask apps and then all of a sudden they're like, oh, I probably should start with Django.
I shouldn't have done this.
Doing all this work.
Yeah, I made all these Flask apps and then I probably should use Django.
So, yeah, we kind of talked about it and we were speaking at a conference.
We were speaking at OSCON and I was like, what do you think about writing a book?
I've got way too much time.
Let's just see if we can get rid of that.
I remember that feeling.
She was just whirted out one.
Whether you're having coffee or drinks or something, we should write a book.
And I think I just said, yeah.
And I had no idea what we were going to write.
No, you did not.
No, you were like, you were like, I don't know.
You were smart.
You were much smarter.
I was like, we could do it.
It's fine.
We could do it.
And you're like, let me think about it.
I remember you're like, I got to think about this.
So, yeah.
And like calling it lightweight was sort of the antithesis of what people refer to Django as.
So, like, oh, I don't want to use that.
Yeah.
Rather than let's plop in Django ORM, let's plop in like an admin back in all these big building blocks.
Your book takes the approach of saying, basically, let's see how you can strip this down to a very small core and then build up your own like starter templates and tweak it however you need.
Right.
And you're also bringing together a lot of interesting technologies that are not just Django.
Mark, how do you pick those?
Pick some of the ones that we're most familiar with, either from projects that we've built or, you know, professionally or projects that we've built on the side, you know, experimental projects.
One that we knew would work together, one that we had some familiarity with and that had some of the same sort of staying power as Django did.
You know, Tornado is one that I really like.
It's been around for a long time.
And it does this one thing really well.
And it's not flashy.
It's stable and boring, just like I like my technology.
Yeah.
Tornado plays a really important role.
And definitely when we get to the eighth async section or the real time web section of this conversation, it will definitely come back to it.
So when you guys wrote the book, who do you have in mind for it?
Julia, like who is your ideal student or who should read this book?
Definitely an intermediate.
I don't know.
I've had this conversation with a few folks, like beginner, intermediate, and advanced.
It's like, is there really an intermediate?
Is there really like, is it black?
I don't know.
So when it comes to, I think it's more like people who have actually built a Django application and have some sort of familiarity also with JavaScript.
So I think that that is part of the audience.
The people who are also in, like have this infrastructure, they're already using Django.
They already have it as part of the applications and whatever product or platform that you're using.
And they're kind of locked in and they're seeing all these newer technologies kind of come out.
And they're like looking at them longingly and they just don't understand how to connect them all.
And so, you know, I don't know.
I think it plays to a couple, quite a few different audiences.
Ones that need to be convinced to use Django that maybe haven't used it before, but have programmed in other languages and other web frameworks.
Okay.
Yeah.
I do think it's interesting to see how these pieces fit together.
And, you know, Mark, to your point, it's really challenging to choose those.
So you're choosing Django, obviously, but you're also choosing Backbone and Redis and some of these other things.
And especially in the JavaScript front end space, that's a real topsy-turpsy world, right?
And so, you know, by the time you're done with the book, you don't want the framework to be passe, right?
I don't know.
I think I've had a couple of folks ask me why we chose Backbone.
And I think Mark put it very well, which is just kind of one of the, like, boring and stable things that work.
And I feel like Backbone is pretty straightforward.
There's nothing flashy or fancy about it.
So when it comes to those things, I don't know.
It's like, what's in fashion right now?
It's like, it works.
It works, you know?
Yeah, you don't want to get caught with bell bottoms in the 80s.
Well, yeah.
I think it's also a matter of it's so basic, simple, and straightforward.
A lot of the concepts in Backbone can then be played out in other front end frameworks that you're, or JavaScript frameworks that you're wanting to use on your applications, right?
Like, it's not going to be too dissimilar than something else that you're going to choose, whether it's going to be React, which is the thing.
I think at the time of this book, it was Angular, and people are starting to kind of, like, look at Knockout.
But I don't even know.
I don't really remember.
The debate, I think, we had was Angular versus Backbone or Ember.
That's right.
Yeah.
I know Ember is really popular in the Ruby community.
It follows a lot of conventions there.
I don't see it used quite as much in the Python community.
Maybe that's just my own bias showing through.
The concern we had with Angular was the steeper learning curve.
And we kind of wanted to focus more on how do these pieces work together rather than let's try to teach you Angular and Django at the same time.
And I think that's where Backbone kind of won out out of those three.
It had a pretty shallow learning curve.
And it sort of used the same terminology as Django did on the back end.
And they play fairly nicely together, which works well for the book.
Yeah.
And I think in the end, like you guys said, it's pretty simple.
So if you decided you wanted to go use Ember or something shiny new like Aurelia or whatever, just, you know, you could translate it pretty straightforward if you knew that other framework.
Well, not only that, but just showing how do they connect, right?
Like how do you connect these two things together, which I think is a mystery for a lot of people that have been showing that translation.
And then, oh, oh, that's how you do it.
Yeah, absolutely.
I think that's a lot of the key lessons that are the takeaways from your book are this piece connects to that piece in this way.
That piece connects over to this piece in that way.
You know what I mean?
So you guys are using Python 3 for your book.
And that makes me really happy.
I love it whenever I see Python 3 stuff being used.
And I've got some good stories on Python 3 coming up.
But was that a – how much did you guys think about that or how much just go, yeah, Python 3?
Two years ago, it was a different time than it is now.
I think at first we sort of thought, oh, we'll do dual code base.
We'll do 2 and 3 and just see how that goes.
You know, we've written a lot of reusable components that support both and familiar with doing those.
But as we started working through that, there's a lot of explaining, oh, well, why do we need this?
Why do we have that?
And it got a little clunky.
We spent a lot of time explaining the differences between Python 2 and Python 3.
instead of the message that we wanted to give, which is, you know, how do you use Django with Backbone?
Or how do you do WebSockets with Django?
How do you build a single file application with Django?
That's what we wanted to teach people.
And we were getting caught up teaching people the difference between 2 and 3.
So in the end, we abandoned Python 2.
I think that's for the best.
People shouldn't be building new applications from Python 2.
Hopefully they're not.
We feel strongly about that direction.
We also had a lot of support from another O'Reilly author, Harry Percival, who had just published a book that was Python 3 only.
And he felt very strongly about that same direction.
So we have received some criticism for making that choice.
I think someone called our use of Python 3 academic at one point.
But even two years ago, we were building applications with Python 3 here professionally for our clients.
So I mean, that's definitely the way to go.
We've had a disservice to write a book even two years ago based on Python 2.
To add to it as well, it's sort of like trying to push the community forward.
A little nudge, right?
Yeah.
If we only write it in 3, that's it.
It's like you just got to learn it and got to keep moving forward.
And I think sort of like aiding this Python 2 usage doesn't help towards moving everyone towards 3.
Yeah, I think all of us that are doing stuff in public, if we can take these small steps, they add up to make a big difference.
In aggregate, you know, one of the things that made a big difference in the Python 3 usage measured on PyPI relative to like the year previously or whatever was actually to do with Django.
And they switched the default language of their documentation and their tutorials to Python 3.
And that actually made like a pretty measurable difference in the amount of Python 3 usage, like as seen from PyPI downloads.
So let's talk about the first part, Julia.
So one of the things you guys start with is the world's smallest Django application.
What's the story of this?
So it's sort of like bashing that concept of this is following the instructions of how you actually build an application, like doing the polls tutorial application.
And this is the way you have to do every Django app.
And a lot of what that chapter is, is saying, nope, you don't.
You can actually just have one file to do it all.
So when we're trying to think of the progression of the chapters, that was something that kind of starting with, like, I really like how you were saying that you got the concept how the chapters build off of each other.
So we start with the tiniest little thing.
How much of that is like a reaction to Flask coming along saying, here's our one file.
That is the web app.
And you thinking like, wait, Django can do that.
It doesn't by default, but it can.
Totally.
That's totally.
Yes, of course.
That's right.
I think.
And it's funny because I've had a lot of people tell me and I'm probably Mark as well kind of come up and be like, after reading just the first chapter, like, I never thought of it that way.
Never.
Yeah, it's it is interesting.
Like, you know, the the starter projects there and, you know, like pyramid has it scaffolding.
And the thing that comes out looks really complicated when you're new.
Like, there's like, why are there so many files here and all this stuff wired together?
But it's easy to see that as required rather than just the conventions this community is following, you know?
Yeah.
Totally.
So another thing that you bring up and I thought was really interesting that you covered here were components or parts of this thing called the 12 factor app.
What's the 12 factor app?
12 factor is pretty growing popularity, you know, way of sort of building and deploying applications, sort of a set of principles around what makes an application easy to deploy and scale in multiple environments.
And we didn't want to preach too much, I guess, about that methodology.
But you talk about some of the challenges that people face when they are using Django.
I think this question of how do I have multiple settings files comes up all the time.
And certainly for that sweet spot of our audience of someone who's maybe built something and wants to deploy it, make it real, you know, put it out on the public internet.
Then they come with, well, how do I configure this for my local development and for this production environment?
And 12 factor is one way to do that way that I think works fairly well.
And so that's sort of the direction we try to nudge people in.
Yeah, sure.
And I thought it was pretty cool.
One of the philosophies you were talking about there is to try to have the minimal amount of deviation from like dev to QA to prod.
So the thing you're working on is close to the real thing, right?
Absolutely.
And making those differences obvious, sort of seeing which settings can be changed, which ones can't is part of that.
Yeah, absolutely.
So the next section that I thought was interesting and worth talking about, it's this part where you kind of speak to stateless web applications.
And you say that much of the focus in Django is about building reusable applications that can be installed and configured for like a Django project, but that can get really complicated.
And so you talk about breaking your web app into like small composable services.
And I think that's a recent trend that people have been leaning towards wanting to do microservice architecture.
You can't see my air quotes, but they're there.
And so, again, I think that's where people are reaching for other frameworks that they felt accomplished this better.
And part of the story that we wanted to tell was this can be done in Django, too.
You don't have to throw away your existing Django knowledge to build these types of applications.
In fact, Django has a lot of utilities that make it easy to build these types of applications.
One of my favorite things about that chapter in particular is I feel like we do a really good deep dive into some of the HTTP caching layers that Django provides, you know, those utilities that Django provides.
And I think that's really important for those types of applications at scale.
I agree as well.
Like, how much do you feel that developers are spending too much time chasing the shiny new thing?
I feel like, you know, especially around things like Node.js and the front-end frameworks, it's like, well, that's six months old.
We should be doing something different now.
You're doing it the wrong way.
It's old.
It's like, wait a minute.
Isn't that too quick?
I feel like it's not necessarily because as developers are always trying to strive to be better, faster, performant.
And I think that's, we're always trying to look of that optimal utopia of combination of things and combination of technologies.
Right.
So I think that's a, that's a big part of this.
Yeah, I agree.
It's a real challenge because it's fun to work on the new shiny stuff.
Right.
But at the same time, if you build an app that takes a year to build, you don't want to have like three different technologies mixed in because there were seasons, technical seasons.
So one of the things you guys did in this stateless web app section was you created like a placeholder image thing for prototyping out the site.
You could say, well, I don't really know what image the graphic artist is going to put here, but it's going to be 200 by 300.
So give me a 200 by 300 image, which is pretty cool.
And use this package called pillow.
People love to learn about new packages on the show.
Tell people what pillow is.
Pillow is the Python imaging library.
I mean, it's a work of, well, now it's really assumed the old pill module.
So it's really, I think, the gold standard for Python image manipulation.
If you need something to either modify or tweak an existing image or something like we're doing, which is generating new images, it's your go-to in Python.
This portion of Talk Python to me is brought to you by GoCD from ThoughtWorks.
GoCD is the on-premise, open-source, continuous delivery server.
With GoCD's comprehensive pipeline and model, you can model complex workflows for multiple teams with ease.
And GoCD's value stream map lets you track changes from commit to deployment at a glance.
GoCD's real power is in the visibility it provides over your end-to-end workflow.
You get complete control of and visibility into your deployments across multiple teams.
Say goodbye to release day panic and hello to consistent, predictable deliveries.
Commercial support and enterprise add-ons, including disaster recovery, are available.
To learn more about GoCD, visit talkpython.fm/gocd for a free download.
That's talkpython.fm/gocd.
Check them out.
It helps support the show.
Julie, the next thing that you guys did was you said one of the popular trends is to use something to generate like a static site.
So maybe it's data-driven, but the thing you actually want to deploy to the server is static because it's easy to scale, cheap to run, and things like that.
And so you showed how Django can actually be used to build static sites, right?
Static site generators is something that in my current role, I pretty much that is the thing that we use a lot in prototyping.
And a lot of what this chapter is about is about doing things like rapid prototyping, right?
So you have a client or even just like want to build something quickly or you want a blog and you don't need a database.
That's something that a lot of folks go to particular things like other frameworks like Jekyll, Pelican, and a bajillion others that I don't know the names.
So there's two ones that come to mind.
But there's a way that you can do this with Django.
And again, going back to the user that is sort of locked into using Django on whatever platform they're on and then also wants to be able to do something, like I said,
that they're using Django.
So it's like showing how you can actually do that with technology.
Yeah, that's neat.
And some of the stuff that you covered in there, I think when people are new to web development, they don't really know about it or they don't pay attention to it until it becomes challenging, let's say.
And you were showing how you can do sort of bundling, minification, cache busting, all that kind of stuff with Django, right?
That was cool.
Yeah, and asset management too, right?
Which is a huge thing.
Something I've been thinking a lot about lately is asset management in general and how it plays across all these different frameworks, right?
And all these different things like how do we compress files?
How do we take things that need to be compiled from one thing to another or copied over?
And I mean, that's something that just showing someone how to do that in general, I think, has a huge value.
So in this particular chapter, being able to show not only how to build something in Django like a static site generator, but also like asset management, which, you know, Django's always been something that I just love because you have so much freedom.
I feel like there's not a lot of rules around it, which is another theme of this book, right?
Like this underlying theme.
There are no rules.
Break it apart.
Smash it.
Just use one file.
Like create a static site generator.
Whatever.
Yeah.
Do what works for you, right?
Exactly.
And that's something that, you know, I love about the framework and kind of wanted to show here.
So.
Yeah.
I thought that was, that was really interesting.
So then Mark, you started talking about how to build RESTful APIs, because if you're going to have a single page app, a spa, it's going to be pretty boring without some server side communication, right?
Absolutely.
And I think we have great tooling in the Django community for doing this.
And Tom, for see his work relentlessly on Django REST framework, and we're all better for his work and contributions.
So, yeah, we wanted to show off sort of how easy and fun and elegant it can be to build, you know, RESTful API in Django.
So where do you think people get hung up building like HTTP APIs other than calling maybe it RESTful and it's actually not?
They get a fight about whether it's truly RESTful.
I mean, like, what do they get hung up in just about like mistakes that you see them making?
Well, there's certainly a lot of debates that people like to have.
And I certainly like to have my fair share of those base as well, but mostly about wanting good support for validating data that comes in and generating responses to go out.
And ideally, being able to accept different formats on both ends, being able to take in JSON and return JSON, being able to take in XML and return XML or YAML or whatever their favorite means of communication is.
So I do think people get hung up on trying to make things perfect or fit some vision of what they feel RESTful is.
But I think REST framework in this community gets you a good percentage of the way without a lot of work.
Yeah, and the fact that there's that nice framework that's really ubiquitous in the Django space makes it – there's a lot of shared knowledge on that, right?
That's cool.
Yes, and great complementary utilities.
It calls itself a framework, and I think it's true because other people are able to build off of it.
It's not let's cram everything together, everything that anyone could ever want.
It's let's give it extensibility and let other people in the community add things that might be missing from the core.
And, yeah, there's just a lot of great activity going around that project.
Yeah.
Okay.
Yeah, that's really cool.
One part you guys were – I don't remember what part of it.
I think it might have been this section where you were using the forms classes for validation
that you would typically use for HTTP get post type forms, and you were applying it for validation.
Was that in the RESTful part?
We did that in the stateless application.
We actually validated the incoming path expressions.
Oh, right, right, right, for the sizes of the images and stuff.
Yeah, I thought that was cool.
Why don't you tell people just real quick about how that's more flexible than, say, just straight HTTP angle bracket form?
Well, that's, again, a big theme of the book.
And maybe it's an unspoken theme, but it was definitely a big motivator for us is showing using Django,
you know, the pieces of Django like it's a collection of libraries rather than a framework.
It's something you can kind of take and use as needed.
And the forms library is, you know, the forms section of Django is an amazing piece, not just for the HTTP aspect,
because, you know, it does help you with sort of rendering HTML forms on the front end.
So the biggest piece is about that validation.
You know, we've used forms to validate incoming, you know, CSVs, all kinds of crazy stuff.
And it's got a really simple API, you know, nice declarative, you know, these are the fields that should be there.
And then it takes a dictionary and it says this dictionary is valid or is not valid.
And it has methods for, you know, cleaning data that it comes through.
Right.
And it's not just this value type has to be a parsable integer.
It's like this has to be an integer between 1 and 10 or something like that, right?
Like lots of nice little rules and whatnot.
Absolutely.
And as you said, that dictionary, that blob of data doesn't have to come from an HTTP request.
They're really decoupled in that way.
That dictionary can come from anywhere.
And I think it really speaks to, you know, maintainers of the Django project to build something that's as flexible as it is.
Yeah, that's excellent.
That's very nicely decoupled there.
That's cool.
So I want to make sure that we spend some time on the last third of your book or last quarter, I guess quarter, but maybe telling people about the middle part.
So they know kind of what kind of apps you're building and so on.
So, Julia, the idea was you're going to model some kind of like Scrum Kanban board type thing, right?
Yeah, and we were trying to come up with an application like what we would use in our day-to-day and basically how that would tie together.
But also, like how can we sort of start with like a simple, a super simple part of it, right?
And build from that.
So that's sort of the concept of why we're going to build like a Kanban board.
Yeah, so basically with Backbone, you build a Kanban board and you can edit it through Django and so on.
And then you get to this part where you're like, well, it's cool that we have this board, but it would be way cooler if we could edit it together on the web.
And so this brings you to a section called Real-Time Django.
Like tell us about that.
Yeah, Real-Time Django.
Yeah, so what is Real-Time Django?
That sounds cool.
Sounds fun.
It can mean anything.
Yeah.
But Real-Time just covers like such a host of sins.
Many sins have been committed in the name of simulating something real-time, right?
So this is like I don't want to have to refresh my browser.
I want to open a page like Google Docs or something, right?
And we can all type and it just, it changes as we all work with it, right?
That's the basic theme.
Yeah.
And like also bringing like it's that whole concept of like you don't think Django can do this, but it can, you know?
Yeah, that's really, really excellent.
So you guys run down some of the options for Real-Time Communications.
You're like, well, we've got WebSockets.
We've got server-sent events.
We've got WebRTC.
There's even like old-school long-pulling.
And which one did you end up picking?
We ended up going with WebSockets and we spelled this out kind of why.
I mean, the server-sent events, you know, are maybe a better map to how we end up using WebSockets themselves.
But WebSockets are just better known and better browser-supported.
Definitely two years ago, they had much better browser support.
Most frameworks that gear towards doing, you know, real-time web typically ship with WebSocket types of support out of the box.
And server-sent events are not.
It's usually kind of an add-on.
So we ended up going with WebSockets.
Yeah, certainly two years ago.
That makes sense.
I think those kind of features have come a long ways in two years, right?
It feels like the HTML5 stuff has definitely grown quite a bit from 2013, 2014.
Yeah.
I mean, it's every day.
People would just keep pushing and pushing and doing crazier things.
And it's cool to see those technologies have definitely matured to where two or three years ago, something, you'd say, well, maybe this could be an add-on.
It's probably not required for your app to really be functional.
And now I think, you know, people are really gearing towards more apps where it's like a core piece of functionality that absolutely needs to be there and work for the app to be useful.
Yeah, I totally agree.
And you guys point to a website that I'm a big fan of called caniuse.com.
And as you're talking about these features, you're like, oh, check this out on caniuse.com.
Why don't you tell people quickly about that if they don't know about it?
That's a good resource.
Yeah, it's a place where you can see if a particular feature is going to work on various different browsers, but not just the browsers, but the actual version of the browser.
So if you're not quite sure if this is going to be available, a lot of what I do now is making sure things work on really, really old browsers for many reasons, many good reasons.
So, yeah, I use that all the time.
Yeah, you know, the web is always like a compromise at some level, right?
Like, oh, there's this new cool thing.
Oh, but it's not supported on IE.
Or there's this other great thing, but it's not supported on Safari, on mobile.
Then you have to decide, like, do I really want to have a degraded experience for these people?
Like, and so on.
And sometimes the answer might be yes.
Like, for example, if you want to use CSS3 2D transforms, the only browser that that doesn't work on, according to caniuse, is Opera Mini.
And it might be totally reasonable to go, I don't care.
It's just going to look bad on Opera Mini.
Yeah, but there's fallbacks that you can actually use.
If you want to go forward and try something, there's definitely fallbacks that you can use.
So I encourage those that are listening, like, don't feel like you can't move forward with those particular technologies.
Right.
There's a lot of times some kind of polyfill you can do, huh?
Yeah, Modernizer is a big one.
Yeah, absolutely.
Let me take just a moment and tell you about one of our sponsors that makes this show possible.
This portion of Talk Python is brought to you by PyUp.io.
Have a website based on a popular framework like Django?
You really want to make sure that you're on the latest point release that includes all bug and security fixes.
With PyUp.io, you no longer have to search for updates manually, possibly missing an important update for a security vulnerability.
When you add one of your GitHub repos to PyUp's online service, the PyUp bot immediately starts searching for requirement files.
It will send you one big pull request that updates all your dependencies, and the bot constantly checks for new releases on PyPI.
This PR contains all the info you need in one place, and your test will run automatically, telling you if an update broke your code and why.
Try it today by visiting PyUp.io.
That's P-Y-U-P dot I-O.
Accounts are free and come with one private repo.
If you need more, plans start at just $4.99 a month.
Okay, so you decide WebSockets, and then the real challenge, and I think I wanted to ask you guys about this in a more broader sense, not just the context of your book.
But in order to do WebSockets, you have to have these long-term, many, many, many connections back to, you know, if your site's successful, hundreds of thousands of browsers, right?
And WSGI and a lot of the Python frameworks, Django, Pyramid, Flask, they're not really built to handle the asynchronous web or these long, long, long-running connections or HTTP2.
A lot of this stuff, do you think it's time for, like, a WSGI 2 or some rethink of how web processing is done?
Yes and no.
I mean, it would be nice to get a refresh of something that is definitely geared more towards long-term connections over HTTP.
But I still think that there's a lot of web applications that are still really request-reply based.
And that's fine.
There's always going to be that need.
You don't want to throw sort of everything out in the pursuit of something that has that support.
HTTP2 is probably a really good one where, yes, you want something to, yes, you want something to hold the connections open and make use of the multiplexing that they allow.
But that doesn't necessarily have to be exposed at the framework level for it to be useful.
Right.
That's true.
Like, for example, Nginx supports that already.
Yeah.
So maybe if you're using GeniCorn or Microwiskey behind Nginx, like, your actual data-driven Python requests might not do HTTP2, but the rest of it could, I guess.
Yeah.
What's interesting or confusing may be in this HTTP2 world, you know, a lot of the things that we've been doing and some of the things that we've even touched on earlier, concatenating files and minifying them all together,
some of that really loses its usefulness.
And instead, you want to make those be additional requests.
And you potentially want to be able to do browser pushes on your page load and say, you know, here are all the CSS files that are going to come with this page.
Here's all the JavaScript files.
So we kind of have to unlearn some of the things that we've done and then add new hooks into our frameworks to be able to make use of the new features.
Pretty interesting idea.
I wonder if Google will start ranking sites higher that follow HTTP2 someday.
If that happens, I suspect it'll push people quickly.
Like, for example, they recently started ranking SSL encrypted sites higher than non-encrypted ones.
Mobile friendly ones for mobile searches higher and so on.
Well, they certainly rank or they value, you know, page load time.
And to the extent that making good use of HTTP2 can help your page load time, I think you'll see that reflected whether they make an explicit HTTP2, you know, differentiator or not.
Yeah, it's still up in the area.
So one of the things that you did is you said, well, because this is how I kind of got out of this conversation is like Django and the whiskey servers in general are not super for this long term connection story.
So you brought up Tornado.
And so tell us about Tornado.
Like, what's the use case here?
Well, as I said before, I really like Tornado.
It came from FriendFeed before they were acquired by Facebook.
And it's been kept going ever since.
It's a it's a it's a it's own sort of event based network IO for concurrent, you know, highly concurrent web applications.
And it has a small web framework that comes with it where you're basically talking directly with the socket.
A little bit different than how you write a whiskey application.
You know, a lot closer to the raw TCP connection.
And so part of it was sort of love of Tornado.
It does great things.
It supported Python 3 for a while.
And again, that really goes with sort of our message of pushing Python 3 forward.
What we wanted to capture was, OK, Tornado is awesome.
You want to use Tornado.
You already know Django.
How can you make them play nicely together?
And I think that's the most important aspect of those chapters is let Tornado do what Tornado does well and let Django do what Django does well and learn how to make them work together.
Right.
Absolutely.
So you can focus the WebSocket processing on a small Tornado site and then do the rest basically through Django.
Right.
And the trick is how to make them talk together because it's pretty easy to set up a WebSocket system.
So you can say, well, here's all the listeners on this channel.
I made some change broadcast out a message to everybody.
This thing changed.
And maybe it's not a message, but a JSON document that like tells backbone to update the UI in this way or that.
Right.
But then they could also go to the website and like add a new card to the Kanban board.
And you want them to see that.
Everybody else.
And you want to do that through WebSockets.
So tell us quickly sort of what's the connection there.
How do you pull that off?
Essentially the choice that we made was to expose.
So Django server already had a REST API.
That's what the previous chapters were all about.
And that's what the browser is primarily talking to, you know, adding new items to the board and moving them around.
And then along with the WebSocket, you know, the Tornado server has its own sort of small.
I don't really want to call it RESTful because it's not very RESTful.
But another HTTP endpoint where it can accept changes that come over.
So essentially the browser makes a request to the REST API on the Django server.
And that change gets propagated over by making another HTTP request over to the Tornado server.
And then that gets broadcast out to any of the connected WebSocket clients that might be interested.
Okay, cool.
So the flow goes something like the JavaScript client talks to Django via REST.
Then the Django server tells the Tornado server via an API that this thing has happened.
And then Tornado pushes that back down to everybody except for maybe the person who made the change via WebSockets.
Is that right?
Yeah.
And it's kind of a server.
That's pretty awesome.
One thing that you mentioned there that I hadn't played with or really thought about is Redis Message Broker.
Sure.
Because so far the stuff we talked about would work well if like Tornado, the process is managing all the connections.
But what if you have like scale out to where you've got like 10 Tornado web front ends or WebSocket front ends, whatever you call it.
How do you get that to work?
So maybe tell everyone how you were working that out.
This is one of the things that I kind of love and hate about this section.
I think I would probably rewrite it.
We chose Redis because it's simple and it sort of gets the job done.
But I would have really loved to use Rabbit if I didn't feel like it would have added a ton of complexity here to sort of talk about AMQP and all those things.
So we owned Redis.
Basically Redis is the inter-process communication protocol between the Tornado instances.
So Django doesn't know anything about Redis and doesn't publish to it directly.
Redis is just the intermediary between if you're running multiple Tornado processes.
And it's got great PubSub support and it's broadcast only, but that works for our example.
So it's definitely sort of the familiar pragmatic approach to doing a simple broadcast example.
It seems like a good choice.
I mean, you don't want to spend a whole other chapter on like message brokers and whatnot, right?
Yeah.
I mean, we wanted to make it so that it could scale beyond a single process because it's a big pet peeve of mine when I sort of see this sort of toy example.
I say, look how easy it is.
And it's like, well, it's easy because it doesn't scale like beyond a single process and it has no fault tolerance or high availability possibilities at all.
And it's really disingenuous to say, oh, this is, oh, look how easy it is.
Well, it's easy when you cut out all those things that people care about in your applications.
Yeah, that's a good point.
And for WebSockets, it's more important to have some kind of like fault tolerance, right?
Because people are constantly connected.
It's not just an instantaneous request and they have their page.
They're like connected and waiting.
But yeah, Redis is easy to get up and running with.
People are pretty familiar with it.
A lot of people use it for caching as well.
And it's nice to kind of have technologies that can do a role when you aren't pushing the boundaries of scale on it that you can make use of Redis in other ways once you have it.
It's really interesting.
I didn't think about that use.
That's great.
So I think that's about all the time we have to talk about the book.
But I found it really interesting.
I like the take on the lightweight aspect.
I like the take, Julie, like you said, like, but you can do that with Django.
So here, I'll show you.
Nice work.
Thank you.
Yeah, you bet.
So you said at the beginning how you got started, you know, like, hey, let's write a book.
What kind of advice do you have for others who are thinking about writing a technical book?
One of the mistakes I feel like I made was assuming like, oh, yeah, I can write a chapter like that quickly.
You know, I could do that like so quickly and one a week.
It's going to be what?
Like, well, you split it up.
That's like a month.
We're done.
I think I think we had gotten like maybe Mark.
It was like two chapters done.
And we were like, oh, we've got this.
Like, this is no big deal.
And that was a mistake.
So my advice would be to write 50% of the book before you even pitch it.
Like, get it.
I mean it.
Like, just write it.
Like, and at least get a really healthy outline down because you think you know, but you don't.
Yeah.
When it comes down to editing, especially when it comes to a technical book, because you're not just writing content about like and teaching content.
And Michael, I think you understand this for like what you do every day.
You know, it's not just about that.
It's also about refinement and testing and verifying and seeing how can I make this better?
It's like this mega, mega project.
So.
Yeah, for sure.
And if you are like try to make a jump that's too big, you're going to lose people.
Right.
And so you've got to make sure that you can continually carry them along.
And it's a big challenge.
Or if you're too academic in your use of Python, which I love that Mark brought that up because we weren't sure if that was a bad.
I mean, I think they gave us like one star for that.
We're like, I'm actually pretty proud of that.
I think that's actually kind of cool.
Throw me in with Stanford at MIT.
I'll take that.
That's awesome.
Two questions I always ask my guests on the other way out the door.
So, Julie, I'll go with you first.
What is your favorite PI PI package?
We have over 90,000 now.
And I'm sure there's some that you use that are great.
Well, I really enjoy keeping my things neat and tidy, especially when it comes to using Git.
And while branches are cheap, I still like removing them when I'm not using them anymore.
So I really like this little package called GitSweep, which verifies what's been merged in and what hasn't and cleans them up.
Oh, nice.
So if you do like a pull request and that the request is accepted, the GitSweep will automatically delete the branch, something like that?
Yeah, yeah.
So you can just keep everything nice and tidy, hence the name GitSweep.
Okay, that's awesome.
And then, of course, PyLadies, which has a very dear place in my heart.
Nice.
What is the PyLadies package?
I know what PyLadies, the organization, is.
So it's actually the kit.
So let's say you want to start a PyLadies in your local area.
Then you can just pip install PyLadies.
And then you can get a whole information package of how you can get started.
And I believe that the package is, I can't remember the name.
Sphinx.
It's built off of Sphinx.
So I don't know if it's based off of Read the Docs or it's just straight up Sphinx.
I can't remember.
Sure.
But yeah, it's just a nice little thing.
Linroot made it because she's amazing.
Yeah, that's awesome.
Yeah, I had her on the show.
She's great.
Mark, how about you?
My favorite package is Django.
Obviously, it's my livelihood.
And I'm thankful to have found it.
And beyond that, I mean, I like a lot of sort of linting tools.
I'm really grateful for Flake 8 and the suite of tools around PyCode style and PyFlakes.
I make a lot of typos.
And so it helps me catch when I misspelled a variable in one place to say, hey, you're
trying to use a variable that's never been used.
Or, hey, this variable that you thought you were using isn't being used.
You just spelled it wrong.
It helps me find a dead code or code that is imported and not used.
So I really always install Flake 8.
Yeah, that's great advice.
All right.
So the other question I always ask is when you write Python code, what editor do you use?
Julia?
I don't have really a favorite.
I think it's whatever I need to use at the time, depending on what space I'm in.
Because I use Sublime, but I've also used Atom.
And I use Vim sometimes.
I just kind of like don't want to leave command line.
But it's not like, I don't know, I just don't really have a favorite.
It's whatever works at the time.
Whatever I'm trying to do.
Be flexible.
Awesome.
Mark?
As pretentious as it sounds, I really like them.
I switched a couple years ago.
And it's been really nice.
It really helps me focus to just sort of see one file open at a time.
I have to be very deliberate about moving out of that file, moving to another one.
And whereas I found myself, when I switched from Sublime, and I just found myself kind of having
a thousand files open in a billion tabs and clicking around everywhere to try to find where I was going.
And just Vim has really helped me focus on one task at a time.
Oh, yeah.
That's a really cool advantage.
All right.
You guys, final call to action.
What do people do to get started your book or take advantage of this whole idea that you're promoting as lightweight Django?
Well, if you haven't checked out the book, it's available for purchase on O'Reilly's website.
You can get e-book or a beautiful dead tree version with a hummingbird from the front.
Nice.
And I'll be sure to put the link in the show notes so people can check it out.
We have our code examples up on GitHub, which you can see without purchasing the book, though.
They maybe won't have a lot of context around them.
Additionally, I recorded a webcast with O'Reilly about intermediate topics in Django.
So if you sort of fit that audience of, you know, I've done a little bit of Django and I'm not sure where to go beyond the polls sort of tutorial, you can check that out.
I cover using Django along with Celery for background task processing.
And I talk about using Playgate to keep your Django code clean and up to date.
All right.
Awesome.
Julia, Mark, thank you for being on the show.
This is a really interesting take on Django, and I enjoyed talking about it with you.
It's been great talking to you.
Thank you for having me.
This has been another episode of Talk Python to Me.
Today's guests have been Julie Elman and Mark Levin.
And this episode has been sponsored by GoCD and PyUp.
Thank you both for supporting the show.
GoCD is the on-premise, open-source, continuous delivery server.
Want to improve your deployment workflow but keep your code and builds in-house?
Check out GoCD at talkpython.fm/gocd and take control over your process.
Are there bugs or vulnerabilities hiding in your app's dependencies or even in their dependencies?
Stop guessing and start getting notified by registering your GitHub repo with pyup.io.
Are you or a colleague trying to learn Python?
Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course, Python Jumpstart, by building 10 apps at talkpython.fm/course
to experience a more engaging way to learn Python.
And if you're looking for something a little more advanced, try my Write Pythonic Code course at talkpython.fm/pythonic.
You can find the links from this episode at talkpython.fm/88.
That's right, I've added a new short link.
So anytime you want to get to a particular episode, it's always talkpython.fm/show number.
There's all sorts of cool stuff on the show pages, so be sure to check them out.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm.
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
Corey just recently started selling his tracks on iTunes, so I recommend you check it out at talkpython.fm/music.
You can browse his tracks he has for sale on iTunes and listen to the full-length version of the theme song.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Smix, let's get out of here.