It's been an amazing year for Python. We've seen its meteoric growth continue to become the most
popular major programming language. We've seen significant grants and funding come in for open
source. And this just might be the year that the Python 2 or Python 3 question was finally settled.
Join Brian Okken, my co-host from our Python Bytes podcast, and me as we look back at many
of the major milestones for Python in 2017. This is Talk Python to Me, episode 145, recorded
December 21st, 2017. Welcome to Talk Python to Me, a weekly podcast on Python, the language,
the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on
Twitter where I'm @mkennedy. Keep up with the show and listen to past episodes at
talkpython.fm and follow the show on Twitter via at Talk Python. This episode is sponsored by
Linode and Rollbar. Please check out what they're offering during their segments. It really helps
support the show. Talk Python to Me is partially supported by our training courses. Have you tried
to learn Python but got stuck or lost focus? We know how it feels to try and jam fact after fact,
loop construct after turn hair expression, into your head. At best, it's boring. At worst,
it can turn you off programming altogether. That's why we built our course, Python Jumpstart by building
10 apps. This course guides you through carefully planned applications. It starts simple, but
progresses to quite real apps. Best of all, you won't be learning dry facts. You'll be learning like
the pros do by building real applications and learning in context. If you want to start building with Python,
just visit talkpython.fm/course to get started.
Before we get to the interview, I want to share a quick update about our Python courses with you.
Do you work on a software team that needs training and could really use a chance to level up their
Python? Maybe your entire company is looking to become more proficient. We have special offers
that make our courses here at Talk Python the best option for everyone you work with. Our courses don't
require an ongoing subscription like so many corporate training options do. And they're roughly priced
about the same as a book. We're here to help you succeed. Send us a note at sales at talkpython.fm
to start a conversation. And here's a bit of a timely announcement. We're giving away a PyCon ticket
to one lucky individual. So every year PyCon in the US sells out. If you don't have a ticket,
go buy a ticket or, or you could become a friend of the show. Many of you already are. And thank you
for that. If you're not just go to talkpython.fm/friends into your email address, and you'll be
in the running on January 8. We're going to do a draw using a little random dot choice and figure out
who the lucky winner is who wins a free ticket to PyCon. So be sure to become a friend of the show
if you want to be part of that. Now let's get to the interview. Brian, welcome back to Talk Python.
Thanks for having me back for this special episode. I always love these year and review shows and
it's really excellent that you could be here to do it together with me. So you were on previously a
couple years ago to talk about the testing column, how the testing pyramid was broken and we need more
testing column, right? I guess. I can't even remember what we talked about. So that makes sense.
It's been a long time, but I do remember the testing column is now a thing after that episode. That was
cool. So people may know you from our work together on Python Bytes, our other podcasts. They may know
you from your test and code podcast and they may know you from your book, right? Yeah. Just so this last
year got the book, Python testing with pytest with pragmatic publishers published this last year. So
yeah. Yeah. Awesome. That's definitely one of the things we'll review as we go through here. So we're
going to get together today and talk about what we think were the major accomplishments, major pieces of
news, not necessarily the major projects, say, in terms of like projects you find on GitHub or something,
but more like the trends in Python in 2017. So I think that's going to be a lot of fun. Do you want
to kick us off with what may be one of the biggest trends out there? Yeah. One of the biggest trends
that we saw in 2017 is a dramatic shift from Python 2 or legacy Python to Python 3 or modern Python.
I love the term legacy Python and modern Python or just Python and legacy Python. I think it's actually
making a difference. And Matthias, who originally started that trend, well done, man. That is awesome.
So I think this is finally the year where the question seems silly to ask. Like it's been a few years.
We're like, yeah, you can do Python 3. You can do Python 2. You know, how is this progressing? It should
really be going quicker. But to me, it feels like if somebody says, I'm going to start a new project
with Python 2, you kind of smack them and go, what? What are you doing, man? Come on. Like,
it seems like that's the feeling in the industry these days, which is really good, right?
I definitely think so. And I think there's been a switch. Hopefully there's been a switch in
the education, like in colleges and stuff, people using Python 3 instead.
There's a couple of milestones around this that we wanted to highlight, right? One of them is this
article by ActiveState, the guys at ActiveState called Python 3 versus Python 2. It's different
this time. They really lay out some of the things, some of the facts around this. For example, 95% of
the top 360, don't know why they chose that number, 360 packages listed on py3readiness.org,
95% of those are Python 3 ready. So traditionally, the blockage has been this sort of catch-22 around
packages not supporting Python 3, right? Yeah. A couple of years ago, that was definitely the case
of people trying to switch and they found out they had some key package that hadn't been migrated yet.
But for the most part, that's behind us. Yeah, absolutely. So Django, Django played a big role
this year in sort of putting a nail on the Python 2 or legacy Python coffin, right? Right. I mean,
even just this month, Django 2.0 came out and it does not support Python 2. It's 3.4 and above.
And with a little side note that 3.4 is only going to be supported for another year-ish. So move up to
modern Python. Yeah, that's awesome. 3.5, I guess. But I think if you're going to move up, may as well
just move up to 3.6. Right. Yeah. The only reason I think to not move to 3.6 is if you're working on
like a Linux distribution and it doesn't support it or it's a complete pain to like a non-standard. But
even so, 3.5 already has async and away. It has all, it has type hints. It has a lot of the magic
features that you really care about, which is awesome. Yeah. So Django, the current shipping
version of Django does not support legacy Python. That's awesome, right? Yeah, that's awesome. And I
think that Django, when it used to default to Python 2.7, I think that was part of what was
keeping people behind. Yeah. Even previous to this, I actually don't know the timing on when this
happened. It was a little while ago, but they switched the default instructions from their
tutorials or getting started tutorials from saying use Python 2 to saying use Python 3. And that made like
a significant statistical differences in just the consumption of Python 3 libraries on PyPI just from
the documentation switch, which was pretty powerful. Let's talk about some of the, there's a few other
switches. Heroku switched their default to 3.6 recently. Yeah, that's awesome. It was like 2.7.13
or something. And so now if you go to Heroku and say, give me a Python app, you get Python 3 unless
you say no, no, I mean legacy Python, excuse me. NumPy has laid out their plans for dropping Python 2.
Looks like you've got till 2020. Yeah, the NumPy folks are not moving super quick, but the fact that
they say we are dropping Python 2 support should put it on the radar of everybody in the data science
world, like same thing, start moving, right? I think their motivation is very similar to the Django
folks in that they've had extra work and extra burden. It's harder for people to come and
contribute to the project because they have to contribute two versions sometimes, a Python 2
and a Python 3 version of everything. They're like, why? Why do we have all this code? So they're like,
not only do we think it's time to move, but we think we will have a better chance of adding new
features and getting new contributors if we drop this Python 2 thing. So that's pretty cool.
This was laid out at the last 2017 PyCon was a talk with Instagram switching to Python 3 and how they did it
on one branch. And that was amazing. That was insane, right? But you're right. So I think that was a great talk and that did show to everybody else. Man, if these guys can do it with a very complex code structure, you can too.
My biggest takeaway from that was, oh my gosh, they did that on the main branch.
They made that change without branching.
They got it working in production and everything, which is pretty awesome.
And the whole way in which they did that was cool.
But my main takeaway was, whoa, if they can do it, pretty much anyone can.
There may be a few companies, like let's say Bank of America.
They've got an enormous Python 2 code base.
But unless you're at that scale, this is sort of a case study in how you could make this happen for you.
What's interesting is we started the year with still talking about trying to get people to move on to Python 3.
And at the end of the year, we're not really talking about that so much anymore.
We're talking about some of the new exciting projects that are Python 3 only, like API Star and all the type int libraries and stuff going on.
That's really cool.
API Star is by Tom Christie, same guy that did Django REST framework.
Things like I want to re-envision a REST framework, but with Python 3 features.
Really interesting use cases of things that are only available in Python 3.
And then we have, you said, like we've got PyAnnotate, mypy, and MonkeyType, which are all around this type hinting, type annotation thing.
And it's available only in Python 3.
I started the year not thinking type hints were a good idea, but I think they are now.
Everybody else has convinced me.
I try to use them judiciously, but I do really like them.
I think on this section, on this topic, our first one, this move to Python 3, we should leave people with the Python 3 death clock.
Yes.
Or the Python 2 death clock.
What, PythonClock.org?
Yeah, PythonClock.org.
And so we have about two years, three months, and 14 days until Python 2 drops, even security fixes.
All right.
And so if you have a manager or your team's like, I don't know if we should really do this switch.
We should just stick with Python 2.
We already all know it.
Like put this up on a big screen somewhere and just let it count down.
It's a little scary now.
I think in about two years it'll be real scary.
A similar timer shows up on some of the Python libraries, the documentation, like requests has that at the top, I think.
Yeah, that's awesome.
All right.
So let's move on.
Speaking of requests, you'll find requests on PyPI, right?
Yeah, along with a whole bunch of other things.
A whole bunch of other things.
What if you misspell requests?
That would have been a problem for a while.
Yeah.
Yeah, absolutely.
So one of the things that happened this year was that hackers subverted PyPI and they got 10 malicious libraries installed.
And the malicious code is not in the library itself.
They literally took the open source code and just put it into their library.
But when you pip install things, it has the setup.py that runs to configure everything.
That can do anything.
And so they put malicious code in there.
It was a weird thing.
It was almost, it was kind of bizarre.
What surprised me is that they had it running so that the actual package, the real code was still running.
It was just the setup was all messed up.
That's right.
So you would not be able to determine that you had installed one of these virus packages by it not working.
Because it was literally the same code.
Like you imported it ran.
It just happened to be really, really bad.
So the way they did this was by typo squatting, basically.
If you type requests but you misspelled it somehow, right?
They might create a package that is this virus injected version by copying requests and uploading it to PyPI under misspellings of these various things.
Yeah.
It's just wacky.
But the community kind of got around this and got it dealt with pretty quickly, which was pretty nice.
The couple things they had, the new pip won't let you do misspellings of some of the common packages.
You're not allowed to create standard library names, right?
Like they used URL lib or something like that.
Like that was not actually a package, but you would reasonably import it.
So you, you know, they could trick you into pip installing it.
Anything that's been seen has been taken down.
All of these malicious libraries are gone now.
And then, well, hopefully.
And then somebody went around and squatted on a whole bunch of names to make sure that it wasn't taken by somebody else.
Yeah.
So they're like, all right, we're going to just put a message or some sort of import or install warning here and just say like, no, you can't, you can't have these.
There's also some, some utility that would generate variations or misspellings of your package.
So you could squat on your own if you run a popular package as well, which is, which is pretty nice.
So I don't know.
On one hand, this is really disturbing.
On the other, if you look at how many times those things were downloaded, they're in the, like the tens or hundreds, not millions.
So that's one thing.
One of the good things about this, I think, that came out is, I mean, some people's first reaction might be, why didn't the team that runs PyPI, why didn't they proactively fix this or know about it?
And it highlighted the fact that all these people are volunteers.
And maybe we need to do something about that.
Well, yeah.
And PyPI itself is utterly critical for making the Python ecosystem work, right?
Anytime you set up a new machine, you set up a new project, you do an automated build, there's probably some sort of pip install dash r requirements dot txt process that happens, right?
And that flows through PyPI.
The amount of data that flows through there is super incredible.
And it turns out that there's just a handful of people, as in one or two, who work on this at all.
And like you said, they're volunteers.
So one of the big pieces of news is that Mozilla came along to fix this, right?
Right.
So the next thing we were going to talk about was the huge grant, the huge, in open source terms, $170,000 grant from Mozilla to help with the sustainability of PyPI, which is just awesome.
Yeah, that is really, really great.
I'm so glad to hear that they're doing this.
It's, on one hand, it's not a huge amount of money, but it really does put the foundation in place.
A lot of this is going towards moving over to warehouse, I think.
So that'll be good.
I think that'll help with a lot of open, a lot more people being able to contribute to the maintenance and update.
That's really interesting.
One of the problems with PyPI, the website, is that it's super old and crusty.
It's not like Shiny Flask.
It's not cool new Django 2.0.
It's like hand-rolled, custom, couple of files, thousands of lines, funky code.
And it's great that it works, but it really needs to be modernized.
And so one of the things that this includes is moving over to what you already mentioned, warehouse.
Warehouse is the new implementation.
And that's what you'll find at PyPI.org, not the PyPI.python.org slash PyPI.
Why that's there twice, who knows.
But the new one, and that's actually written in Pyramid.
It's a well-factored code with unit tests.
I think that move is a great move.
And I'm excited to put the old version behind us as soon as we can.
It still works great.
But I understand when people complain about having to maintain some legacy code that's crusty.
Yeah, and the problem was people would want to come and submit a feature for it.
And they would look at it and go, actually, no, this is a complete mess.
I can't work on this.
I'm out of here.
And so we have this sort of dual problem.
One, that there's people who are working on this critical thing only in this volunteer mode, more or less.
And then there are people who want to contribute to it, but its current state repels them, technically speaking.
So it's like a double negative, right?
For some of this funding to try to convert over, then maybe we can get swarms of open source people to help out.
Yeah, that'd be awesome.
So I'm looking forward to that little pre-release, pre-production warning at PyPI.org to go away.
Thanks for Mozilla helping on that.
And just related really, really quickly, you talked about major open source projects getting funding.
NumPy got a whopping grant for $645,000 to improve it over the next two years.
That's awesome open source support right there.
That's a lot of money, but that's a drop in the bucket for the people that are really leaning on data science a lot.
Yeah, it definitely drives a lot.
What I would just like to see is Django get that.
Pyramid get that.
Request get that.
You know, other, like so many of these popular open source packages that are these third-party packages are so critical.
So many companies making billions of dollars on it.
They could throw in a little bit.
A little bit of support, right?
It does seem interesting that, I mean, requests is used, for instance, as an example.
There's so many like that, though, that are used by so many people that, and they still have, you still have to have day jobs.
That's right.
All right.
So let's move on to something slightly political.
But I don't want to make this overly politicized.
But I think there is something we really should touch on a little bit here, and that is I can see positive sides of the whole Democratic side of the U.S. government, the Republican arguments some of the times.
But this anti-fact, anti-science, anti-climate science bent of the current administration, it's real danger to like a lot of what we really need to focus on.
I mean, climate change, in my opinion, is the biggest challenge we are facing currently as humans, period.
And it's been in danger, right?
It doesn't even matter what side of the fence you're on.
Taking data down is like data on climate change should be, it's just data.
It's not something we should argue about.
Right.
Facts should be facts, period.
You can argue about the interpretation, but facts are facts.
And they're really important for us to make the right decisions.
And so one of the huge challenges is there was a suspicion that when Donald Trump was going to become president, that many of these government-controlled, super important data sources were going to disappear, right?
Like NOAA, the Oceanic Institute, EPA, National Renewable Energy Laboratory, all these different places, right?
Theoretically, some of this data could be vanished, right?
I don't know what the current state is, though, of some of that data, but some of it is at the very least harder to find.
Yeah.
And you hear about things like there being forbidden words at the EPA, like you can't use the word climate change, for example.
How scary is that?
So there were these data activists who decided we're going to save this, right?
Yeah.
So they went off and it started right at what?
Right at January 20th, the inauguration, started copying as much data as possible and storing them in European servers.
That's a pretty, I mean, it's a huge effort, but that's a pretty cool thing.
Yeah, it is.
So there were 60 scientists and programmers who gathered at the Department of Information Studies in UCLA.
There was a bunch more in Toronto, Philadelphia, Chicago, all over the world.
And I'm sure requests and beautiful soup and a lot of that was in full force going around just sucking all of this data off these websites.
And you might say like, okay, these people are paranoid, whatever, right?
But in fact, right at noon, the day Trump was sworn in, a lot of the climate change pages on whitehouse.gov just disappeared, right?
So those were grabbed just in time, I think.
That's just weird to take stuff down right away.
Quite an interesting data hacktivist sort of story.
I mean, they weren't actually hacking things, but they were definitely out there using code and probably a lot of Python to try to rescue the data, which is quite interesting.
There are definitely probably people using Python, but also one of the things that I'm glad we highlighted the story is that there is not a, you can't separate your interaction with the governments and the world and the fact that you're a programmer.
And in this story, we see that programmers are able to save things that politicians might want to get rid of.
So it's good.
It was really, really heartened to see this.
Quite an interesting story.
This portion of Talk Python to me is brought to you by Linode.
Are you looking for bulletproof hosting that's fast, simple, and incredibly affordable?
Look past that bookstore and check out Linode at talkpython.fm/Linode, L-I-N-O-D-E.
Plans start at just $5 a month for a dedicated server with a gig of RAM.
They have 10 data centers across the globe, so no matter where you are, there's a data center near you.
Whether you want to run your Python web app, host a private Git server, or a file server, you'll get native SSDs on all machines, a newly upgraded 200 gigabit network, and 24-7 friendly support, even on holidays, and a seven-day money-back guarantee.
Want a dedicated server for free for the next four months?
Use the coupon code Python17 at talkpython.fm/Linode.
All right, so another thing that I feel like was really significant this year was the whole async and await, asyncio thing in Python.
We saw a lot of new web frameworks come in in 2017, or at least we heard about them.
I don't know how many of them are production ready, but they're definitely on the way, which is pretty cool.
One of the challenges for any of these web frameworks to be async enabled, and maybe it's worth mentioning why that might be good, right?
Especially because you have the gill, right?
It's hard to add parallelism to your Python code in simple ways.
But usually what do web servers do?
They call other web services, microservice-type things, or external services, like payment gateways.
They talk to databases, and then they do a little bit of stuff.
Maybe they talk to the file system, and then they get back to you.
But a lot of it is this kind of orchestration bit, really.
And all of those can be pushed into the asyncio loop, right?
If you use, like, Postgres, there's an async version for Postgres.
Async library for consuming this.
You can await, like, a query, which is really awesome.
But if your web framework doesn't support it, like Flask doesn't support it, Pyramid doesn't support it, Django doesn't support it, you can't use that.
Like, it's useless, right?
You cannot use that in your website.
So you really need a framework that will, all the way down through it, allow you to sort of release these awaitable coroutines.
I'm probably talking through my hat on this one, since you're more of the web developer.
But we did see three interesting ones come along, and probably more.
Like, I'm not giving it, I guess I should probably throw AIO HTTP in there as well.
But the three that come to mind for me are SANIC, which is a new web framework entirely built on alternate implementation.
I think UV loop is what SANIC's based on.
So you can go and write code against your web framework and use all of this async and await stuff.
There's Jepronto, which is really interesting.
And Jepronto, I think, caused a lot of a stir because they originally published an announcement on Medium.
And they have, like, a really pretty impressive graph right on their GitHub page saying, we're getting millions of requests per second on a single server.
They show it next to Go and Node.
And it's like, I don't know, 20 times faster?
I mean, it's a lot, lot faster than the other things.
And so on one hand, they caused a big stir about people going, well, you're doing your benchmarking wrong for this reason or that reason or whatever.
So Jepronto is pretty interesting.
The thing that's unfortunate is it's kind of stalled out in the last 10 months.
It came on really strong.
And there's just not a lot happening in the commit history.
Interesting.
Yeah, but Sanic, on the other hand, it's going pretty strong.
And a new one that we recently covered on Python Bytes is called Quart.
And Quart takes the Flask API and makes it async and awaitable.
So you could take the code that you know and make it work, basically, in this async world that you want.
This one does seem to be fairly active as well.
I'm going to have the guy who's created it on this show on Talk Python in a few weeks as well.
So we'll dig more into it.
But basically, it's Python asyncio web microframework with exactly the same API as Flask.
But you can say your views can be async def method name, not just def method name, right?
So it's really, really cool.
Yeah.
Hats off to him for coming up with just taking the Flask API and reusing that.
Why come up with something new?
That's right.
That's smart.
I also saw Armin Roeneker talking about this kind of stuff, who's behind Flask, talking about some of these ideas.
And stuff around Flask on Twitter.
I don't know if there's anything officially happening directly in Flask, but I wouldn't be surprised.
That'd be a good thing to throw money at.
Hey, everybody throwing money at projects.
Yeah, exactly.
Let's make our, instead of switching to Go or something, let's just make our web frameworks actually do what Python can do.
One of the things that is at the core of all of our Python programs is CPython.
And if people want to contribute to it, how and where that's happened has kind of been a journey, right?
I didn't realize the history.
I knew that in 2017 it moved over to GitHub, but it's moved around a little bit.
It has.
It was somewhere where I actually don't even know really where it was pre-2006.
In 2006 it was on Subversion.
Then they moved it to Mercurial.
Back then Mercurial, Git, they were kind of vying for action.
But, and you know, Mercurial makes a lot of sense.
It's written in Python.
So Python should be hosted on Python.
It's kind of meta.
But in 2017 it really is the modern workflow for social programming and open sources on GitHub these days.
So the big news is Python, CPython code moved to GitHub.
I'm not going to make a judgment call as to whether that's a, whether all of the activity on GitHub is a good thing or a bad thing.
It just is.
So you can't ignore that most open source developers are using GitHub.
So it makes sense to put that there too.
Absolutely.
It's like saying, I want to go and do stuff on social networks, but I don't want to be on Facebook.
It's like, well, that's just the way a lot of it is.
I'm sorry.
Yeah.
So this was actually done by, sort of motivated by Brett Cannon a little bit.
And he said one of his goals was he would like the ability to review an external contribution from submission to commit all on a tablet while at a beach on Wi-Fi.
And he can do that with GitHub, but he couldn't do that with Mercurial and the other things.
So that's pretty cool.
It was also motivated by people submitting patches, but the whole flow of reviewing it, there's not the pull request style and stuff there.
So it's, it just wasn't really, the request would set idle for like a year.
You'd make some change to CPython and people wouldn't even like paid attention.
Right.
Oh, that's terrible.
That'll take away your motivation for sure.
Right.
Yeah.
And whatever we can do to get Brett on a beach more often.
Yeah, that's right.
Well, so well done.
Well done.
Everyone involved in that.
Now, the next one I want to talk about has to do with Python becoming a very, very popular language.
And this is not a, an achievement per se of 2017, but this is where certainly some hit some milestones and the world has started to pay a lot of attention to it.
The growth of Python, I mean, it's been growing fast for a long time, but having it overtake some of the other languages in certain surveys happened this year, which is great.
One of the places I love to get this data from is Stack Overflow and they have their yearly developer survey, but they also have on staff data scientists that just analyze stuff and write reports about it.
Right.
Which is pretty awesome.
So one was called the incredible growth of Python.
And they said June, 2017 was the first month that Python was the most visited tag on Stack Overflow.
Not JavaScript, not Ruby, not C#, not Swift, Python, which I think is pretty awesome.
Yeah, it's great.
Hopefully that's not because it's the most confusing language.
What I thought was really interesting is you can go read the article, the incredible growth of Python on Stack Overflow.
And if you look at the comments and comments, at least at the time when it came out, were turned on.
So people were commenting at the bottom.
It read a lot like, you know, people were sort of disappointed that their favorite language didn't make it.
Well, Ruby's used so much in Silicon Valley.
Why is Ruby not here?
Or I don't, you know, I think Python must be so complicated.
That's why it's so much, you know, it's got so many visits, right?
You should just use C++, something simple that doesn't need explaining.
You know, and you're like, wait a minute.
No, there's no way that C++ doesn't need more explaining than Python and its language syntax.
And, you know, there's a lot of things that could contribute to this.
I think partly PyPI might actually be part of it.
Like, if you want to talk about requests, that's got a Python tag.
You want to talk about Django, that's got a Python tag.
You want to talk about SQL CanE, that's got a Python tag, right?
Like, all these different things around Python that people use and love also contribute to this factor.
Yeah, and we're seeing so many different groups like use Python that weren't before.
There's a lot of people not using R anymore for data science and machine learning, switching to Python.
We've got technical people, universities, a lot of people that used to use visual programming in the science and engineering fields are using free Python tools now.
Yeah, absolutely.
It's just great.
Yeah, and so there's two graphs I want to call people's attention to.
One is projections of future traffic for major programming languages.
So Python, JavaScript, Java, C#, PHP, and C++.
And you see the graph and almost all of those major languages are either flat or slightly downward trending, whereas Python is like hockey stick.
And then the projection has it just blowing away from all these.
It's really incredible.
The other one has to do with Python compared to smaller growing technologies.
So things like Rust and Go, there's so much hype around Go and how this is going to change the world and everyone should be there.
If you compare the amount of interest in Python versus interest in Go, it's like 20 to 1.
It's incredible.
And then not just that, but the rate of the graph of interest in Python is actually growing faster than in Go or Rust or any of these things.
So I feel like people probably hear, oh, you should switch to Go.
You should go write that in Rust.
You should use R or whatever, right?
But in reality, these are not the same thing.
I don't think things switch too quickly either.
One of the things that came up, I think it was in this article, is the growth in high-income countries in Python is faster than a lot of other countries.
The non-high-income countries, I don't even know what that means, but tend to follow by a year or two.
So we should see even more growth worldwide in Python in the upcoming years.
I think it's awesome.
You know, on one hand, I feel like people probably say, oh, well, programming is not a popularity contest, right?
Like, so what if it's more people use this versus that, right?
A lot of people use PHP and PHP is crappy.
Well, on one hand, PHP has a downward graph, right?
So one of my rules of thumb is when you look at the trajectory of a technology that you want to bet your career on, that graph should have a positive derivative.
It should be pointing upwards, preferably increasingly so.
A second derivative is being positive, also good.
But you should not necessarily bet your career on downward graphs.
And also, if you think about, like, what hang-ups people have when they start Python, it's not things like, like, see, you got to come up with memory management and all that stuff that's really hard to deal with.
There's a lot of hard to deal with stuff in C++.
And I spend a lot of my time in C++, so I kind of love it too.
But the big hang-up for Python is just the spaces thing.
And that's easy to get over.
And then you're on to making great things.
Yeah, you're definitely on to making great things.
You know, as soon as you pip install, like, a couple amazing things, you're like, oh, wait.
Spaces, they're cool.
I can deal with this.
And then after a week, you're like, wait.
Why do I have to put curly braces?
What are all these symbols?
Like, this is so annoying.
Yeah.
Yeah, pretty funny.
Pretty funny.
So I think this is definitely interesting.
People out there thinking about where they should focus for their career.
Definitely have a look at this article and draw your own conclusions.
But I think it's really a generally positive thing for everybody involved in Python in a lot of different ways.
Yeah, especially us.
Yes, especially us.
People love to listen to us talk about Python news.
Awesome.
So one of the things I feel like has been the trend really since the internet, before the web, since the internet came out, early William Gibson days has been like this real fascination with technology.
Technology is, it's almost assumed that it's going to make lives better.
We can connect more.
And look, if we connect on Facebook more, we can share our views and they might be different.
And then we'll come to a greater understanding.
There'll be less fighting and less disagreement and whatnot.
That's actually seemed to play out for quite a while.
But I'm starting to feel like there's a trend saying maybe not so much anymore.
Yeah, I guess I'm not quite sure where you're going with this.
Well, there's a couple of really interesting articles and sort of trends coming out around how technology is not necessarily, by default, a positive thing for a human society, for society.
Right?
So there's a couple of articles.
One is called Tech People Think They Aren't, Are They Good Guys?
They Aren't.
That was in Wired.
It came out recently.
There was one in the MIT Technology Review called The Dark Secret at the Heart of AI.
And then the last one is The Real Threat of Artificial Intelligence on The New York Times.
That's a pretty big turn from where we have been over the last 20 years.
That's true.
But I think in new technologies and programming, people don't really understand what it is.
And so they're often scared of the unknown.
Yeah, that's for sure.
I think, you know, the tech people, they think they're the good guys.
They aren't.
This article in Wired, they talked a lot about how people in Silicon Valley,
they really focus on Silicon Valley, but also in tech in general.
They need to be aware that they're losing this view where, well, these startups and these
tech startups, they're good by default unless they prove otherwise.
Right?
They kind of have to prove that they're doing the right thing.
And so, you know, we used to talk about how big data was going to do a lot of good things
for people.
Also the dangers of it.
But Netflix recently had some kind of social media post that was like, hey, we just noticed
that there are 10 people that watched this, I don't know, some Christmas show or something.
We just noticed there's these 10 people that watched the same Christmas show for 18 days
in a row.
How sad or something like that.
I mean, like, that's crazy, right?
No.
There's a lot of like wondering like, okay, well, why would you watch the same show 18
days in a row?
Maybe it's just on in like some store and they always want to show this Christmas show.
Maybe you fell asleep 17 times.
You just want to finish it.
That's like, this time I'm going to finish it.
I'm getting a coffee.
I'm going to watch it.
I can't even remember what the show was, but it wasn't something amazing.
I feel like there's this interesting trend of people being more skeptical.
You know, we have Apple and the encryption more sort of thing going on with Congress and
the FBI at the beginning of the year.
There's a lot of stuff there.
Being cautious is a good thing.
Like things, people adding things like Google Home and Amazon's, whatever Amazon's thing is
called.
Their Echoes and various things.
Yeah, into their house and in other forms of presence in your life that didn't used
to be there.
We do kind of have to trust these companies to not do anything evil with all of that information.
And yeah, maybe it's good to be a little skeptical and have some people checking out, making sure
that there's nothing weird going on.
That's for sure.
And I do feel like the default behavior or default assumption was, well, it's Google.
They're probably fine with my email or it's Amazon.
What are they going to do?
You know, I'm just buying a book from them and so on.
And it's kind of not so much that way.
I have an Amazon Dot, which is one of the Alexa type things.
And it's sitting here.
I made it quiet so it wouldn't disrupt the podcast.
And the way you do that is you press this mute button on it.
Okay.
But the color that it changes to when it's not listening is really interesting.
It changes to a bright red ring.
Like there's something wrong and dangerous about it when it's not listening.
Oh, interesting.
You would think it would turn, I don't know, green or white or like everything's safe.
Tell, you know, say your secrets, whatever you want.
Right.
We're not listening.
But no, it turns bright red when it's not listening.
Interesting.
Yeah.
Pretty.
What does it look like when it is listening?
It's normally dark.
But then if it's listening to you, it gets like a blue ring that kind of whirls around.
It's a pretty cool device.
I really like it.
But I only have it in my office, not, you know, the rest of the house.
So there's two other things around this that I also wanted to talk about.
And those were, I guess they're the same subject, but from two perspectives.
And that's sort of this artificial intelligence machine learning.
And so let me set the stage for you.
Like last year, there was this experimental vehicle just created by researchers at NVIDIA,
the graphics people.
And it was an autonomous car, but it wasn't like what Google or Tesla or GM is doing, where
there's some sort of conditional programming for it.
They didn't provide it a single instruction on how to do what it did.
Instead, what they did is they taught it how to drive by have it watch and observe what humans
do to cars while humans drive.
That's pretty interesting.
It's pretty interesting.
But then it leads, you know, like this type of thing is not just done with cars, although
it's pretty illustrative with cars.
It's also done with mortgages, with cancer diagnoses, various other things, police activity.
And so if this car crashes in unexpected ways, if it's going down the street and then it just
swerves over and crashes into a tree, how do you know why it did that?
You can't ask it.
It's just, well, it learned pretty well.
And most of the time it drives right.
And sometimes it doesn't.
And so there's some really interesting ramifications of like living with machines that are integral
to our lives that can't be separated from human society once we depend upon it.
Like at one point we didn't need the internet, but imagine if you took it away now, right?
The economies would crumble.
So there's actually a proposal for a law in the EU saying that you need the ability to
have the computer tell you why it made a decision about you, right?
So if you are denied for mortgage, a mortgage loan, why?
Right?
Things like that.
And so these full on deep learning AIs that just learned through observation, they can't
tell you why they did a thing, right?
They're not like looking at themselves knowing why.
Possibly not.
But I think that systems can be made such that they are interrogatable.
Yeah, I hope so.
But I think it's a really interesting trend that we're seeing more and more of this AI
stuff appear, you know, up and down our interactions where we know less and less about it.
A couple of interesting things about that is now I don't have the reference for it, but
there was some talk about how soon should we have driverless cars on the streets?
And one view is that we should make sure that they're very, very safe.
And another view is that they just need to be safer than people.
Yes.
If you lower the death rate, even if there still is a non-zero death rate, it's still
better than what we have now.
And that's interesting.
I think that's a super interesting trade-off.
And from a high-level ethics perspective, I think you say as soon as it is safer than
human drivers, if by deploying this technology, we save lives, it seems like the right thing
to do.
But, you know, if one of the behaviors of that life-saving AI is, well, there was an
oncoming car, so I swerved and drove off a cliff to avoid hitting it and killed my own
driver instead of I saw there were five people in the oncoming car, for example.
So I saved four lives, right?
Or maybe even more.
By driving this one person off, the news coverage and the human shock of being in control or subject
to one of these types of systems, it's going to be really interesting to see how that plays
out.
But still, from an ethics perspective, I think as soon as it's ready, to be safer.
But I remember a lot of these questions, like the five versus one, these came up with, I remember
these in philosophy in college, and I never really thought it would actually apply to technology
right now.
That's a really interesting point, because neither did I.
I remember like, well, if there's a burning building, and there's a baby inside, and there's
these people over here, and you could only save either the baby or the three older people,
like, which is the right, you know, these weird trade offs, you're like, well, nobody's
ever going to do that, right?
Like, nobody's going to make those conscious decisions to sit there and go, at best, it's
like you made a split reaction in an emergency, and you did what you did, right?
But when you're building these systems, you're like encoding it, basically.
Yeah, it's like, no, this isn't a rhetorical question.
We really need to know which one should we pick.
That's going to take some serious adjustments, I think.
Yeah.
Since I don't know a lot about machine learning and stuff, when I had Catherine Jarmel on test
and code, I asked her this, is it even possible to build systems that are interrogatable?
And her answer went over my head, of course, but she was confident that if those laws are
in place, that it's going to be possible.
So that's a good thing.
Yeah, that's interesting.
I think, you know, if you design for it, you can probably at least get some kind of, you
probably can design for it.
But, you know, we'll probably have to do that as an explicit requirement, right?
Yeah.
Because otherwise, you just make it work, and you're happy, right?
Yeah.
And the interesting thing about that is, I think, will benefit otherwise, too.
The reasons why artificial intelligence system came up with a conclusion might surprise us,
and it might teach us more about how we can better make decisions even without the AI.
When Google beat the world champion in Go, the top Go players started to study the non-intuitive
solutions that the AI was, you know, applying to the game to beat this guy, and they started
becoming better and learning new techniques.
It's pretty interesting.
And that'll hopefully happen in, like, detecting problems in CAT scans and things like that.
Right.
Reliability in airplane motors so they don't crash and stuff.
Awesome.
So I definitely think it's both very positive and kind of, you know, we're going to go back
to philosophy class as a society as well.
Yeah.
Pretty interesting.
This portion of Talk Python to Me has been brought to you by Rollbar.
One of the frustrating things about being a developer is dealing with errors.
Ugh.
Relying on users to report errors, digging through log files, trying to debug issues, or getting
millions of alerts just flooding your inbox and ruining your day.
With Rollbar's full stack error monitoring, you get the context, insight, and control you
need to find and fix bugs faster.
Adding Rollbar to your Python app is as easy as pip install Rollbar.
You can start tracking production errors and deployments in eight minutes or less.
Are you considering self-hosting tools for security or compliance reasons?
Then you should really check out Rollbar's compliant SaaS option.
Get advanced security features and meet compliance without the hassle of self-hosting, including
HIPAA, ISO 27001, Privacy Shield, and more.
They'd love to give you a demo.
Give Rollbar a try today.
Go to talkpython.fm/rollbar and check them out.
So another big piece of news that we got was that Python is starting to appear inside of
some major applications.
Yeah.
Surprising ones, I would say, actually.
They definitely surprised me.
Yeah.
We may have jumped the gun on the Excel and Python, but hopefully that'll happen.
Yeah.
If we can encourage Microsoft to do that.
I think it's definitely one of the highest voted things, but it's not a thing yet.
So one thing, let's start with something that is out right now in 2017.
And that is SQL Server, Microsoft's database server, which is ubiquitously installed inside
these large corporations, right?
It's like inside these large Microsoft Windows enterprises, like SQL Server is just the de facto
database.
And what's interesting is now Python 3.5 ships inside SQL Server so you can do in-process machine
learning against your data.
Oh, that's great.
Yeah.
Cool.
Yeah.
So if you've got 100 gigabytes of data in your database, instead of trying to suck that across
the network or move it around, you just write a stored procedure that is in Python and you
can use all the Anaconda libraries, TensorFlow, all that kind of stuff like in process, which
is really going to be pretty awesome.
Yeah, that's great.
So that's a concrete thing that has happened.
A thing that may happen, has got a lot of news in the last couple of weeks is Excel.
I'm not sure who this came from, but I think it was part of the, what is it?
People vote on new features or something?
On UserVoice, they have a, like, what new features do you want in Excel?
And this is, I think, the most popular one.
And then they sent out like kind of a survey to find out why people want it.
We pushed it on Python Bytes to try to get more people to answer this survey.
I personally don't care a whole lot about SQL Server or Excel, but especially with Excel,
if currently what you use to program Excel to automate it is this thing called VBA, which
is like a crap, it's even for Visual Basic, it's like a crappy cut down version of that,
right?
It's like a bad programming language.
And so there's this push to get Python to be the new macro programming language of Python,
of Excel inside Excel, which would be really awesome because within companies, like Excel
is probably the biggest database, in quotes, that companies have.
And so many people work inside of Excel and are sort of Excel experts.
And if we could get all of those people somewhat interested in Python, that would be just positive
across the board, I think.
Yeah, definitely.
It'd be great.
Yep.
So everyone out there, go upload this thing on User Voice.
We'll put the link in the show notes.
It might increase the use of Excel, which might not be a good thing.
Yeah, that's true.
Maybe it would be negative.
Try to keep that down.
All right.
So the last one for the year has to do with Google, right?
Google implements Python on Go.
Is that what we have here?
That is what we have.
So the people at YouTube created this thing called Grumpy, which I don't know about naming
open source projects, but if you want people to love it and to be attracted to it, call it
Grumpy might not be the best name.
But instead of having Python implemented in C, as it is in CPython, it's that underlying
implementation written in Go with some of that.
So Go's sort of claim to fame is like asynchronous from the scratch, like parallel from the very
core of it.
And so if you want to do the parallel asynch.io type stuff we were talking about, then having
Go as the foundation makes that incredibly easy and powerful.
Do you have any idea where this stage is?
Is it still something that's going along?
Are people contributing to it?
That's a pretty interesting question.
I'm not entirely sure what's going on with it.
I actually think it's a great name.
It's because people grumpy about having to use Go can use Python instead.
It's not super active.
The last check-in was November 21st.
So that's, and you know, that's about a month ago, right?
But this also could just be, you know, how much is Google pushing this out?
I think what this was, was really a research project for what would it look like if we implemented
the foundations of Python instead of with C, we'll do it with Go.
And the reason is, this is from the YouTube team.
And they process, well, YouTube is running the front-end parts on Python 2.7, I believe.
And they're processing over a million requests per second.
So they have, let's say, unique parallels and requirements.
Yeah.
Right?
And so they're like, okay, well, what can we do if we wrote this in Go?
And it's pretty interesting.
But it's, like you sort of pointed out, it's stalled out a little bit in the last four or five
months.
So I'm not sure if it's going to go somewhere, if it's like being worked on internally, and
then they're going to release it, or I don't know.
Maybe somebody from YouTube can contact us and tell us what's going on.
I did have them on the podcast.
So I had a guy named Dylan Trotter on the podcast who was a major player in this at YouTube and
talked about it.
But that was back on episode 95.
So the news is not super fresh.
One of the things you brought up here in the notes is that we often think about Python as
just CPython, but like this Grumpy is another version that's running on Go.
But there's a whole bunch of others that...
There's a crazy bunch of others for people who don't know.
Yeah.
So you have another article called How Big is the Python Family?
Yeah.
This is from the Czech.io guys who have like the gamification of Python learning.
And they wrote this article called How Big is the Python Family?
And so we already have CPython, Jython, Iron Python, or one variation of Python on .NET.
Then there's Python 4.NET.
There's Cython.
There's PyPy.
There's MicroPython.
And now we have Grumpy.
And there's also some others with Java that I can't remember the name of, but it wasn't
in the article.
I think that's cool that people are tackling it for different things.
I'm really excited to see MicroPython keep growing.
Yeah.
I'm actually really excited about MicroPython.
So those of you who don't know, MicroPython runs on microcontrollers, so like $5 chips.
And basically your Python program is the running operating system on there.
And you can just do whatever you want.
You can do insane things like you can actually take a Lambda expression and hook it directly
to a hardware interrupt on the microcontroller.
Yeah, that's just awesome.
I mean, it's such a cool little project.
Nice.
All right.
Well, that's our year in review.
I know we missed a bunch of things.
And if you feel like something's important, go to talkpython.fm/145 and put an entry
in the show notes.
But before we kind of round out what we want to talk about, there's a couple of honorable
mentions left there, Brian.
I think it was notable that you, both the podcast Python Bytes and Talk Python, hit every week
of the year in 2017, which is pretty awesome.
Thank you.
Yeah, that's actually quite awesome.
And thanks for your help with that, because I went and tooled around Europe and the Middle
East for like four weeks.
And you and some other folks kept it rolling.
So awesome.
Oh, that's right.
We had some guest hosts.
That was fun.
That's right.
And Test and Code had seven episodes.
I'm going to try to get an eighth out before the end of the year.
That's not as impressive, but still going.
No, but you did 52 weeks of Python Bytes as well.
And that was awesome.
Those were very fun.
And the ones that you did have on Test and Code were really good.
The Chaos Monkey and some of the other ones, really, really nice.
All right.
Thanks.
One of the most, like the most recent one with testing in machine learning and data science
with Catherine was really popular, which is great.
That's cool.
Yeah, she does a great job.
The other thing, we didn't bring them up because they weren't huge stories to talk about.
But I think there's a trend to more and more people embracing virtual environments with
things like PipEnv coming along.
And I think that's a good thing.
People should be using more of those.
So yay.
Yeah, yeah, absolutely.
We have PipEnv.
We got Tox.
And there's some other ones as well.
I think Fade was one that would do interesting things.
There's a lot of libraries that are really trying to make virtual environments just seamless.
It's really nice.
I've been using PyCharm a lot.
And it has picked up its game with using local virtual environments, too.
And that's a good thing.
I love that.
Create a new project that says, all right, new virtual environment and a well-known folder
within this project.
Unless you want to say otherwise.
This is what we're doing.
I really think that's the right move.
Yeah, it's great.
Awesome.
Another big thing, which you hinted at at the beginning, is you shipped your book, which
was quite the long labor of work.
But you have your Python testing with pytest out.
And that's been going strong, right?
It's been phenomenal.
And people have been sending me pictures on Twitter of them and their book all over the
world.
I've got one picture from France and one all over, a couple in Africa.
It's pretty cool to see all over the world.
It's great.
Yeah, it's been great to sort of sit on the sidelines and watch the reception of your
book.
So it's on Pragmatic.
You can get it there in paperback or digital.
And you can also get it on Amazon, right?
Yeah, definitely.
Awesome.
Okay.
So it's time to ask you the two questions that are always at the end of the show.
All right.
So what is your...
If you're going to write some Python codes, let's say it that way, what editor do you open
up?
Now it's PyCharm.
When you interviewed me a couple of years ago, it was sublime, but I have fully switched
to PyCharm.
Interesting.
I mean, that's my favorite as well.
Everyone knows.
I go on and on about it.
But tell me why.
Like, what was the motivation?
The integration with pytest, of course, is pretty darn good.
And they're working with me to try to get it even more integrated, which I'm pretty impressed
with the development team to be responsive to some of my requests.
Yeah, that's awesome.
They are very responsive.
The other thing is I write a lot of Markdown, and the Markdown plugins make writing Markdown
in PyCharm just very, very nice.
So it's got a side-by-side viewer, and it's cool.
I like it.
Yeah, nice.
All right.
And notable PyPI package?
Well, pytest.
Awesome.
But I want to add that when I'm using pytest, lately, I'm always adding the pytest HTML plugin
from Mozilla to make reports.
It's pretty cool.
Okay, nice.
So you get better, like, HTML-friendly, pretty output, not just text logs.
And it pops up, like, some JavaScript widgets so that you can filter and sort and all sorts
of great things.
Yeah, we started the show talking about, with our first item, talking about the Python 3
and the type annotation stuff with PyAnnotate, mypy, MonkeyTypes, and all that.
pytest has some PyAnnotate integrations as well, right?
Oh, right.
Yeah, I forgot about that.
That's pretty cool.
Yeah, I can't remember.
Was that PyAnnotate-pytest or pytest-PyAnnotate or pytestAnnotate?
I don't know.
I could put the link in the show notes, but yeah, it's pretty cool.
All right.
Well, I think it's been quite the year in the Python space for 2017.
So any final thoughts on what you've observed?
I just want to thank everybody for their support of both this podcast and all the Python podcasts
out there.
And I want to mention to people that if you like this sort of banter of a couple of people
talking about what's going on in the Python world, you should also check out Python Bytes
if you haven't already.
I also want to say thanks.
That's a great point.
People out there who listen and give us feedback and really enjoy what we're doing, that's why
we can do this, right?
So thank you, everyone.
And definitely visit pythonbytes.fm and subscribe over there because we do this kind of stuff
more or less weekly, but in shorter form over on Python Bytes.
The number of people that showed up to say hi to us at PyCon, we had a booth there.
That was an incredible turnout.
A lot of people, a lot of listeners coming up and talking with us.
It was great.
Yeah.
That was really, really awesome.
And we're going to do it this year, right?
In Cleveland.
Yeah, definitely.
Brian and I have a booth lined up along with some other folks.
And so be sure to stop by in 2018.
And get your PyCon ticket because it's going to sell out very soon.
Definitely.
I got mine.
I got mine as well.
All right, Brian.
Thank you so much for getting together and sharing this look back at 2017 with me.
Oh, yeah.
Thank you.
You bet.
Bye.
This has been another episode of Talk Python to Me.
Today's guest was Brian Okken.
And this episode has been brought to you by Linode and Rollbar.
Linode is bulletproof hosting for whatever you're building with Python.
Get your four months free at talkpython.fm/linode.
Just use the code python17.
Rollbar takes the pain out of errors.
They give you the context and insight you need to quickly locate and fix errors that
might have gone unnoticed until your users complain, of course.
As Talk Python to Me listeners, track a ridiculous number of errors for free at
rollbar.com slash talkpythontome.
Are you or a colleague trying to learn Python?
Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course Python Jumpstart by building 10 apps at talkpython.fm/course
to experience a more engaging way to learn Python.
And if you're looking for something a little more advanced, try my Write Pythonic code course
at talkpython.fm/pythonic.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and
direct RSS feed at /rss on talkpython.fm.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now, get out there and write some Python code.
I'll see you next time.
Thank you.