Welcome to a very special episode. This is the 100th episode of Talk Python to Me.
It's the perfect chance to take a moment and look at where we've come from and where we're going.
Not just with regard to this podcast, but for Python in general. And who better to do this with
than Python's inventor himself, Guido Van Rossum. In this episode, we discuss how Guido got into
programming, where Python came from and why, and Python's bright future with Python 3. This is
Talk Python to Me, episode 100, recorded January 18th, 2017.
Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem,
and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm at,
mkennedy. Keep up with the show and listen to past episodes at talkpython.fm. And follow the
show on Twitter via at Talk Python. This episode is brought to you by Rollbar and Hired. Thank them
both for supporting the show. Check them out at Rollbar and at Hired underscore HQ on Twitter and
tell them thank you. Hey, everyone. I just want to take a moment and reflect on this milestone of 100
episodes and say a big thank you to everyone out there who's listening. The reason this podcast is
successful, the reason I've kept doing it, is because so many of you tell me that you appreciate what I'm
doing, that you enjoy all the guests that I have on the show. So I want to say thank you because
without you, obviously, I would not have 100 episodes. I get to live basically my dream job.
I talk to all these brilliant people in the tech industry and I get to share it with you and get
all the great feedback that so many of you give me. So thanks. And we really have a special guest
guest this episode with Guido and a look at Python over the years, the past, present and future. And I
really hope you enjoy that. If you're out there thinking, hey, I really love the show and I'd like
to support it. There's a couple of things you can do that are really easy. One, if you give us a review
on iTunes, that actually makes a big difference how we rank within iTunes. So that's kind of like Google
rank for podcasts. That would be great. If you want to do more than that, one of the really great ways to
support me and what I'm doing would actually be to buy one of my classes or recommend my training
content to your employer or your team at work. If you're into that, check that out at training.talkpython.fm.
And there's even a little Patreon link if you just want to give a dollar or two a week. So thank you
everybody for making this possible. Thank you for helping me reach 100 episodes and been great to share
them with you. And I hope you've really enjoyed them yourself. All right. With all that said, here's Guido.
Guido, welcome to Talk Python. Glad to be here, Michael. I'm honored that you're coming on my
100th show to celebrate this special episode. And I know everyone in the community is really going to
appreciate this look at the history and the present and future of Python with you. Well, let's have it.
All right. Absolutely. So we're going to dig into a whole bunch of things about Python. But before we
get there, let's start with your story. How'd you get into programming in the first place?
Well, that was a long time ago. In high school, I did not know what a computer was. I believe I had
not even ever heard of the word. I was an electronics hobbyist, though. And I got started, I think,
around the age of 10, building very simple analog circuits, things like a little radio receiver that
made from a kit. And I gradually discovered simple digital electronics and integrated circuits were
becoming available to hobbyists like me, even with my very small amount of pocket money.
And that's where I was when I graduated from high school. Then I went to university to study
mathematics, the University of Amsterdam. And it was like a completely different world. They had a
mainframe in the basement, and there were programming classes. And the languages that I remember were
Algal 60 and Pascal. And I was basically instantly hooked, even though the first year I remember the
only way I could input programs to the computer was through punch cards.
Yeah. If you start programming, basically via hardware, and then you have this machine that
you can just feed anything, the whole world opens up, right? Even if it's punch cards, it's like it can
do anything I ask it to do almost, right?
I was so happy that I didn't have to sort of solder stuff together anymore.
That's right.
Because that was always my weak point.
Oh, it's such a difference. So you got started really in the early days, and when the term hacker
meant something entirely different, right?
I don't think I knew the word hacker. That was like decades later that people
told me I had been a hacker.
Oh, I didn't know. How interesting. Okay, excellent. And then you got into,
you started working on programming languages. How'd you go from playing with mainframes and
punch cards to working on things like ABC and stuff?
Well, I guess I developed an interest in learning different programming languages. That probably
started out when my first year there were two different languages being taught, I think,
Algol and Pascal. And then I hung out with a bunch of physics students and their favorite language was
Fortran. So right from the start, there was this discussion about Algol, no Fortran, no Algol,
Pascal. And somehow that interested me. I always stayed on the Algol and Pascal side, actually.
I was told that when I was in college that Fortran was the most important language I'd ever learned in
my career and I should just focus on that. And I was pleading to take some C and C++. I'm like,
can I do where the, nope, Fortran is where you got to start. You can do those as an elective afterwards.
Oh gosh, yeah. Well, it wasn't, in our university, it was more like a split between the math department and the physics department.
The natural sciences were using Fortran because they were sort of processing measurements and math people were more interested in sort of pure computer science.
Yeah, very interesting. Yeah.
The professors actually sort of crossed that bridge. So later, I actually, I encountered the author of ABC,
Lombard Meartens. I encountered him in my personal life, sort of an extracurricular activity where I was helping some volunteer group doing programming.
And he was also helping that same group. He was something higher in the organization.
And he realized that I was a good programmer and had interest in programming languages and their design and implementation and had some,
some interesting skills there. And when I was about to graduate, he just offered me a job.
And that was that.
That's fantastic, right? You're probably thinking, how am I going to find a job? What am I going to apply? What am I going to do? Oh, it just landed in my lab. How wonderful.
And then I learned really what the job of a language designer and implementer is. And I didn't get to design any part of ABC.
I got to discuss it with Lambert and other team members endlessly. But basically, the design was already complete when I joined the team.
And all they needed was someone who would implement it. But by fighting every aspect of the language that I didn't understand,
I prompted Lambert and others to explain what their reasoning process in the language design phase was.
And that helped me learn how to be a language designer.
Yeah, I'm sure it did, because they designed it more or less in the abstract, right?
And they said, all right, now it's time for a rubber to hit the road. And you'd actually make this thing work that we've specced out, right?
Yeah, much like that.
Yeah. And how much of that experience do you feel like made it possible for you to actually create Python?
Like, to me, thinking of I'm going to create a language, I'm going to create the CPython implementation and the standard library and all that.
It's very daunting and a very big challenge.
But do you feel like you kind of got a first round practice at it, doing this thing at ABC?
Absolutely. Without having been on the ABC team for four years, I would never have been able to do that.
I wouldn't have felt comfortable. I wouldn't have known how to.
I wouldn't have known enough about language implementations.
I mean, Python really is sort of the next version of ABC with all the things that were great about ABC retained and all the things I thought were not so successful in ABC removed.
And a very small number of my own ideas replacing them.
I see. So Python was sort of your, let me do this now knowing what I know, let me make a better version of something similar to this.
Correct. Yeah, that's exactly what I was really thinking.
And I sort of, I managed to present it to management in a slightly more objective fashion.
How do you pitch it?
Well, part of it was that management was not very closely involved in my day-to-day activities.
There was a certain amount of software that had to be written.
And however it was written was great, as long as we had sort of the running applications to prove it at the end.
I was more or less at liberty to invent a different strategy for sort of eventually building that software faster.
That's fantastic. And that's some of the origins of Python.
What happened to ABC? It's not around.
And Python is one of the most popular programming languages in the world.
Like, why did those things take different paths?
I've written about this in my old Python history blogs a few times.
The ABC project actually four years into it, at least for me four years into it, was canceled by upper management at CWI.
The reason being that there was no observable sort of user uptake.
There were very few people interested in the language.
There were even fewer people who are actually using it.
And the team just couldn't move that needle.
And part of that was that this was well before the Internet.
And there was a little bit of Usenet.
But it was difficult to distribute a language implementation and get people to use it.
Right. There's no GitHub.
There's not even SourceForge, right?
There's no web.
There's so many of the pieces that make these work.
It's worse than that, even.
There was no electronic way to distribute the source code at all when we got started.
I remember taking a long vacation to the United States with a nine-track computer tape in my luggage.
And taking that tape to two or three different places in the U.S.
where there were people interested in using ABC so that they could load that tape onto their computer and get the sources.
Because the amount of source code was larger than you could possibly send as an email.
And I think attachments hadn't been invented yet or were still limited in size.
Yeah, you have to, like, base 64 and code it.
Just put it as text or something, right?
I'm going to send you a thousand emails numbered 1, 2, 3, 4.
Yeah.
Even the first Python distribution had to suffer through some of that.
But by then, in 91, it was about 20 compressed maximum-sized messages to some Usenet source code group.
Yeah, Usenet was just starting to become popular.
And the internet was starting to be a thing.
But web browsers didn't really come out until, like, 93, 94.
So it was quite early days in 91.
Mm-hmm.
Talk Python to Me is partially supported by our training courses.
How does your team keep their Python skills sharp?
How do you make sure new hires get started fast and learn the Pythonic way?
If the answer is a series of boring videos that don't inspire,
or a subscription service you pay way too much for and use way too little, listen up.
At Talk Python Training, we have enterprise tiers for all of our courses.
Get just the one course you need for your team with full reporting and monitoring.
Or ditch that unused subscription for our course bundles,
which include all the courses and you pay about the same price as a subscription.
Once.
For details, visit training. talkpython.fm/business or just email sales at talkpython.fm.
Did you envision Python being open source from the beginning?
What was your thinking around that?
I would say yes.
ABC actually, in the sense that the concept even existed, was meant to be open source.
We were not interested in selling it.
We were just interested in promoting the language.
If the words open source had existed in the early 80s, we would have said ABC is open source.
As it was, I don't think we had even realized that there was some kind of need of a license.
Right.
That's amazing.
Just the nomenclature didn't even exist to describe.
Yeah.
You had to use words in a description.
This is a thing we're giving away.
You don't have to...
So on, right?
By the time I started on Python, there were a few models that were pretty solid.
I remember, I think the big model that I just copied almost literally was the MIT license
that went on X Windows.
Now, you're not supposed to call it X Windows.
It was X11.
But that was a big, essentially open source piece of free software that was very sort of
intentionally open source because the authors wanted to unify windowing software across different
hardware platforms.
There were all these different flavors.
How do you write apps that run on all of them and things like that?
That was a big problem, huh?
Exactly.
Okay.
So how do you think having Python open source has helped or maybe even hindered Python over
the years?
Oh, it's only helped.
I mean, if it hadn't been open source, people would not have been interested in picking it
up because it's one thing to download an application that's not free software, that's not open source
and use it to process some of your data.
But it's quite different to start writing your own software using a language that hasn't proven
itself yet.
Right.
So if maybe you're willing to pay for one of the top three most, the tooling and whatnot
for the top three most popular languages, because you know that's kind of where a lot of the
momentum is.
But in order to break into that space, being open and free really allowed you to wedge yourself
in there, right?
And it was also the model that some languages that I felt I was competing with most directly,
like Perl and Tickle TK, those were also open source.
I do not recall any names.
I do not recall any names, but I recall there were several similar level scripting languages
being designed and distributed at the time that were not open source and very intentionally
so, where there was someone who had a brilliant idea for making a better scripting language.
And for all intents and purposes, their language probably was better than what was available.
But their model for funding their work was sell copies of the interpreter.
And that just never worked.
Yeah.
In those early days, it was still not entirely clear what business models would work for the
developer community and what wouldn't.
People were really experimenting.
And I'm sure some things were lost because bad choices were made.
But I'm really glad Python is still going strong.
Did you ever imagine back in 1991 that open source and Python would be where they are today?
I, in general, suffer from a terrible lack of imagination and vision.
So at no point in Python's history have I ever adequately predicted where Python would be
five years from there.
So no, I had never thought that this would happen this particular way.
To me as well, it's just amazing.
Just even over the last five years, the way things have changed is so amazing.
GitHub has come into existence.
We're seeing companies that used to be fiercely proprietary become much more embracing of
open source.
I recently saw the CEO of the Linux Foundation or the head of the Linux Foundation standing
next to Satya Nadella at a Microsoft conference saying they love each other.
Yeah.
This is a different place than we were a while ago.
I think it's great for everyone, though.
I think it's a very, very positive path forward.
I'm very happy with how this all has turned out.
I'm hopeful that a lot of technology will continue to be open source.
Yeah, I think it will.
And I think it's great to see companies taking open source and building business models alongside
of it that are sustainable.
Companies like Continuum or Scraping Web or these guys that have a great popular open source
project and somehow they're adding value on top of it, but they're not abandoning open source.
Without open source, you have to do all the work yourself as the company that owns it.
And now, maybe if you're a large company like Google or IBM or Microsoft or Apple, you don't mind because you have tons of developers.
But for anyone who is smaller, the value of a community is so tremendous because you sort of, you will still be able to make money on a variety of consulting and support projects.
So that's how open source developers support themselves generally.
I'm actually an exception.
I'm just employed by some large software developer that uses a lot of Python.
That's been my personal model.
But yeah, companies like Continuum or Canonical.
Canonical.
Everything they produce is open source, but they make a lot of money through handholding of customers who don't want to sort of hire their own software developers.
That's a model that works for many types of open source software.
I think it's great to see people being successful in that model and experimenting with other ones as well.
Let's talk a little bit about language design and trade-offs.
Sure.
Python has been growing in popularity pretty dramatically over the last 5, 10 years.
It had been around for 15 years before.
And it seems like not only is it still relevant and popular, but that popularity and relevance is growing.
And I think part of that is expanding into different areas, like the adoption of Python in the data science space, I think, has brought many new people to the Python ecosystem where Python is their primary language now.
How do you trade off sort of serving these different environments or these different ecosystems?
Somebody wants in a language as a data scientist might be very different than what somebody wants as a web developer.
On the language design side, I don't usually take applications into account that much.
I've seen some language designs where people proudly announced, well, in our language, a URL is a standard data structure that is built into the compiler.
They say that as if that's a good thing.
And all it buys you is that you can leave out some string quotes.
And internally, it usually is still a string.
Right, exactly.
It's a gimmick.
And Python has always presented itself as a general purpose language.
And you can do many things in Python.
And I didn't design Python for web development, obviously, because the language is older than the concept of web development.
And ditto for data science.
And for web development, it turns out enough people from different backgrounds are interested in doing simple web development using Python that we ended up with a bunch of stuff in the standard library.
But still, often the most successful APIs for even for web development are actually third-party packages.
And what the standard library provides is more low-level than that.
Like the standard library has to provide things like sockets.
And in fact, one funny story is that I think in the first year that Python existed, before we made it open source, actually, I was teaching myself how sockets worked.
Because sockets were sort of a new thing in our environment at that point.
We had a bunch of Unix machines.
And I had never really known how the networking on those machines worked before.
And then some colleagues started writing little C programs that used sockets.
And it was so cool.
And the programs always crashed because they didn't do the right error checking.
And I wanted to know what those sockets were about.
And my way of teaching myself was, oh, I'll write a Python extension that wraps the socket API.
And so I just read the man pages and say, okay, well, there is like the socket call and the bind call and the listen call.
And I wrapped each of those in as low-level an extension as possible with proper error checking.
Because Python has always had this philosophy, if something goes wrong, you get an exception.
And then I started combining those calls in sort of random combinations and figuring out what errors I would get when and what forms of simple programs actually worked.
And that's how Python socket module came into existence.
And later, once the worldwide web started being promoted, I joined a variety of mailing lists around that.
And started writing my own little web servers and started writing my own little web servers and clients.
And eventually that turned into URL app.
But what people actually use for web stuff is third-party frameworks like Twisted or for web serving, pure web serving.
It's like Django or Flask.
For web clients, it's requests.
And so the standard library doesn't even contribute that much beyond the sort of the low-level sockets.
Sure.
So I totally agree that it's the packages that have absolutely made Python successful.
I mean, just looking on PyPI.org right now, we've got 96,000 plus packages.
And that's really a testament to how amazing the whole ecosystem and community is, right?
How do you decide when something should be in the standard library or something should be an external package?
And have these ever moved in or out?
Yeah, I have to admit that the standard library is still fairly uneven.
Because long ago, say 15 years ago, I had much less of a filter about what things would be good standard library modules.
I had very strong backwards compatibility requirements.
Like once a module is in the standard library, it can sort of grow, but it can't just change in an incompatible way.
Or at least that's a major project with deprecations and all that.
But I didn't have much of a bar for including stuff in the standard library.
And that started out with various hobby projects that I wrote myself and played with for two months in 1991 or so.
That were still in the Python 2 standard library around 2010.
Many of those things finally got ripped out for Python 3.
But of course, Python 2 is still there.
More recently, sort of, as Python matured and became more popular.
And from my perspective, it's just sort of been steady exponential growth.
So I can't tell you whether it was 5 or 10 or 20 years ago that Python suddenly started becoming popular.
So the current rules for inclusion in the standard library is a combination of something that is useful for multiple application areas.
A new API for web development will not make it into the standard library just because that's one area.
It has to be something that's useful for a wide variety of applications.
It doesn't necessarily have to be all applications.
But something like, well, sockets are obviously something that is useful across the board.
Right.
Many things in the standard library that really belong there also have to do with sort of the language itself.
Like introspection tools, partial functions, those kind of things are good standard library things.
The disk module.
For example, yeah.
Yeah.
Things that are bad for inclusion in the standard library is usually almost any piece of code that is under active development.
Because Python only issues a sort of feature release every 18 months or more.
And that's just a really slow pace and you can't even get people to upgrade quickly.
So if someone has a new idea for, let's take an example.
We have async.io, which is in the standard library, but it doesn't have a built-in web framework.
Well, why is there no async.io based web framework in the standard library?
Because the async.io based web framework that exists is under very active development and constantly changes and not always incompatible ways.
And so people would just be much worse off being stuck with whatever was the async.io based web framework around Python 3.5.
You would actually hinder it by putting it in Python, right?
Exactly.
You would force it to freeze its API so that could never change in a breaking way.
And it couldn't release more than every 18 months.
And probably some third-party package would come along, mimic that, but iterate faster and be better anyway.
Yeah.
Okay.
This portion of Talk Python to me has been brought to you by Rollbar.
One of the frustrating things about being a developer is dealing with errors.
Ah.
Relying on users to report errors, digging through log files trying to debug issues, or a million alerts just flooding your inbox and ruining your day.
With Rollbar's full-stack error monitoring, you'll get the context, insights, and control that you need to find and fix bugs faster.
It's easy to install.
You can start tracking production errors and deployments in eight minutes or even less.
Rollbar works with all the major languages and frameworks, including the Python ones, such as Django, Flask, Pyramid, as well as Ruby, JavaScript, Node, iOS, and Android.
You can integrate Rollbar into your existing workflow, send error alerts to Slack or HipChat, or even automatically create issues in Jira, Pivotal Tracker, and a whole bunch more.
Rollbar has put together a special offer for Talk Python to me listeners.
Visit rollbar.com slash Talk Python to me, sign up, and get the bootstrap plan free for 90 days.
That's 300,000 errors tracked all for free.
But hey, just between you and me, I really hope you don't encounter that many errors.
Loved by developers at awesome companies like Heroku, Twilio, Kayak, Instacart, Zendesk, Twitch, and more.
Give Rollbar a try today.
Go to rollbar.com slash Talk Python to me.
Has anything been brought in to Python that was originally external that you can think of?
I don't have specific examples in mind, but it's definitely happened.
That definitely occasionally happens.
It's usually more of a threat where we say, if someone comes with an idea and they say, this functionality should really be in the standard library.
And nowadays, we usually say, well, that looks more like an application.
And how do we know that it's actually useful?
Prove us that the design you have in mind works by first releasing it as a third-party package on PyPI and show us how popular that is.
And then there are also maintenance requirements.
We would require that a contributor commit to several years of keeping that code up to date and fixing bugs and stuff.
Sure.
And we've had things that actually were kicked out of the standard library because they were too much of a maintenance burden for the core development team.
I think the last time that happened might have been BSDDB or BerkeleyDB, which was a very large package.
It's currently much happier as a third-party package than it ever was in the standard library.
Yeah, I'm sure it can, like we said, grow much faster and so on.
There are databases in there.
For example, SQLite ships with Python.
SQLite is one of those things that is so popular and so versatile and so useful for so many different application domains that that was definitely the right decision to include that.
Also, SQLite itself is incredibly stable.
Yeah, that's one of the things.
It's not changing much these days.
It's very reliable.
That's great.
Yeah.
So speaking of contributors and having people commit to a certain amount of support over time if something's going to come into the standard library,
how do you ensure that the core development community invites and retains the best contributors?
You know, I have a lot of respect for the core developers, but how do you make sure that ecosystem is healthy and vibrant?
I don't think we're doing that very consciously.
We have a nominal mentorship program in place, but it's mostly used to get people thinking about contributing to open source projects in general.
I don't think that's where we get most of our new core developers.
In practice, new core developers almost always happen because somebody has an itch to scratch and they happen to be a really good programmer or at least sort of made a serious study of Python and start contributing.
And the people who review their code, and sometimes that's just one or two core devs who sort of mentor that one person, find that their contributions are of high value.
And then at some point, the mentor or one of the mentors proposes, hey, Python dev, core developers, what do you think of giving person X commit bit so that they can commit their own code after it's been reviewed?
Yeah.
There's often real discussion about the sort of potential new contributors maturity, not just in terms of their pure programming chops and how well they know CPython or Python or whatever their area of contribution is.
But also, do they have the right character?
Are they likely to sort of not commit something when the reviewer says that it's not ready?
Yeah.
I suspect one aspect of that is there's a lot of people that come into programming.
They see a shiny new thing, you know, something with Node.js or some other technology that seems like it's just going to take over the industry and then it doesn't necessarily.
So there's probably a level of maturity of experiencing tech evolution over the long term so that you can bring that to the language, right?
That would be too high a bar.
You can't say people can only start contributing once they've gone through one technology boom and bust cycle.
You need them to have a certain character that is okay with the pace of things and the thoroughness.
And you definitely want them to have a very good attention to detail.
Sure.
I guess you definitely want a different level of attention to detail and being meticulous when you're working on the internals of CPython rather than if you're working on some package that's used by a thousand people a month.
Right?
These have different requirements for that kind of stuff.
So you've been a champion of diversity in the whole Python ecosystem.
And I just wanted to point that out and say thank you because I think it's really making the Python space better in some tangible ways than other environments.
Just by way of a story, I went to a conference in London last year and I took my 16-year-old daughter.
And this was not a Python conference.
It was decidedly not.
But I was speaking on Python to these folks.
Say, hey, you should also learn Python.
This is cool.
And I brought my daughter.
We went to the speaker dinner.
And there was 28 or 29 speakers there.
And I think one or two women.
And my daughter looked at me after a while and goes, where are all the women?
And I said, you know, this is a sad part of the tech industry.
And when I go to Python conferences, I don't feel that way.
And, you know, I just think that's great.
I've personally always been a feminist, although never a radical one.
Encouraging women has been a pretty natural thing for me.
I think there was a specific series of events.
I'm not entirely sure when it was.
But I remember that there was some upset maybe around a decade ago at OSCOM.
Some people started pointing out that the open source community was, despite all its sort of pride in, well, we care about results, rough consensus and working code and all that.
There was not a very diverse community.
And the number of women that someone counted at OSCOM, I believe, was well below even the already low industry level.
Somehow that was a wake-up call for me.
There had always been a few active women in the Python community.
And I had never really counted how many there were or if it was always the same, two or three.
But because of that discussion at OSCOM that I did not participate or even witness in person, but that's through various blogs and peripheral vision came to my attention, I thought, hmm, what's the situation in Python community?
Actually, we're not doing great.
So I sort of made a mental note of, well, maybe we ought to think of how to increase the participation, the diversity in the Python community.
And I think that the main consequence of that was that when there were actually women who came to the Python community or were already part of the Python community and said, we would like to do something for women.
We would like this community to be more welcoming.
I thought, that's a great idea.
Let's do that.
Rather than responding in a way that some other tech communities have responded by feeling threatened or, oh, we don't have to do anything.
If you're a good coder, you're a good coder, and then you're welcome.
And if you're not a good coder, you're not welcome, regardless of whether you're a man or a woman.
That's actually a fair amount of bullshit in that attitude because those people don't realize how much bias there is.
So I've always been very open to the PSF and PyCon and various groups to try and reverse that bias by giving sort of diversity funding to PyCon attendees, for example.
I think I should call out specifically Jessica McKellar.
Yeah, I was going to bring her up.
Who has been a champion of this for a long time, has given some fabulous keynotes about this.
And all I had to do was be supportive there, I felt, which came naturally.
I remember in real world politics, since I was a voting age, I've always made a point of voting for a woman whenever I could.
I think the progress that's been made is really fabulous.
And you're right that Jessica deserves a lot of credit for that.
She's been fabulous at it.
We're in a much better place than we were before with regard to that.
Although I still think there's more work to do.
Absolutely.
I'm super happy to see the direction.
There's always more work to do.
And diversity is not just about women either.
Absolutely.
Let's remember that.
Yep.
Absolutely.
Yeah.
It was just the top of my mind because my daughter was so struck.
I took her to this tech conference thinking, oh, I'd really like to show her how cool the tech scene is and stuff.
And she did come away with that, but she also came away with a little bit of the feeling like maybe this is not for women, which made me a little bit.
I just felt bad about it, right?
Oh, that's terrible.
Yeah.
Yeah, it is.
It is.
All right.
So let's shift gears a little bit.
And, you know, one of the things that there's a lot of talk about right now and I think is turning a corner in a very positive way, but has definitely been a hot topic lately and over the last few years is Python 3 and the migration towards Python 3.
Ha, yes.
I love your attitude when I see you speak at conferences and do keynotes and there's a great 2.8 sign with like a slash through it just saying, look, we're moving forward.
We're not going back, you guys.
How do you feel we're doing on that?
That certainly has been a much harder, more arduous journey than anybody had really anticipated.
I think right now we're well over the hump.
The destination is in sight.
It's clearly better on the other side, but the mountain range was much bigger and colder, whatever, scarier than we had anticipated.
And I think that honestly, the mistake that all of us in the Python core and actually the whole Python community, the mistake we made was underestimating Python's popularity.
We sort of thought of Python as this sort of relatively small language with a relatively small number of dedicated followers who would all jump at the occasion of making their code more readable and converting to this new version of the language that was so clearly better in many ways.
And what we just underestimated how much code people had written in Python 2 and really old versions of Python 2 and that.
How much documentation there was that would have to be updated, not just the standard library and the core Python documentation and the reference manual, but all the hundreds of Python books and websites and answer sites and stack overflow questions and this and that.
And we just initially, when we first started talking about Python 3 and actually the first time that the term came up was around the year 2000.
It then took seven or eight years before we did anything about that.
But initially, from our perspective and certainly from my perspective, but that everyone who participated in Python 3 had sort of had the same experience.
Everybody was excited about this change.
We're going to fix the language.
There are all these Python words.
Andrew Cushling, who was an important early core developer, had very influential posts about Python words.
And he enumerated maybe a dozen of sort of key issues with the language that we were hoping we could somehow improve.
And one example was in 2000, we introduced Unicode.
And we introduced it in a way that would be incredibly backwards compatible.
And by the time, three or four years later, when that design had completely been settled, we started finding out that applications actually became more brittle because they didn't expect the Unicode to pop up in places where it happened.
And that was one of the things that we wanted to fix in Python 3.
But we had no idea that there were people with millions of lines of Python code that was all interrelated and written by people who were no longer with that team or that company or that project.
And what it would take.
I mean, had we known we could have taken a different tack, we could have made Python 3 somehow, we could have changed a few features in Python 3 to allow somehow Python 2 and Python 3 code to coexist in the same virtual machine.
Sure.
Somehow make it just seriously deprecated, but not gone.
Things like that.
Some of the features, right?
Yeah.
And had we really wanted that, we could have done that.
But we underestimated the difficulty it would be for the average Python user to convert their code because we thought, well, someone has a few scripts here and a few scripts there and maybe a thousand line library that they're using.
But in fact, all the numbers were 10 or 100 times larger.
Making it so much harder to switch.
Right.
Yeah.
And once we realized that this was not an ideal situation, there wasn't any chance of sort of backing out.
And there was also no chance of suddenly accelerating the conversion process.
So we've done the best we felt we could do, which included actually backporting many things to Python 2.7 in terms of libraries that were available and backporting even more things on PyPI.
Like you can use the enum 3.4 package, I think, in Python 2.7.
Then you can use enums in Python 2.
But we also wanted to give the community and the users a sort of a clear message about the future of Python.
And that's where the sort of no 2.8 banner came from.
You can always say, well, maybe we should have done this differently, but I'm not sure that it would have necessarily been different.
You could have said, well, we're going to leave our leave basically some 2.7 or Python 2 in Python 3 and just put new features and clean up around it.
But then you still have all this old code that still is written in the 2.7 style.
And the fact that people adopt the three features, you know, maybe they wouldn't.
And it would have been more of a hindrance rather than just going, all right, we're just going to have to make this break and just jump the gap to get there.
It's a complex problem.
There's no perfect solution.
And you also can't just stop evolving the language.
Absolutely.
I feel like Python 3 is going so fast and doing so much.
It's really positive.
Thank you.
Yeah, you're welcome.
A term that I started using, I got this from Matthias, I'm forgetting his last name, I'm sorry, from some of the Jupyter projects, is referring to Python 3 as Python and Python 2 as legacy Python.
And I think that's an interesting way to think about it.
You can try to change the language in the hope that people will subliminally be sort of influenced.
I don't know how effective that is.
I think the highway administration or whatever it's called have attempted to remove the word accident from our language and replace it with crash.
That's wishful thinking in my view.
And I think it's pretty consistently just say Python 2 and Python 3 whenever the distinction is important.
You know, to round out this part of the discussion, I definitely feel like we've crossed the boundary.
Absolutely.
It's gained momentum.
I see more and more projects that say either we're Python 3 first or Python 3 only.
And if people want to backport it, that's fine.
And this is the right path.
The really good news is that basically all important libraries work as well with Python 3 as they do with Python 2 or better.
Which means that the early problem with Python 3 adoption was that, well, if you had a thousand lines of pure Python code, that was very easy to port.
But if you had a thousand lines of Python 2 that depended on seven different packages, you had to wait for those packages to be ported.
That has taken a long time.
And that was, I think, one of the issues that we underestimated.
But that problem has now pretty much been solved completely.
NumPy, Django, Flask, everything you can think of works with Python 3.
Even DateUtil.
If you want to start writing application code from scratch, there's nothing to stop you from using Python 3.
Yeah, and that's really great.
I do feel like that was maybe the single biggest barrier.
Because even if people had the intention of switching to Python 3, but they depended upon these libraries and they just couldn't run, well, they're like, well, you're going to throw their hands up and say, well, I can't convert all these libraries.
I got work to do.
Yeah.
So let's do one more Python 3 topic here.
Let's touch on a little bit of your favorite Python 3 features.
The two that I've been, I think, most directly involved in were AsyncIO until two years ago.
Since then, AsyncIO has matured to the point where I don't have to be involved all that much directly.
Although I carefully encouraged and reviewed the development towards Async and the wait, which has been absolutely marvelous.
The other favorite of mine is function annotations and what we now have in Python 3.6, the variable annotations, PEP 5.26, and the whole type checking area.
That's sort of, I think those are my favorite features.
One other thing that I'm personally very happy with is the proper distinction between bytes and text in Python 3, as opposed to the messy way of dealing with that in Python 2.
Sure.
That absolutely solves that whole, it's Unicode, it's not Unicode.
What is it?
Yeah.
Unfortunately, that has also been a major porting barrier.
Yeah.
I've heard that several times, especially people working on web frameworks or things that touch the network.
The whole generation of Python 2 programmers grew up basically between 2000 and 2010 or so, who knew all the ins and outs of the compatibility and incompatibility between Unicode and bytes.
And sometimes it interoperates with Unicode and sometimes it doesn't.
But people got used to the ambiguity and actually exploited it in their APIs, which made their APIs difficult to port forward to Python 3.
This portion of Talk Python is brought to you by Hired.
Hired is the platform for top Python developer jobs.
Create your profile and instantly get access to 3,500 companies who will work to compete with you.
Take it from one of Hired's users who recently got a job and said,
I had my first offer on Thursday after going live on Monday and I ended up getting eight offers in total.
I've worked with recruiters in the past, but they've always been pretty hit and miss.
I tried LinkedIn, but I found Hired to be the best.
I really like knowing the salary up front.
Privacy was also a huge seller for me.
Sounds awesome, doesn't it?
Well, wait until you hear about the sign-in bonus.
Everyone who accepts a job from Hired gets $1,000 signing bonus.
And as Talk Python listeners, it gets way sweeter.
Use the link Hired.com slash Talk Python to me and Hired will double the signing bonus to $2,000.
Opportunity's knocking.
Visit Hired.com slash Talk Python to me and answer the door.
So does it surprise you that new things are still happening in the CPython internals?
Like dictionary got a major reworking in 3.6, things like that?
It doesn't really surprise me.
Dictionaries in particular are such an incredibly fundamental part of Python.
It's used everywhere internally and it's used everywhere in applications.
That it's totally par for the course that every, at least once every decade, maybe twice,
some major innovation happens in that area.
I remember when I first started Python, there was like, well, dictionaries were obviously something
I needed.
And in reaction to ABC, which used B trees, or at least some kind of, some form of balance
trees, which I thought was very tedious and I had worked on that code forever in ABC and
it was always buggy.
So I thought, well, for Python, I'll do a hash table.
That's a new thing.
And I just sort of looked it up in Knuth volume three.
What's the basic hashing algorithm?
How do you hash a string?
What, how do you implement a hash table?
Well, open hashing or linked lists.
And I made my decisions and implemented it.
And then I had to move on to other things, module objects, functions.
You'd write the whole thing, basically.
I had to build a whole language, long integers, exceptions.
Bytecode.
So the first innovation, I think, happened when someone with an actual mathematical schooling
in that area realized that I had copied an old algorithm from Knuth that was no longer
state of the art.
And there was something like, I think I, Knuth had discovered that it was good if you had
hash table sizes that were primes or something, relative primes.
And it turned out with changes in processor architecture, powers of two were suddenly better.
Interesting.
Yeah.
I've always heard primes as well.
So that's still in my mind.
Yeah.
That's no longer the state of the art.
I mean, the caches in CPUs and the whole sort of L1, L2, L3 memory architecture has affected
language implementations dramatically.
And I don't even know all the ins and outs of that area.
Unfortunately, we have other core developers who keep up with that and do that for me.
Yeah, that's great.
That's great.
I'm sure we could dig into all sorts of those things.
And there's so many other questions that I'd like to ask you.
I want to be respectful of your time and not go too long.
So let me ask you one final Python question.
What do you see coming in Python 3.7?
And what would you like to see there?
Well, I mentioned earlier that I'm not the greatest visionary.
Also, 3.6 only came out a month ago.
Unlike some other languages, there's no secret cabal of people who are already planning what
the language looks like five years from now.
I'm sure somewhere there's a C++ committee that's anxiously designing C++ 19 or 20 or
whatever the next version is going to be.
Well, that's not how we do things in Python.
And we just over time during the alpha stage of the next feature version, we tend to just
collect ideas and peps and proposals and often a real life experience with the previous major
version or the previous feature version, I should say, directs an evolution.
For example, we added async.io in Python 3.4, which led Yuri Selvanov to come up with the
idea of async, def and await and introduce that in 3.5.
That's great because that really cleans up that API.
That is an incredible improvement.
And that is not something we could ever have done if we hadn't had the clunky version of
async.io with yield from in Python 3.4.
Because the generators actually are...
I love the story of generators.
I can talk for hours about that because it has been such a rich source of language improvements
from the very early for loop to iterators and generators and coroutines and yield from and
then async await and sort of small improvements to that in 3.6 even.
That's been a very gratifying thing.
But I don't know what's going to happen there next.
Yeah, sure.
I expect that static typing, optional static typing, what we're doing with mypy, that's
my current project at Dropbox actually.
I expect that will also make strides forward.
But I don't know what those strides are going to be yet.
Yeah, so it really highlights that the language is a journey and a living thing, right?
You take a step and you look around.
You see the landscape that you just ventured into from a different vantage point and you
can pick a different destination or you can plan your next step or day trip or whatever
metaphor you want to use.
You can plan your next evolution based on where you are and what you see.
And sometimes that just comes like, whoa, we see all these people using this feature that
we just introduced two versions ago in a completely novel and interesting way.
And now we suddenly realize, oh, there is a better syntax or a better API that's sort of
waiting to come out.
And that's very exciting.
But I don't have sort of very specific plans for 3.7 yet, let alone beyond.
The only thing that's sort of possibly way beyond would be the gillectomy, the removal of the
gill.
But that is also not at all certain.
And it's a very complicated story.
You could talk about the gillectomy for quite a while.
And it's definitely an interesting area of focus.
So does this sort of, you take one step, you climb one mountain, you see a new horizon of
possibilities.
Is that one of the things that keeps you interested in working on Python and maintaining and overseeing
the language over time?
The excitement of actually climbing a mountain and then seeing a whole new valley that you
didn't know existed certainly motivates me.
Yeah, that's excellent.
So I guess we'll leave it there.
Let me ask you two questions before we get out of here.
First, if you're going to write some Python code, what editor do you open?
Emacs.
Well, actually, I don't open it.
It's already open.
It just stays open.
Well, my shell also mostly runs in Emacs.
Yeah.
Oh, fantastic.
Okay.
I know you don't like to play favorites, but if there's a notable PyPI package that you really
want to highlight that maybe people don't know about, but you'd like to say, hey, you guys
really should check this out because it's a cool example of something.
What comes to mind?
Actually, the one thing I want to highlight here is mypy, which is a static type checker
for Python that I didn't write, although I have now been contributing to it for well over
a year.
But it's still mostly Yuka Lettosolo's creation.
And we're using this at Dropbox.
And we have over 400,000 lines of annotated code in a code base that totals over 4 million
lines of code.
So we have a ways to go, but we also have more than 500 developers who are interacting
with this tool and who, by and large, are very happy with how adding type annotations makes
the code more understandable and more readable and sort of makes them more confident when they
want to undertake refactorings and things like that.
And mypy, the specific reason I want to highlight mypy as a PyPI package is that until a week
ago, if you tried pip install mypy, you would get some completely unrelated package that is
named mypy that has not been maintained for five years.
And we finally convinced the leadership of PyPI to give us that project name.
And now you can say pip install mypy.
Actually, you have to do pip 3 install mypy.
And it will actually do what you expect it to do.
Okay, well, that sounds really excellent.
It's great to see you guys using the type annotations and things like that at Dropbox.
Guido, we have a final chance for you to give a call to action to the community.
What would you like people to do?
What's on your mind?
Well, I really do want people to give mypy a try.
That static type checker is pretty amazing.
And I need to point out that there are still a lot of misunderstandings about what it can do
and what it cannot do.
Static types are completely orthogonal or almost completely orthogonal to unit testing, for example.
It doesn't mean you have to write fewer tests.
Occasionally, there are a few trivial tests you don't need to write because the static checker
catches the same issues better.
But by and large, a type checker just catches a very different category of errors than unit tests.
So they complement each other nicely.
And together, they give you more confidence.
One of the misunderstandings about mypy has also been that when we started PEP 484, that
the sort of the introduction of type checking was Python 3 only.
But we've actually amended that.
And for the past year, we've been using mypy successfully with Python 2 code base.
Okay.
So you can search PEP 484 for Python 2 support.
And it's completely there.
And mypy does as well on Python 2 as it does on Python 3.
Oh, that's really nice.
Yeah, the other misunderstanding about mypy or static typing in Python is that people have
predicted that they will turn your Python code into Java, which is obviously nonsense.
There's no interfaces all over the place.
Yeah.
So I think actually moving that type annotations to Python 2 is really important because it provides
some foundation for when you do want to upgrade to Python 3.
Correct.
My secret plan at Dropbox is actually that once we have a large enough fraction of the
code base annotated, we can start converting into Python 3 in a semi-automated fashion in
a way that would not be possible without those annotations.
We're not there yet, but that's my secret plan.
That's fantastic.
All right.
Well, I want to say thank you again for the conversation.
I really enjoyed talking with you and I'm sure everyone out there learned a lot.
So thanks so much for being on the show.
My pleasure.
Yeah.
Hope it's a good one.
It's going to be a great one.
Bye.
This has been another episode of Talk Python to Me.
Today's guest has been Guido Van Rossum and this episode has been sponsored by Rollbar
and Hired.
Thank you both for sponsoring the show.
I want to say thank you again to Guido as well.
It was really an honor to have him on the show.
I know he's a very busy guy and there are tons of demands on his time.
So thank you, Guido, for being on the show.
I know everyone really enjoyed hearing your perspective.
Rollbar takes the pain out of errors.
They give you the context and insight you need to quickly locate and fix errors that
might have gone unnoticed until your users complain, of course.
As Talk Python to Me listeners, track a ridiculous number of errors for free at
rollbar.com slash Talk Python to Me.
Hired wants to help you find your next big thing.
Visit Hired.com slash Talk Python to Me to get five or more offers with salary and equity
presented right up front and a special listener signing bonus of $2,000.
Are you or a colleague trying to learn Python?
Have you tried books and videos that just left you bored by covering topics point by point?
Well, check out my online course, Python Jumpstart, by building 10 apps at talkpython.fm/course
to experience a more engaging way to learn Python.
And if you're looking for something a little more advanced, try my WritePythonic code course
at talkpython.fm/Pythonic.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct
RSS feed at /rss on talkpython.fm.
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
Corey just recently started selling his tracks on iTunes, so I recommend you check it out at
Talk Python.fm slash music.
You can browse his tracks he has for sale on iTunes and listen to the full-length version
of the theme song.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Smix, let's get out of here.
Thanks for listening.
my voice, there's no norm that I can feel within, haven't been sleeping, I've been using lots of rest,
I'll pass the mic back to who rocked it best.
I'll see you next time.
Developers, developers, developers, developers, developers, developers, developers, developers, developers, developers.
Thank you.