Python Year in Review 2018 Edition
Just us on this episode of Talk Python To Me to count them down.
Episode Deep Dive
Guests Introduction and Background
Dan Bader is the creator of Real Python and a longtime advocate for Python best practices. He's deeply involved in writing Pythonic code, curating Python news, and educating developers through Real Python.
Brian Okken is best known for his work in testing with Python. He is the author of Python Testing with pytest and host of the "Test & Code" podcast. Brian regularly shares insights on how to leverage pytest and other testing practices to create reliable, maintainable Python applications.
What to Know If You're New to Python
Here are a few essentials touched on in the conversation for new learners, so you’ll get the most out of the topics in this episode:
- Python 3 is now the clear choice for all new projects due to its performance improvements and modern features.
- The Python community values readability and tooling that automates style checks (e.g., Black for code formatting).
- Package management is central to Python, and you’ll interact with PyPI for installing libraries and sharing your own code.
- Python’s applications in web development, data science, and embedded/IoT are expanding rapidly, 2018 was a key year in that growth.
Key Points and Takeaways
1) Guido van Rossum Steps Down as BDFL
This was the top story of the year. Guido van Rossum, Python’s Benevolent Dictator For Life, announced he would step away from deciding on major Python governance and enhancements. This shift opened questions about how Python’s community would move forward in making decisions. Although a major change, it lets Guido focus on more enjoyable aspects like mentoring and occasional coding rather than full-time leadership debates.
- Links and Tools:
2) Python 3.7 and Performance Gains
Python 3.7’s release highlighted improved speed and features, making Python 3.7 often faster in most benchmarks than Python 2.7. Developers loved that upgrading from Python 3.6 to 3.7 rarely required significant changes, and they got free performance boosts. This year reinforced that Python 3’s ecosystem had matured and surpassed Python 2 in both speed and capabilities.
3) Python’s Soaring Popularity
From an Economist article spotlighting Python's "meteoric rise" to surveys showing 40% of professional developers already using Python, 2018 confirmed it is a dominant language. Its user-friendly syntax and versatile library ecosystem attract beginners, data scientists, and experienced programmers alike. Python’s applicability across diverse sectors, web, data, AI, scientific computing, continues to fuel its unstoppable growth.
- Links and Tools:
4) Data Science vs. Web Development: A 50/50 Split
A notable finding from the PSF and JetBrains survey was that about half of Python users identify as data scientists and half as web developers. Many Python developers were surprised; they assumed web dev was still larger. This 50/50 split reinforced how integral Python had become for data-focused tasks, machine learning, and advanced analytics in addition to traditional web and backend work.
- Links and Tools:
5) The Rise of Black (Code Formatter)
“Any color you like, as long as it’s Black.” The introduction and rapid adoption of the Black code formatter brought standardized styling to Python in 2018. By auto-formatting Python code, teams cut down on “style” debates during code reviews, freeing them to focus on design and correctness. It also boosted the popularity of pre-commit hooks and showcased how standardized formatting can improve both morale and consistency.
- Links and Tools:
6) Rolling Out the New PyPI.org
After years of partial upgrades, the Python Package Index (PyPI) finally relaunched as pypi.org. The fresh design looked more modern, improved discoverability, and adopted stable hosting infrastructure. Python users celebrated a more professional first impression for newcomers and better community engagement around distributing and consuming Python packages.
- Links and Tools:
7) MicroPython and CircuitPython (Embedded Python)
Python’s traction in embedded systems grew dramatically. Projects like MicroPython and CircuitPython enabled hobbyists and professionals to program hardware using Python instead of C. Adafruit’s CircuitPython efforts, plus boards like BBC micro:bit, demonstrated the fun, educational potential of Python for IoT and hardware hacking.
- Links and Tools:
8) Python 2’s Final Countdown
Guido officially stated Python 2.7 would reach end-of-life on January 1, 2020, no more security patches or fixes. While this plan was known for years, 2018’s reaffirmation prompted a sense of urgency. Popular libraries and frameworks began dropping Python 2 support, urging developers to upgrade. As the clock ticked down, the phrase “Legacy Python” took hold.
- Links and Tools:
9) End of Innocence on PyPI (Malicious Packages)
In 2018, PyPI faced security challenges with typo-squatting and malicious uploads. These attacks inserted harmful code or data-collecting scripts when unsuspecting users installed similarly named packages. While not catastrophic, it signaled PyPI’s transition from a purely community-trusted site to one requiring vigilance, security measures, and user education on verifying package authenticity.
- Links and Tools:
10) Versioning Shifts: Calendar vs. Semantic
A smaller but thought-provoking trend: some libraries switched from traditional 0.x or semver to calendar-based versioning (e.g., 2018.12.0). Sanic and pip are examples. Advocates argue it makes release dates obvious, encourages consistent update cycles, and emphasizes continuous improvement. Others remain partial to semantic versioning’s clarity about backward compatibility.
- Links and Tools:
Interesting Quotes and Stories
"What a time to be alive." -- Dan Bader, remarking on Python’s rise and how easy it is to get started yet still build production apps.
"Faster never really beats everything... it's faster development time too." -- Brian Okken, reflecting on Python 3.7 vs. older Python versions.
"You either control the robots or you get replaced by the robots." -- Michael Kennedy, joking about the surge in Python popularity for automation and AI.
"A large segment of developers, maybe half, are ‘dark matter developers’ who don’t blog or attend conferences." -- Dan Bader, discussing surprising usage stats in data science.
"Python 2 is done. I’m not even going to fix security problems after 2020." -- Guido van Rossum, clarifying the end of life date.
Key Definitions and Terms
- BDFL (Benevolent Dictator For Life): A title given to Guido van Rossum for his central decision-making role in Python’s development.
- PyPI: The Python Package Index is where most Python libraries and frameworks are hosted and downloaded.
- Typo-squatting: A malicious practice of creating packages with misspelled or similar names to popular ones to trick users.
- Black: An uncompromising code formatter for Python that enforces a single, standardized style.
- MicroPython/CircuitPython: Minimal Python runtimes designed for microcontrollers and embedded systems.
Learning Resources
Here are a few curated resources to continue your Python journey:
- Python for Absolute Beginners: Ideal if you're just starting out and want comprehensive coverage of Python fundamentals.
- Getting started with pytest: Great for leveling up your testing skills, something Brian Okken emphasized in the episode.
- Python 3, an Illustrated Tour: Focuses on modern Python features (especially relevant to the discussion about Python 3.7 and beyond).
- Write Pythonic Code Like a Seasoned Developer: If Black’s formatting style intrigued you, this course helps further refine idiomatic Python coding styles.
Overall Takeaway
Reflecting on 2018 in Python reveals a community accelerating toward Python 3, delighting in new tools like Black, modernizing essential infrastructure like PyPI.org, and tackling security challenges with a more proactive approach. While Guido’s stepping back marks the end of an era, the future is bright. The language’s popularity, spanning web development, data science, embedded, and more, promises continued innovation and collaboration among Pythonistas worldwide.
Links from the show
Brian Okken: @brianokken
Dan Bader: @dbader_org
#10: Python 3.7
Cool New Features in Python 3.7: realpython.com/python37-new-features
#9: Changes in versioning patterns
ZeroVer: 0-based Versioning: 0ver.org
Calendar Versioning: calver.org
Semantic Versioning 2.0.0: semver.org
#8: Python is becoming the world’s most popular coding language
Economist article: economist.com
#7: 2018 was the year data science Pythonistas == web dev Pythonistas
Python Developers Survey Results: jetbrains.com
Covered in depth on Talk Python 176: https://talkpython.fm/176
#6: Black
Project: pypi.org/project/black
Soundgarden : “Black Hole Sun”: youtube.com
#5: New PyPI launched!
Python Package Index: pypi.org
#4: Rise of Python in the embedded world
Covered at Python Bytes: pythonbytes.fm/92
#3: Legacy Python's days are fading?
Python 2.7 -- bugfix or security before EOL?: mail.python.org
Python 2 death clock: pythonclock.org
#2: It's the end of innocence for PyPi
welve malicious Python libraries found and removed from PyPI: zdnet.com
#1: Guido stepped down as BDFL
[python-committers] Transfer of power: mail.python.org
Proposals for new governance structure: discuss.python.org
Episode transcripts: talkpython.fm
--- Stay in touch with us ---
Subscribe to Talk Python on YouTube: youtube.com
Talk Python on Bluesky: @talkpython.fm at bsky.app
Talk Python on Mastodon: talkpython
Michael on Bluesky: @mkennedy.codes at bsky.app
Michael on Mastodon: mkennedy
Episode Transcript
Collapse transcript
00:00 It's been a fantastic year for Python. Literally, every year is better than the last with so much
00:05 growth and excitement happening in the Python space. That's why I've asked two of my knowledgeable
00:10 Python friends, Dan Bader and Brian Okken, to help pick the top 10 stories from the Python community
00:15 for 2018. Join us on this episode of Talk Python to Me to Count Them Down. This episode was recorded
00:21 on November 27, 2018.
00:23 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the
00:41 ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter,
00:46 where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm
00:50 and follow the show on Twitter via at Talk Python.
00:53 Brian, Dan, both of you, welcome back to Talk Python.
00:56 Thank you.
00:57 Hey, thanks. Great to be back again.
00:58 Yeah, it's great to have you back as always. It's going to be so much fun.
01:01 And it is time to look back on 2018. Can you believe it's the end of 2018? I just remembered
01:08 how to write 2018 and not 2017 on all the stuff I have to sign.
01:11 Yeah, it's gone by fast.
01:14 It definitely has. It definitely has. So it's good to have you back. We're going to go through
01:20 what the three of us have decided are some of the top stories in the Python space for 2018. And I just
01:28 want to quickly throw out there for folks listening. This mostly does not include data science stories.
01:33 There's a little bit here and there, but it's mostly pure Python. And the reason is, I'm also doing a
01:38 show on data science year in review. So save those big stories for that one. Don't want to overlap.
01:43 All right. Well, we're, we're going to start and do a top 10 countdown. And we're going to go from
01:51 really interesting to the absolute big news at the end. And you can see what story we have picked to be
01:58 the number one story of 2018 for the Python space. Let's do a little warm up here. And Dan,
02:04 I'll let you kick this off. What was our number 10 item?
02:07 Sounds good. So our number 10 is the Python 3.7 release and more specifically the performance
02:13 optimizations in there. And I know you had a really good show on the podcast here with Anthony Shaw,
02:19 where you guys talked about, you know, some of the changes in 3.7. And I just thought that 3.7
02:26 should be on this list because it was a really, really good release. You know, there's a lot of like
02:29 free performance improvements. It's always nice to see that. And it's just been awesome to see Python 3
02:35 and see Python 3 specifically gets so much love and that it's under active development and it's
02:39 making this forward progress. And I think it's also the release, or I think it was either 3.6 or 3.7,
02:46 where Python 3 is now faster in all of the benchmarks than legacy Python, except for startup time. So I
02:54 thought, you know, that's kind of a nice milestone for Python 3.
02:57 I think that's a huge milestone for Python 3. I mean, this is sort of the year that the tide has turned
03:02 turned for Python 3 in a big way. I think it was starting to turn before, but really this year,
03:07 I think that's, that's one of the themes. And this is a major aspect of it, right? That Python 3 is now
03:14 faster. There were always these couple of reasons people say, I'm holding out to Python 2. You guys,
03:19 with your fancy Python 3 and your new keywords can just go play because my code is faster and faster
03:26 beats everything. Right? Well, maybe not. Yeah. Well, I mean, faster never really beats everything.
03:32 It's, it's faster development time too. And when you have a better data structures, you can sometimes
03:36 develop faster. However, you're right that a lot of people were holding out because of the, some of the
03:42 speed and, you know, and for certain, some applications that makes sense. But yeah, I think 3.7 is definitely
03:49 faster. And, and the other thing is a lot of people had already, some people for some of their projects
03:54 had already switched to Python 3.6. And that was probably from 2.7 to 3.6 was not an easy switch.
04:02 Having a switch from 3.6 to 3.7 be like, oh, I don't have to do anything. It's just better now.
04:08 That was a nice thing to have. Yeah. That's super nice. And I think that's a really great point you
04:13 raised, Brian, that, you know, careful what you're optimizing for, right? Like if it was really all about
04:18 speed, wouldn't we just be writing assembly code still, you know? Yeah.
04:23 I bet you, I bet somebody good at writing assembly could crush the performance of Python or a lot of
04:30 C++, but you know, they would release an app, a small app every 10 years and it would be buggy,
04:36 right? Like there's just, it's not the most important thing. That's, that's for sure.
04:41 One of my, my favorite examples there is, what's this, it's, what's this game called?
04:45 it's rollercoaster tycoon, which was written by one person in wind 32 x86 assembly. It's like a hundred
04:53 percent assembly code. And it's actually an amazing, amazing game. And it's, it's just,
05:00 it just blows my mind. Like everything is, was written in assembly language.
05:04 Wow. And it probably runs like totally smooth on like a 286 or something, right?
05:09 Yeah. No, I think you needed a Pentium or something, but, now somebody's worked on like a Python
05:17 going to need, I don't know, multi-core CPUs and whatnot to run at the same performance.
05:23 Yeah. Who knows? But you know, another, another interesting thing that's coming out
05:27 is more and more talk about actually compiling Python, which I think, you know, didn't make
05:32 the list, but it's definitely something that, that came up like my Pisces, for example, and Cython and
05:37 whatnot. Yeah. Yeah. So if we're going to release a new library or an awesome game, like roll rollercoaster
05:45 tycoon or something like that, and we were going to do it in open source, probably the main full release
05:51 after it's like totally stable and spent out for a year, it'd be like a zero dot. Oh one or,
05:57 or O three, maybe an O three, like a zero dot 12. I don't know, something really, really small,
06:02 right? Like that. Right. And that's been a theme that's gone through a lot of, a lot of open source.
06:09 So I think I'd like to highlight a little bit of shift in just the versioning that seems to be
06:14 happening around projects. I want to kick off this part of the conversation with talking about
06:19 Mahmoud Hashemi's project called zero over Brian, do you remember covering that on Python bites?
06:25 Yeah. He released it like as an April fool's thing, but it actually highlighted a lot of,
06:30 a lot of things about different projects that were zero over after like, you know, years.
06:36 Yeah. So the idea is kind of like I was describing at the beginning, like so many projects just never
06:41 even reach version one dot O. And in some sense, it doesn't matter, right? Is the version number
06:47 smaller or bigger? That tells you whether you have an older or new one, right? So that's fine. But there's a lot
06:53 of people, especially coming from the enterprise space or from the commercial software space that
06:58 sees things like zero dot 20 and go, Oh, that thing's like a super alpha. We're going to stay
07:03 away from that until that reaches one O. And I think it has a tangible effect on turning off a certain
07:09 category of folks, especially people who are maybe unfamiliar with open source and things like that.
07:15 Yeah, I think definitely. We, I mean, a lot of people do a commercial software development
07:20 and like I do, and, the zero, zero versions always, or almost always mean
07:28 the API is in flux. You can't trust it. You shouldn't program against it because it can,
07:34 the developers can change their mind at any time. And it isn't until the 1.0 version where we have a,
07:39 a stable, a stable API that people can count on. Yeah. And so I think a lot of people perceive it
07:45 that way, right? Yeah.
07:46 And it's interesting to consider the, the version number as part of the user interface,
07:51 you know, for, for an open source project, which makes absolute sense because of those,
07:56 those effects where people are like, yeah, you know, is this, is this really ready to go here?
07:59 Is this an early release or whatever? I mean, I'm wondering if it's really going to have a big
08:04 impact, you know, if people are moving to something like, like dating their releases, like just
08:08 calling them 2018 dot something, something, and you sort of get the November release, but it's,
08:13 it's an interesting experiment for sure. I mean, it, it definitely looks a little bit nicer than,
08:17 you know, having, I don't know, like somebody put pandas, pandas here, which had like 83 releases
08:22 and it's been running or it's been active for eight years and it's sitting at 0.24.
08:27 0.0.dev.
08:29 0.dev.
08:29 0.0.
08:30 Nothing wrong with that, but I can see how it turns some people off. Yeah.
08:36 I don't know how dev zero means production, but you know, to me that just says like, dude,
08:41 stay away from this. Don't ship this.
08:43 That's really scary.
08:43 Yeah. I think the, you know, maybe the grandfather of zero for it might be putty,
08:49 you know, that little, SSH tool you get for windows cause it doesn't come built in.
08:53 that'll trade app. it's 20 years old and it's 0.7.
08:57 One concern that I have with moving to the date based or like year based version numbers is how,
09:04 like if you have the semantic versions, you can do things like, okay, I want,
09:08 give me all the minor patches for this package and all the security updates, but I don't want to jump
09:13 to the next major version, which maybe has a breaking changes in it. And I know that's not a perfect system.
09:18 in general, you want to pin your packages, but it's kind of nice to have the ability to do that.
09:22 And with, you know, who knows what's going to happen in like July, 2018 or something 19.
09:27 Yeah, absolutely. So if you look at some of the tools that I know you use and I use, for example,
09:32 I use pyup.io and use something slightly different, but they let you go to your requirements file or
09:37 your pip file and say, I want you to keep this on the latest pin version possible with this constraint
09:44 where you could say like 0.1.star or something like that. Right.
09:47 Right. Yeah. But like, how would that work with, you know, date based.
09:52 Exactly. So let's dig into that a little bit. So what are the two options? I guess the two options
09:57 are just like to start shipping like full versions, like Django is now Django two and the Django two,
10:03 the main change from one to two is we're dropping Python to support, which is kind of ironic. Maybe it
10:09 should be Django three, but whatever, you know, it indicates a major change, like you were saying
10:13 there. Right. So one of the projects that's switching to calendar based version is Sanic,
10:20 right? The async web framework. And what they're doing is they're trying to ship a version,
10:25 a major release every quarter. So they might be, you know, 2018.07.0, then .1.2 to,
10:35 for like incremental changes, but they're also doing LTS releases around their calendar version.
10:41 So every December is an LTS version. So 2018.12.0 and so on would be supported for a year and would
10:50 be the LTS equivalent, the long-term support equivalent. If you don't want to switch to
10:55 something beyond that, that's how that group is handling it. Interesting. Yeah. I kind of like that.
11:00 Then you can glance at a version and, I mean, if everybody sort of supports something like that,
11:05 you can glance at all of your versions and go, Oh, everything's within a year or so. So it's,
11:10 it's probably fine. Exactly. Like Brian, you know a lot about testing. If I told you I'm using
11:14 web test.2.0.32, is that new? Is it super old? Is that a beta version? Like what is it? Like,
11:21 what is that? Right. You don't know. Yeah. But if I told you I was using web test 2019.1.0,
11:27 like, Oh, you must be like testing the new one that's coming or, you know, something I,
11:31 it's just communicates it. So Sanic is doing this. pip is doing this. and then you threw some
11:36 others in there, Brian, what are these? I'm not familiar with them. There's some descriptions of
11:39 calendar version and semantic version, calver.org and semver.org. I love that those are domains.
11:44 I think Mahmoud did at least one of them, maybe both. He has this knack for creating these little
11:51 sites and these little projects that just pop out of nowhere. But the, the, the, one of the things I want to
11:57 point out is I was talking with, somebody about calendar versioning and that versus semantic
12:03 versioning. And one of the things is that the major number, whether it's two or three or it's a date
12:08 based often means, you're not going to, you're not going to remove features. So you promise not to
12:16 break customer code within a major version. The date base kind of says we're at least going to stay.
12:22 It's safe for a year. Yeah. Whereas a semantic versioning, you know, a development team can go
12:27 through major versions as fast as they want to, if they feel like breaking things fast. That's true.
12:31 That's true. It's just when you get the feeling to switch the major version from two to three or
12:36 three to four, you just do it, right? Yeah. But it has impact. So, yeah, I think there's some
12:42 interest there. So I think another thing that was really interesting about the calendar versioning that
12:46 I learned talking to Adam Hopkins from the Sanic project was that there's this sense of you need
12:53 to keep shipping, right? Like if you have a calendar version and it's from last year and that's the
12:58 latest version that sends a message in and of itself that this project is, is not getting a lot of
13:04 activity. Whereas without it, you know, you kind of got to be more deeply familiar with the project to
13:09 understand its rate of improvement. Yeah. I think even it'd be cool to have people,
13:14 even if there's not really any changes needed just to release it and say, Hey, we tested it. It's,
13:18 this has all been tested on the new versions and you're good to go. We updated like the readme file
13:24 or something. Right. Well, there's almost always like some minor little thing that can be,
13:29 can be improved and shipped. Right. Another story of 2018 and maybe slightly before,
13:36 but really was coming in strong in 2018 is Python's kind of popular.
13:40 Yeah. So, Python's very popular. It's taking over the world. There was a neat economist article
13:48 that came out and that, I guess that's just sort of related to the, they're highlighting a trend.
13:54 And the article was, Python has brought computer programming programming to a vast new audience.
13:59 But some of the things they point out within there is that, right now, according to a stack
14:06 overflow survey, 40% of professional developers use Python and another 25% wish they did, which is
14:13 interesting. It just, just go do it, man. Also the, they interviewed, Codecademy,
14:19 which is an online training site and their biggest increase in demand, for 2018 is with Python.
14:25 Yeah. That doesn't surprise me. I mean, you think about a very fast growing technology that also has
14:31 a lot of career possibilities. Definitely people are going to be wanting to learn that, right? So if
14:37 you go to somewhere like Codecademy or any other online trading place, like there's going to be,
14:41 unless they're doing something wrong, there should be a lot of demand for Python.
14:44 They noted that since 2014, at least in a, from a survey in 2014, Python was the most popular
14:51 language, intro language at American universities, but that was amongst, people, studying
14:57 sciences or computer science or something engineering. But what we're seeing now in 2018 is the growing
15:03 movement of people that are not traditionally think of themselves as programmers, primarily learning
15:09 Python. And, we'll see that in data sciences, embedded development is now moving towards
15:15 Python and in, in, in a lot of maker senses. And then also the article also pointed out that more
15:22 and more people are just sort of worried that all jobs will be taken over by automation. So they want
15:27 to be part of that. So in order to, to learn some automation on your own, Python is a good way to,
15:32 to be part of that.
15:32 You either control the robots or you get replaced by the robots. Take your pick, right?
15:37 Yeah. Also to throw in there, I saw this on, Nina Sakarenko's Twitter feed recently. so Python
15:45 is becoming, or has been nominated as the official programming language for education in France. So I
15:50 think it's going to be part of the high school curriculum there, which is, that's super cool.
15:55 I think it's, we're seeing the snowball effect, you know, where, where people, it's really becoming
15:59 the dominating language for beginners and educational language. And I think that's super cool.
16:03 That is so cool. And when I saw that on Nina's post as well, I'm like, Oh, that is amazing. And you know,
16:09 I've been thinking a lot about this. Like why is, is Python both popular as a teaching language and
16:14 popular as a professional programming language? Like C++ is popular as a pro professional
16:20 programming language, but they don't suggest that kids start with it. Right. Things like this.
16:25 I think there's this special balance that when the language was created, it has this way to get
16:30 started easily and to pull in just the aspects that you want. Right. So you can start out with not
16:36 even functions and then like, okay, I need some structure. I'm going to put some functions in here.
16:40 Oh, let's start working with classes and things like generators, but none of that, you don't have to
16:47 understand or use or care about that until you're ready. So like Python has a special ability to start
16:52 easy and small, but expand into professional type of apps. Whereas a lot of them are either,
16:56 you're always a beginner type of language, or you're always like, yeah, that's a void star star kids.
17:02 Let's talk about that.
17:03 Totally. Yeah. Actually, Mike, I think you, you called it a full spectrum language way back when,
17:10 when we did an interview in my, my blog with you. And that just really stuck with me because I think
17:16 that's, that's so true. You know, it's, it's not just, it's not like you're learning, you know,
17:19 like the little like turtle, like logo or something like that, where it's like, you learn the basics
17:23 of programming, then you're sort of stuck and you have to move on and learn something else. But no,
17:27 you can, you know, all the skills you learn working with Python, they kind of scale and,
17:30 and grow with you. And that's, that's just a beautiful thing.
17:33 It's special. There's not many other languages like that.
17:35 What a time to be alive.
17:36 It really is an amazing time to be alive.
17:42 This portion of Talk Python to Me is brought to you by Cloudbolt.
17:44 Friends don't let friends violate PEP 8, nor do they let them spend their days in an unfulfilling
17:50 work environment. Good news. Your friends at Cloudbolt want your help developing their state-of-the-art
17:54 cloud management software. Built with Django and ranked as the number one product of its kind,
17:59 Cloudbolt's looking for talented engineers of all kinds. Located in beautiful Portland, Oregon,
18:04 Cloudbolt is an hour from the Pacific Ocean and Mount Hood. You're not in Portland? Not a problem.
18:09 Cloudbolt offers a relocation stipend to the Pacific Northwest and is also hiring solution
18:14 engineers everywhere. Whether you're interested in containers, hypervisors, or just writing clean
18:19 performant Python code, Cloudbolt would love to hear from you. Visit talkpython.fm slash
18:24 Cloudbolt for more information. Brian, you, you touched on that a lot of the growth of Python was
18:31 centered around people who are using Python as a tool, maybe not as their identity, right? Like
18:37 Python developers are like, oh, I'm a Python developer, but other people are just using
18:41 Python, maybe for data science or things like that. And so number, number seven in our list
18:48 is that I'm going to declare 2018 as the year that the number of data scientists have come to equal the
18:56 number of web developers in Python for professional Python developers or users, let's say.
19:02 Okay, you're going to have to back that up a little bit.
19:03 All right. So there was a really cool survey done by the PSF in conjunction with JetBrains. So JetBrains
19:11 did the infrastructure for sort of analyzing it and hosting it and so on, but it was promoted just
19:17 through the PSF and other public sources. So it's not like tied to JetBrains, but they did this really
19:23 great survey. And it's called the Python developer survey 2017, which might sound like I'm wrong in my year
19:29 and sort of I am, but this is at the very end. This is like November, December or something like
19:34 that of 2017. So it's pretty much there. And they did a bunch of interesting, they asked a bunch of
19:40 interesting questions and they found out a lot of things like, for example, four out of five Python
19:45 developers use Python as their main language. So when you see people are using Python, mostly they're using
19:51 that as their main language. It's not like, well, I write and see, but I also have this script that like
19:56 compiles it in Python or some weird thing like that, right? So the majority of people using Python
20:00 use it as their main language. And that's, that's pretty interesting. But the thing that I want to
20:06 point out is they said for, you know, what do you use Python for? And if you pick where Python is main
20:13 versus where Python is secondary, you get slightly different answers. But generally it says for data
20:19 analysis and data science, 50% of the people use it for that and 49% of the people use it for web development.
20:27 And then there's a whole bunch of other areas that like, you know, it's a multiple answer type of thing.
20:32 So they've also checked those off. But in this survey, basically they said 50% of the people said data
20:39 analysis, 49 web development. Does that surprise you guys?
20:42 Not really.
20:43 I think it surprised me. I know there's a lot of data scientists. I feel like on a web development team,
20:51 there's more participants than on a data analysis team. I feel like a data analysis is often done by
20:57 a smaller group of people than like the 30 people that maintain the website or, you know, whatever,
21:01 right? Like that was my conception. I don't know if that's accurate, but apparently no. Either there's
21:07 even more data scientists than I guess, or the there's larger teams of data scientists and smaller
21:13 teams of web developers. So this is all interesting. Another thing that I, I thought was pretty
21:18 interesting around this is they asked people to guess that ratio in this survey and they gave them
21:26 like a five choices, four choices, something like that. And they said, what do you think the ratio is?
21:30 Five times the number of web developers to every data scientist, one to one, you know, maybe worse.
21:36 I can't like 10 to one. I can't remember. But when they asked that question to the group,
21:41 half of whom were data scientists and half of whom were web developers based on the original question,
21:46 the responses they got was only one, only 10% of the people got it right. 20%. And,
21:53 most thought that it was five to one, five times web developers to one data scientist. So I think the
22:02 general perception is there's more stuff happening on the web around Python than there are data
22:06 scientists. But I think also what the most surprising thing to me when I saw that result was half of the
22:12 people that answered that question were data scientists, you know? And so to me, my interpretation
22:18 is that means that a lot of the data scientists feel like they're not as big of a representative group of
22:24 the community as they are. They feel like, Oh, we're like 20% of the community. But in fact,
22:28 they're like half. And I don't know why that is, but I think it's interesting.
22:32 It is interesting. Well, there's definitely like a, if you look at the hype around different packages,
22:37 for instance, there's more packages that are web related than there are data science related.
22:44 Mostly I think because a lot of the data science tools have been bundled into these large groups of
22:50 tools, even though. Yeah. Yeah. So they have like a common set of things that they were like,
22:56 data scientists like to all use sort of this, you know, a handful of different tool chains,
23:01 whereas web developers like to go off and create their own asynchronous library.
23:05 Yeah. Everyone's got their own library to match their new JavaScript library.
23:09 Yeah. I wonder if the Anaconda distribution and things like that also play into that feeling,
23:14 right? You just install it and you just have the tools, right? You don't necessarily go and hunt
23:18 for every little library as much.
23:20 This sort of reminds me of who coined that term? Scott Hanselman, I think the concept of the dark
23:26 matter developer.
23:27 Yes. I was thinking that too.
23:28 Yeah. Yeah. I think, I think you actually told me about it. So I guess the idea is that there's a
23:32 large segment of the programming population or, you know, people who program for even for a living
23:38 sometimes that are just not visible in the community. They, you know, get their work done and it's just
23:45 not, maybe they don't identify as a programmer or it's not just something, it's just not something
23:49 that they do publicly or have like side projects and stuff like that. Nothing wrong with that.
23:54 Yeah. And most people, they don't go to conferences or go on social media and go crazy about their job.
24:00 Like, like if I was a barista, nothing wrong with that, but like, I probably wouldn't go to barista
24:05 conferences and go to barista meetups and like, you know, talk about being a barista on social media
24:10 very much, right? Like it just, it just, yeah, you actually write some awesome stories if you did that.
24:15 Make coffee for me.
24:17 Hey, anyone out there listening, go for it. That's all you.
24:22 I'm going to get that.
24:23 I would listen to that. I think there's a lot, a huge number of companies that have
24:27 one data scientist or data analyst in their company. And so they feel outnumbered because
24:34 they're the only one in their company or there's like the, the ratio is very much skewed within
24:39 their company, but they're kind of all over the place.
24:41 Yeah. Yeah. I totally agree. And I do think there's a little bit of this dark matter.
24:45 stuff possibly happening here. Maybe you're not allowed to talk about your analysis that
24:49 you're doing. So you just don't, maybe you've backed into becoming this data scientist business
24:54 sort of analyst type person. You started out as like a trader and then you, you kind of were the
25:00 person that had enough code to like figure out how to do the overall reporting and analysis and whatnot.
25:05 And you still don't think of yourself as a developer, but you effectively are. Right. So I don't know.
25:10 I think there's a lot of interesting angles here.
25:12 Yeah. Yeah. Also I was just thinking like, you know, a lot more secrecy maybe in the, in those
25:16 areas. Like if you're working on some state of the art, like machine learning thing, I don't know,
25:20 self-driving cars or whatever, maybe you don't necessarily want to go out there and share with
25:25 the world what you just did. Whereas in the web development work, I feel like we're sort of,
25:29 you know, we figured most of it out. And so people are more willing, I think, to,
25:33 to share what they're working on. At least that's the impression that I got.
25:37 And often it's, yeah, I agree. And it's on the web already.
25:39 Yeah.
25:40 Whereas if you're working on the Apple self-driving car, like you can't even acknowledge its existence,
25:44 right? You're like working for the CIA. Yeah. I work for the state department. Okay. So anyway,
25:50 if you want to dig into more of this, I had Eva Jelowalski and, Dmitry Filipov on,
25:57 they're the two folks who did the, this and they printed it in a Python. So on Talk Python 176.
26:02 All right. Item number six, Brian, what do we got?
26:06 We have a darkening of the community. This is a very dark topic.
26:10 It is a little dark, but kind of in a good way.
26:12 Black is taking over. Black is, there's not in a bad way. So the project black is a code
26:20 formatter that just sort of formats all your code for you. And, and, and it like PEP 8 and
26:26 some of those flake eight and stuff tools were to tell you where places where you might want to look
26:31 at your code and change it. Whereas black is, yeah, I'm not, I can tell you if you want me to tell you,
26:37 but I'll just go ahead and change it.
26:39 It's awesome. Sort of taking, there's a lot of projects around the web or around, Python that are picking it up and, and using it as their standard. And, I think
26:49 it's been definitely very interesting in 2018 watching this happen.
26:52 I love that tool. So it's so good. Like, especially if you, if it's a community project, you're working
26:58 on a project with several developers and it's just so easy to standardize everything and you don't have
27:03 to have the back and forth conversations of defining your own code style. And I mean, assuming you like
27:08 the output that black produces, which I personally really like, and, it's, it's been super helpful
27:14 for real Python. We have like an, materials repository, some sample projects. So like every single
27:19 folder in there was created by a different person, by different author. And so I went down that, that
27:24 route of like, okay, we're going to define like a beautiful code style and then tell people how to
27:27 format everything and make sure it's, it's sort of adhere to. And that's really, really hard to do.
27:32 And so I just ran black on the whole thing, set it up on CI. So we'll check, it'll check the
27:36 formatting problem solved. And it took 15, 20 minutes to do that. So yeah. But what a lifesaver,
27:42 such a great tool.
27:43 So a couple of things about it. It's taken, it's one of the things that takes things like style and
27:48 format and stuff like that, and takes those out of code reviews. And so code reviews can just be
27:53 focused on algorithm and design and not on whether you use single quotes or versus double quotes or
27:59 something. The other thing is, black shows an example. It has part of it is a pre-commit hook
28:06 so that you can add a black pre-commit hook to your code. So everybody just as they're,
28:11 as they're checking it in, we'll have black run on their code. And, that highlighted the pre-commit
28:17 tool for me. And I think pre-commit is being used more and more because of black using it.
28:22 Yeah. I've heard that more than once. That's pretty awesome. You're like, wait,
28:25 what's a pre-commit hook? That thing exists in Git? Yeah, definitely.
28:29 Nice. I like it. The slogan, like you can have it in any color you like as long as you want it in
28:35 black. Yeah. I didn't know that that was from Henry Ford until somebody told me about Henry Ford saying
28:39 that about the Model T. Does it have like a theme song or what's going on here? Oh, well, every time I
28:44 use black, I hear Black Hole Sun from Soundgarden. So, I went ahead and just mentioned that in the
28:50 Black Hole Sun is Soundgarden was such a good band. They, I mean, they really came from like a special
28:55 era of music. We had them, Pearl Jam, Alice in Chains, Dinosaur Jr. Like there's a lot of good stuff
29:01 back there, but this is the only one that seems to be making it to a Python package. So way to go.
29:05 All right, Dan, for item number five, I feel like for a really long time, it was like, well,
29:14 pypi.python.org slash pypi is a little bit out of date and a little bit crummy, but we're working on it.
29:21 And it just, it seemed to be in this like stasis of like half improved forever, but not anymore, right?
29:28 Yeah. The new, new pypi, new Python packaging index went live this year and what an amazing
29:34 improvement. I mean, like, you know, throughout the whole project and I know you had a really,
29:38 really great show where you interviewed, you know, a bunch of people working on that transition and
29:42 launching the new and improved pypi. And it's just crazy, you know, like throughout the whole thing,
29:46 it's, it's been like rewritten from the ground up the way I understand it. It's new hosting
29:50 infrastructure and it's much more friendly now for new contributors. Like beforehand, I think it was
29:56 sort of like a home grown web framework because, well, it was that pypi, old pypi was created before
30:03 web frameworks really were a thing in Python. Here's the socket. Let's start with that. Like,
30:07 wait, wait, wait, wait, what? Yeah. And so, yeah, I just love it. You know,
30:11 like every time I'm searching for a package now, it just, it just looks so much better.
30:15 It looks like, it looks good. You know, it's like, it's when you compare it with JavaScript,
30:19 like the, the note JS world and NPM, I felt like, yeah, that was sort of the gold standard for a while
30:25 for what a online package repository should look like and the features it should have.
30:30 And I feel like pypi is now playing in that league and it's super nice.
30:34 I think it is super nice as well. And it's silly to say that it matters that it looks good because
30:40 it's just pip install a thing. But I think it really communicates a message to people who come into Python
30:46 or who are using Python, but especially people who are new and they don't, they're like, oh,
30:50 what is this stuff? Oh, there's a package injection. You're like, whoa, that thing is like from the
30:53 90s. What is that? Right? Like that, I mean, that communicates something about how much the
30:59 community cares for, for that kind of stuff. Right. Yeah. They also rolled that over with like
31:03 everybody in the world using it and very little hiccups. Yeah. It definitely is.
31:09 Really impressive. Yeah. It's super impressive. So a couple of interesting notes here when they
31:13 rolled it out first, it was pypi.io and I thought, oh, they're just being part of this hip new.io.
31:19 You know, it's like, it's like the .ly of yesteryear, you know, it's like Libya all of a sudden became a
31:28 super awesome place to have like your domain name because it had ly in the end. And so io, right,
31:32 it's super popular now, but now it's pypi.org. And the story is the pypi.org was owned by somebody else
31:40 and it took a long time for the PSF to get it from them. Whereas they could go and buy pypi.io. So
31:46 that was like an intermediate step in the whole rollout. Yeah. Some of the nasty details of web
31:52 development. Exactly. And then the other one is, this is a pretty interesting example of a popular
31:58 Pyramid web app, right? They, they started out in Flask and actually said, no, we'd rather go with
32:02 Pyramid, which I thought was a pretty interesting choice being a fan of Pyramid myself. And then Brian
32:08 finally marked down, right? Yeah. So they, the old PyPI was, did not support readme mark.md. It
32:14 would only was a readme dot, what is that? RSL or something or. Yeah. RST. Yeah. So I try to avoid
32:23 restructured text as much as I can. No offense to everybody that loves it. Yeah. I don't hear you.
32:28 It's like markdown, but it's broken. The stuff you put in there doesn't quite work.
32:33 Just kidding. No comment. Just kidding. Just kidding. I'm just saying the markdown
32:38 stuff doesn't always work in there. All right. Number four, item number four, Brian, what we got?
32:43 You touched on this a little bit before. Yeah. I think it's been fun to watch. It was unexpected.
32:48 The rise of Python in the embedded world. We actually did a segment on this in, on Python
32:53 Python bytes, but there's so many in partly from the maker movement and, and a lot of people trying
33:00 to get into controlling their, not just their computers, but their world with, internet of
33:05 things. And also just playing with, you know, adding lights to your bicycle and things like that with
33:10 microcontrollers and hardware hacking. Adafruit's been a big part of that, of, pushing it, but,
33:15 but there's a lot of projects around Python and embedded. Micro Python is one of those.
33:20 And micro Python's being used by both, by both hobbyists and, and serious, developers.
33:27 Hobbyists are serious developers too, but, but I think some companies are using micro Python as well.
33:32 Adafruit has its own version of this called a circuit Python. Actually, I don't know if it's
33:37 related to micro Python or not, but Adafruit has circuit Python. And then micro bit is,
33:44 code, Python base for, the micro bit processor. And then we even have, mu, which is a,
33:51 editor that directly interacts with, microcontrollers running Python.
33:56 It's been pretty fun to watch this year.
33:58 I really think the rights of IOT and Python is just taking root now. And micro Python is so
34:04 interesting to me. I mean, the fact that can take a Lambda expression and tie it to a hardware
34:09 interrupt just like never ceases to amaze me, you know?
34:12 Yeah, that's really cool. And so, and a lot of this is because somebody wants to learn electronics
34:18 or something, but they also want to make it do things. So you have to learn a programming language
34:22 and it used to be, we taught people C and like the barrier to entry for C is huge. Having a Python
34:30 be able to do that is wonderful.
34:31 Yeah, it is. So you talk of the IOT stuff. I learned something from Matt McKay, recently
34:37 on an episode, and I want to share this with you and just see what you think. So you guys click on this,
34:41 this link here. So here's an IOT device using a Python. And the idea is the title of the article,
34:49 this is on the Twilio blog. It's how I potty trained my kids using Twilio and an AWS IOT button.
34:55 So, so it's a little button like those Amazon dash buttons. And he put it near his son's bed.
35:02 And apparently there are kids sharing the room and wanting to make sure he didn't wake up and make a
35:07 lot of noise and wake up with the other kids. So he put this button that basically has like a poop
35:10 emoji and you click it and it says, I got to go. And the, the, the parents would come in and like
35:15 help out the kid while he was learning. And this is written using like the Python API on like AWS
35:20 Lambda and stuff like that. So yeah, the rise of Python and IOT.
35:24 So Q4, 10 years. And his kids are like, dad, did you have to put that on the internet?
35:29 This is awesome. I love this.
35:32 Exactly.
35:32 Yeah.
35:34 This is great.
35:35 Isn't this funny. So yeah, I mean, funny and like an amusing, but also like, actually
35:39 that's kind of helpful. Like I can see how that would be helpful as a parent. Cause you definitely
35:44 don't want your kids waking up and you have to be up for a few hours in the middle of the night.
35:47 So if an IOT button can help that, then get the IOT button. All right. Item number three
35:53 and it's packed full of goodness here, you guys. So legacy Python's days are fading. What do you
35:59 think? Well, I haven't used it for ages. So yeah, I haven't either. Dan is real Python
36:07 running on like what? Two, six.
36:09 Yeah. Not a lot of legacy Python for me anymore. Thankfully.
36:13 That's awesome.
36:13 And yeah, it's, I mean, it's, it's kind of crazy that it's, you know, it's a sort of crap
36:17 on us, the Python two end of life. And now we're not that far away anymore from it. So
36:22 yeah, we're actually super close. And the official item that I'm linking to as this piece of news is
36:29 Guido von Rossum's official statement. So I don't know, four years ago or something, it said, yeah,
36:35 Python two, it'll be done in 2020, whatever. Right? Like that's like a broad range for a period where
36:42 things stops a whole year. So we needed a more clear date. People are speculating. Maybe the date
36:48 of PyCon might be a cool one. Maybe the end, maybe the beginning. So he came out and said, let's not
36:53 play games with semantics. The way I see it for the situation for Python two seven is that end of life
36:59 is January 1st, 2020. And there will be no updates, not even source only security patches after that date
37:04 period. That sounds like the end.
37:06 Yeah. It's a good end though, because the, and some people might complain about the whole security
37:10 patches, but this is volunteer driven. So it's volunteers time. And I think that to let them work on new
37:16 things is good. Yeah, absolutely. And it's not like people are surprised, right? It's not like,
37:24 wait, last month they didn't, nobody told us there's going to be a new Python. Like it's been going on for
37:29 10 years. This will make it 12. It's time. Yeah. And there's a couple of loudmouths that two podcasts
37:35 that have been talking about it for a couple of years. So yeah, I think we're actually making a dent
37:39 with this referring to Python two as legacy Python. I think it's, it's making a difference and not everybody
37:44 loves it, but I know some people it's been working for.
37:47 Do you guys know, you know, with the, no more official security patches, after the, the end of life
37:53 date, I mean, obviously that this is a big like business opportunity for other companies to come in and
37:58 offer, you know, long-term Python two updates and stuff. Does anybody know who's, who's doing that right now?
38:03 Like some Linux distributions, I would guess, but I don't really know what's going on in that space.
38:08 Yeah. Possibly the big news that I just heard. So, you know, there's red hat enterprise Linux,
38:13 right? And they do a lot of that kind of stuff and the versions that they have live on for a long
38:19 period of time, but they just announced that they're dropping Python two support from red hat
38:23 enterprise Linux, which to me sounds like, okay, even the laggards are stepping forward.
38:28 So I think that that's pretty big, but I suspect that there's still a lot of support on the backend.
38:33 There's been a few people who've like talked about forking Python two and just supporting it
38:39 themselves as a single individual, but it seems, I'm sure there's going to be some of that, but like
38:44 as a company, like, let's say, I don't know, your bank of America, you're like, dude, we don't need
38:49 to upgrade. There's this guy. He forked it. We're good. You know what I mean? Like that just doesn't
38:53 seem realistic. It's also not just the C run, the CPython runtime anymore. It's like so much more
39:01 than that. So Django no longer supports Python two, like the scientific stack, like NumPy and stuff are
39:07 dropping support for Python two. So even if you still have the older stuff, you know, you're like,
39:12 well, we can just keep running. It's not just about what's running your code. It's also what you can use
39:16 in your code. And you know, that's going to start looking like it's just zooming off into the distance.
39:20 So not good.
39:22 Well, I mean, I think Python two will actually live on forever in some situations where like, for instance,
39:27 you got something that doesn't need a security patch because it doesn't touch the internet at all.
39:32 And as long as it will run on whatever CPU you're sticking it on for some people will have like a huge
39:39 program that they never touch because the requirements never change. And they use it every day,
39:45 but it doesn't need to be flexible. It doesn't need to be worked on that much. They'll probably upgrade
39:50 when they have to change the program. So yeah, I'm wondering if that also means that Apple is
39:56 finally moving to Python three and making that the default for macOS because the latest release,
40:02 it's still with two, seven, 10, I think. Yeah, that would actually be really, really nice.
40:09 You know, it's starting to shift in Ubuntu and just different Linux distributions, but also on macOS.
40:15 So yeah, that would be, that'd be super cool.
40:17 This portion of Talk Python to Me is brought to you by us. Have you heard that Python is not good
40:25 for concurrent programming problems? Whoever told you that is living in the past because it's prime
40:30 time for Python's asynchronous features. With the widespread adoption of async methods and the async
40:36 and await keywords, Python's ecosystem has a ton of new and exciting frameworks based on async and await.
40:42 That's why we created a course for anyone who wants to learn all of Python's async capabilities,
40:47 async techniques and examples in Python. Just visit talkpython.fm/async and watch the intro video
40:54 to see if this course is for you. It's only $49 and you own it forever. No subscriptions. And there are
41:00 discounts for teams as well. Brian, you're talking about things that don't require security patches.
41:07 I just saw a video from NASA talking about their trip to the moon and setting up a base there,
41:12 which is just awesome. So they had like all this sort of quick, different scenes of people doing
41:18 stuff. And one of the scenes was them working, they had sublime open, like editing a bunch of Python
41:24 code controlling like something about the, the moon mission, which was pretty cool. But I didn't catch
41:28 whether it was two or three. I couldn't tell.
41:32 That's pretty cool. Yeah. Someone else on Reddit did catch that the sublime was unregistered by the
41:37 way. Okay. So I guess the final thought here is we've also talked about this on Python bytes is that
41:48 there's the Python clock.org, the Python to death clock. And it has been updated for this, this new
41:53 end of life statement. And it reads one year, one month, two days. So if people are out there and
41:58 they got to start upgrading, like the death clock is ticking. All right. I also got item number two
42:03 and that I would categorize as the end of the innocence for PI PI and PIP. So we've heard about
42:10 a couple of issues, one, like right before 2018, like late 2017. And then another one again, this year
42:17 about people posting or taking over control and posting some form of evil PI PI package.
42:26 And primarily they look like they're kind of either a research project, a prank, or some kind of like
42:34 Bitcoin theft. So one of them installed like a clipboard monitor that would watch for a Bitcoin
42:42 addresses. And as soon as it saw one, it would like ship that off to like the mothership or something to
42:47 that effect. But I don't know if anything super bad that's happened, but it, I guess until, you know,
42:53 2016, 2017, PI PI as a place was just, it's incredible to me. It's a place on the internet
43:00 that was entirely unprotected and yet safe. I can imagine that. Yeah. If you had like a open FTP
43:08 server and you just put it on the internet, like all sorts of badness would happen to that thing
43:12 in a day. It would be full of all sorts of things you don't want it to be full of.
43:16 And here PI PI was going along and it was just, you know, the community sort of policed itself and
43:21 took care of itself. Plus it used to be really hard to become a person that could push things to
43:26 PI PI. Yeah. Maybe it correlates to the shipping of PI PI.org. Yeah. There seemed to be a lot of,
43:31 like, not a lot, like I'm trying to be more specific, but I think NPM again, in the JavaScript,
43:37 like Node.js world, they had a big, they actually was, it was something similar. I think it started with a
43:42 typo squatting attack, but then it was actually a worm that would then go. So if you got the malicious
43:46 package, nothing would really happen except that package as it was installing was looking for your,
43:53 like if you were registered with the NPM package repository and it would go in and patch the packages
43:59 that you owned to spread that worm further. I don't know if it was some payload, but it was,
44:04 you know, really exploding quickly. And I guess the same thing could happen with PI PI. Like,
44:08 I don't, I don't see why not. Yeah, it definitely could, but it hasn't. And let's,
44:13 let's not give any people ideas. So the, yeah, exactly. Don't say that security by obscurity.
44:24 Come on. Exactly. Right. Security by lack of creativity. Okay. So the two main problems were
44:31 basically like you pointed out, Dan type of squatting, which is like, we're going to upload a
44:37 package that takes a copy of Django and then slightly modifies it, set up.py and publish it under Django
44:47 or Django or whatever, like how you pronounce it without the J, right? So if people pip install and
44:53 they miss the spell, they get your bad thing. Right. So that's one. And the other was to take
44:57 advantage of folks who didn't understand why something wasn't available to them. So like,
45:04 if you tried to use regular expressions and you say re dot, and there's like, it says,
45:09 well, re is not defined. You're like, well, maybe I need to pip install re. No, you just need to import
45:12 it. Right. It's built in. So they were targeting the built-in libraries and I believe all the built-ins
45:16 are not patched. And Brian, didn't we cover something on Python bytes that would help guard against typo
45:22 squatting? It would like try to grab packages that also had misspellings around your package name.
45:28 I think there was something like that. Yeah, I think so. I don't remember what it's called,
45:31 but I think that exists and people could find it. Oh yeah. Yeah. I remember listening to that episode
45:35 and you guys were talking, I forgot what it's called too, but like it generates different variations of
45:39 your own package names and try and like register all of those. Yeah, exactly. It's very work intensive.
45:44 Yeah. So it's interesting on what is the fix? And I don't know what the fix is. The PSF is trying to get
45:49 more resources to support PyPI and the Python packaging infrastructure in general, but like
45:55 largely that's been improve it. Do things like you talked about previously, Dan, on just shipping
46:00 pypi.org. Right. But this validation, if you have so many releases of so many packages, how do you even
46:06 verify it? I don't know. It's tricky. It's not necessarily something that could be, that can be solved
46:10 with a straightforward technology fix, right? Like essentially you're, it's a problem that all of
46:15 these, let's call them online marketplaces for code repositories or whatever you want to call them.
46:20 You know, if you look at the Android app store, Apple app store, I mean the Apple app store, they have
46:25 these, these like crazy intensive review processes, you know, considering that there are millions of, of, of
46:31 apps in this app store and they're reviewing every single one. And I think there's even some manual steps
46:35 involved there. And even there, we've had malicious apps that were trying to steal people's, you know,
46:40 private information and it's a really locked down platform. So I think it's really, it's a challenge
46:45 that, that we're facing now because we're all connected over the internet and, you know,
46:48 people are really relying on these, on these package platforms. So, it seems like it's
46:54 something we'll, we'll have to try out a bunch of solutions and see what will work there.
46:57 I think we should get some of the, all of these new machine learning Python people to get involved
47:02 in and take a look at it because conceptually the normal updates for a package are little tweaks
47:09 on, on what it's doing. And if suddenly it starts, connecting to different servers all over the
47:15 world, or even just one where it used to not connect to anything, that would be a change. It would be a
47:21 different behavior than it was ever doing before that, that should be something that a machine could
47:26 catch.
47:26 Yeah, actually, I think you're right. I mean, there's almost no scenario where pip install a thing should
47:32 create an outbound connections besides to download the package.
47:35 Or, you know, even a package that doesn't normally use any connections or use requests suddenly adds,
47:42 you know, socket connecting or something. that's something to take, you know, at least we
47:47 can filter it down to things people could take a look at and say, is something weird going on here?
47:51 Well, it would actually be sweet to have like a CPython interpreter that is fully sandboxed that,
47:57 you know, can't like, you could just run code that people send you and you can run it safely and it
48:03 wouldn't be able to write to any folder on your disk or open sockets and whatnot. That would actually be
48:07 kind of sweet. I mean, it's all interpreted. So it's seems like that should be possible. I mean,
48:12 it's going to take a lot of work, I guess, but I think it's possible as well. You know, Dan,
48:15 you mentioned the online app store as maybe something. I wonder if there's something like
48:20 that in terms of packaging that we could add. And what I'm thinking is like, if I go build an iOS app,
48:26 I have to declare the things that's going to ask for, like it has to ask for access to contacts.
48:32 It has to ask for access to GPS. So, you know, you declare it needs the GPS feature. It needs these
48:39 other features, right? So what about a package that declares, like I don't declare any network
48:44 capabilities. So I don't talk on the network and things like, you know, I can depend upon packages
48:49 that do, for example.
48:50 On the flip side, none of these have caused a huge amount of damage. And I don't want to go
48:55 to the point of overreacting so that everybody has to take off their shoes to travel.
48:59 Yes, I know.
48:59 Things like that.
49:00 I know. We don't want feel good security that doesn't actually make any difference,
49:04 but makes life hard for everybody.
49:05 Yeah.
49:06 Yeah, I agree.
49:06 That's a good point.
49:07 I don't know what the answer is. I don't know what the fix is or if there is a fix,
49:10 but I would definitely declare the end of innocence for PyPI. That's my declaration as item two
49:15 for 2018. Dan, you have the honor of kicking off.
49:19 What is the number one story in the Python space for 2018? What happened?
49:23 Oh, man. What a sad honor, though.
49:25 Well, yeah, I know. It's not about the messenger, right?
49:30 Well, yeah. I mean, Guido van Rossum stepped down as the BDFL, which is both sad and ironic
49:37 because I guess BDFL stands for Benevolent Dictator for Life, which I think he still is now
49:44 technically, but he's just going to be less involved with the stewardship and leadership of CPython.
49:50 Yeah. He's a little more like the Queen of England now, in a sense, right?
49:55 Like he's still representing Python. He's still very active in the community, but he just said,
50:02 I'm going to let other people deal with the decisions around it. And basically,
50:07 went so far as saying, like, I'm not going to decide for you how you decide. Like it was even
50:12 a meta non-decision. So he said, all right, you all have to figure out how to govern yourselves.
50:17 And it looks like he's coming back in a little bit more in terms of participating.
50:23 Well, right. I mean, I think that's one of the reasons is because his involvement with Python
50:29 was just the stuff that wasn't fun. And now he gets to be involved with some of the things that
50:35 are fun, like speaking and mentoring and things like that.
50:38 You know, I think that's a really interesting point. I'm not sure I would want his job the way
50:44 it kind of looked a year ago, right? Just, you know, arguing over peps and stuff like that.
50:49 And you know that if you do stuff on the internet, that it's, there's definitely a very small minority
50:58 people that are, you know, abrasive, unfriendly. It just sucks. But even just a few people like that
51:04 will suck your energy for sure.
51:05 Yeah. And we still don't know what's going to happen. So we'll, maybe that'll be the story
51:10 for 2019 is what, what happens now.
51:13 I think you're right. I think 2018 is the year Keto step back. 2019 will be the year that
51:18 whatever, whatever takes place going forward is going to be that year.
51:23 So regarding the new governance structure for CPython, which obviously, you know, for any Python
51:28 developer, that should be really dear to our heart because it's going to have such a big impact
51:32 on the future of, Python and the CPython project. And so the last update that I saw,
51:39 basically we have a PEP 8,000 now, which is, a PEP that discusses the, the various new proposals that
51:48 have been forward, that have been put forward regarding the Python language governance. And,
51:53 I just recently saw, it was like two or three days ago, Victor Stinner's post on discuss.python.org,
52:01 where he's comparing the seven Python governance peps. So I guess like actually like the different
52:06 proposals, they're separate peps, but they're also listed under PEP 8,000. And so there's different
52:10 models that have been put forward. Like for example, leading the project with a trio of, three co-leaders.
52:17 there could also be a community governance model where there's no central authority. So
52:21 like a total 10 80 from, you know, a dictatorship to no central authority. I don't know how it would,
52:26 how that would look like, you know, in concrete terms, but there's like different models that are
52:30 being discussed right now. And, I think it's going to be very interesting to see what the CPython
52:35 team ends up with there.
52:37 I agree. And it's, it's pretty important. Like even simple stuff like Lucas Lange, also
52:43 creator black, what we spoke about earlier, he's doing the release management for CPython for the
52:48 next couple of releases. And he was thinking like, Hey, it would be great to release this yearly,
52:52 but we can't even discuss having it released yearly instead of every 18 months because we don't even
52:58 know how to decide whether we, how would we, who decides, how do we vote? We don't know. So,
53:04 so many things like that are just like on hold until this gets figured out. So there's a lot of
53:09 pressure to make it happen.
53:10 Definitely. Yeah. It's like, it's, it's a total bottleneck, I guess for, yeah, like you were saying,
53:14 you know, just to put out a new release, you would have to, they would have to figure that out first.
53:17 And so, I'm, I'm pretty sure that also means it's going to be a decision that's going to be made
53:21 relatively quickly there so we can all move on.
53:24 Yeah, absolutely. One thing you did note though, is that Guido starting to come back and be more
53:29 active in the community again, just not in this BDFL decision maker way, right?
53:34 Yeah. And I thought that was kind of cool. you know, I, I don't know, it might be coincidence,
53:38 but I don't think it was. So as I was putting together, you know, the recent, PyCoders
53:43 weekly issue, I saw that there were a couple of interviews with, Guido out there or articles
53:49 that, you know, had like little sound bites from him. So he did, a really long,
53:53 like video based interview on the MIT AI podcast. So it's actually on YouTube. It's more like a video
53:58 based show, but I think it's also podcast, that I thought was really interesting.
54:02 And, he also started blogging again. So he has this personal blog at, I think it's
54:08 neopythonic.blogspot.com or something like that. And he put out a new post there and it's, I think
54:13 it's, it's, it was kind of nice to see some activity there again, because I'm, I'm sure it's also a sign
54:18 that, you know, now he maybe feels like he has more, more time to do that or feels inspired to do
54:23 things like that again. And I think that's a really, really positive sign and just for his
54:27 personal wellbeing, I think.
54:28 I think it's a positive sign as well. And just like Brian and I were talking about earlier,
54:32 a lot of his energy had previously gone into like debating PEP 572 and just dealing with all the,
54:39 the decision-making and now he can focus back on, you know, what most people would want to focus on
54:46 coding software, cool stuff like that. And it sounds like he's getting back into it. So that's awesome.
54:51 Yeah. Yeah. And I mean, you know, what a, what a crazy responsibility to be at the center,
54:57 like at the focus of it all, you know, and, and to be, to be the one person that needs to make all of
55:02 these decisions. And sure, you know, like he was delegating a lot of these decisions and everything,
55:06 but the way the CPython governance was set up before it was like all focused on one person. And,
55:13 yeah, I mean, it's tough to do that for several decades, right?
55:17 Yeah. I honestly, I was surprised that he had kept going in that role for that long. I mean,
55:22 it's sad to see him step down, but at the same time that was 25 years. Like how many other people
55:28 have a job that goes 25 years? That's high pressure, right?
55:31 Yeah. It's incredible. I mean, thank you so much, Guido.
55:33 Yeah, absolutely. It's Python's a special place and it has a lot to do with him. All right, guys,
55:39 that's it. Those are our top 10 for 2018. It's been a fun year, hasn't it?
55:43 Absolutely.
55:44 Yeah.
55:44 Yeah. I think 2018 just has a more, more good stuff to come. So it doesn't seem like anything
55:50 is slowing down then. And that's pretty awesome. Now, before we round out the show, you have to
55:56 answer the two questions, both of you. So Brian, let's start with you. If you're going to write some
56:00 Python code, maybe test a little something or other, what editor would you use?
56:04 My turn.
56:04 Right on.
56:05 And Stan.
56:06 Yeah. Good one. Dan?
56:07 I'm still a Sublime guy, although I have been experimenting with Visual Studio Code and it's
56:12 pretty sweet actually.
56:13 Yeah. I would say like the Sublime Visual Studio Code, there's probably a lot of bleed over between
56:18 those and with all the activity around Visual Studio Code, it's got to be eating into the
56:22 Sublime world.
56:23 Yeah, definitely. Probably. I like what VS Code is. I like watching what VS Code is doing.
56:28 They're just not quite there with the testing yet. I'll take a look again when they get that
56:33 up to speed.
56:34 Yeah, I agree. I think they're doing really sweet stuff. Okay. Brian, notable PyPI package?
56:39 One preferably that doesn't have any badness in it?
56:41 pytest.
56:42 pytest. Right on. Dan?
56:43 Sweet. So I was going to say black again, but we talked about this at length. So I'm going to say
56:49 pytest-ICDIF. So it's a pytest plugin that gives you better error messages for your pytest
56:58 assertions in some cases. So it'll basically do like a more extensive like diff on, let's say,
57:04 when it's comparing two dictionaries and it'll tell, you know, those keys were added, those keys
57:08 were removed, and it has like color highlighting. And I just found that this is like a really,
57:13 really sweet little tool. And I started installing it in all of my projects that have tests.
57:17 And I highly recommend it.
57:18 That sounds great. I may have to go check this out.
57:20 Cool.
57:21 All right. So a straight sweep for pytest there. That's great. All right, guys. Thanks so much
57:25 for being on the show and being part of this. It's so many cool things to talk about. And I
57:30 really appreciate it.
57:30 Thanks for having us. Thanks.
57:32 Yep. Bye.
57:32 Bye-bye.
57:33 This has been another episode of Talk Python to Me. Our guests in this episode were Brian
57:39 Aukin and Dan Bader. And it's been brought to you by CloudBolt and us at Talk Python Training.
57:44 Spend your work time fulfilled. Write Python and Django code at CloudBolt, developing their state-of-the-art
57:50 cloud management software in beautiful Portland, Oregon. Visit talkpython.fm/CloudBolt to join
57:57 the team. Want to level up your Python? If you're just getting started, try my Python Jumpstart by
58:04 building 10 apps course. Or if you're looking for something more advanced, check out our new
58:08 async course that digs into all the different types of async programming you can do in Python. And of
58:14 course, if you're interested in more than one of these, be sure to check out our everything bundle.
58:18 It's like a subscription that never expires. Be sure to subscribe to the show. Open your favorite
58:23 podcatcher and search for Python. We should be right at the top. You can also find the
58:27 iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss
58:33 on talkpython.com. This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it.
58:39 Now get out there and write some Python code.
58:41 We'll see you next time.