WEBVTT

00:00:00.001 --> 00:00:05.020
When you think about Microsoft, do you think about Python? Maybe not, but you probably should.

00:00:05.020 --> 00:00:09.700
They've been doing an incredible amount of work to improve Python for folks on Windows,

00:00:09.700 --> 00:00:15.340
as well as the broader community. You can, of course, look at the wild growth of Visual Studio

00:00:15.340 --> 00:00:20.340
Code, but did you know that five of the core developers work there and that the majority of

00:00:20.340 --> 00:00:25.240
Python development actually happens on Windows? Join me along with Steve Dower, a core developer

00:00:25.240 --> 00:00:31.000
working at Microsoft who published an amazing retrospective on Python at Microsoft entitled

00:00:31.000 --> 00:00:37.480
Python at Microsoft, Flying Under the Radar. This is Talk Python To Me, recorded December 6th, 2018.

00:00:37.480 --> 00:00:55.180
Welcome to Talk Python To Me, a weekly podcast on Python, the language, the library,

00:00:55.180 --> 00:01:00.020
the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter,

00:01:00.020 --> 00:01:04.700
where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm,

00:01:04.700 --> 00:01:09.180
and follow the show on Twitter via at Talk Python. Steve, welcome back to Talk Python.

00:01:09.180 --> 00:01:10.180
Thanks for having me back.

00:01:10.180 --> 00:01:13.920
Yeah, it's great to have you back. There's so many interesting things that you're doing,

00:01:13.920 --> 00:01:20.360
and you've been really instrumental in pushing Python on its most popular platform, actually,

00:01:20.360 --> 00:01:26.040
on Windows. And I think you actually have a few interesting surprises for the listeners. So I'm

00:01:26.040 --> 00:01:31.080
really excited about what we're going to talk about. But let's set the stage by just talking about what

00:01:31.080 --> 00:01:36.620
you do day to day. You're a Python core developer, and you work at Microsoft. Take it from there.

00:01:36.620 --> 00:01:42.360
Yeah, so I really get two hats in most conversations. I'm either wearing my Microsoft hat or my CPython hat.

00:01:42.920 --> 00:01:47.640
And I feel like most times when I come and chat with you, I'm normally wearing the Microsoft hat,

00:01:47.640 --> 00:01:52.480
which is fun. And that's certainly the one I have on today. So for CPython, I'm one of the Windows

00:01:52.480 --> 00:02:00.340
experts, do a lot of the Windows support and builds and everything. At Microsoft, I'm one of the Python

00:02:00.340 --> 00:02:06.960
experts. And so I get to kind of roam around the company working with a lot of groups as we really

00:02:06.960 --> 00:02:12.340
try and ramp up on Python in a big way, getting to help out various teams, make sure they're doing

00:02:12.340 --> 00:02:19.480
a good job. Like, we don't have a huge base of, you know, Python culture, Python experts throughout

00:02:19.480 --> 00:02:24.520
the company. So there's a lot of really, really good engineers, like expert engineers in all of

00:02:24.520 --> 00:02:30.120
their languages. And bridging that gap between, you know, I'm an expert C# programmer, I'm an expert

00:02:30.120 --> 00:02:37.100
C++ programmer. And how do we make you look like an expert Python programmer is a really big part of

00:02:37.100 --> 00:02:41.320
what I'm doing right now. But it's a lot of fun. Yeah, I guess two things really strike me there. One is,

00:02:41.760 --> 00:02:47.240
you do kind of live in this middle ground where the context matters so much, right? Like when you're

00:02:47.240 --> 00:02:52.500
at, say, PyCon, people are like, Oh, my gosh, you're a C core developer, and you work at Microsoft.

00:02:52.500 --> 00:02:56.640
What's that like? Right? Whereas when you're at Microsoft, they're like, what do you mean you're

00:02:56.640 --> 00:03:01.660
a Python person? Why don't you do C# and C++? What's going on here? Right? So there's like this

00:03:01.660 --> 00:03:07.440
really big context switch, I bet. Oh, yeah, I have virtual hats that I will literally mime putting on and

00:03:07.440 --> 00:03:12.140
off. And I feel like I just need to get real ones. Yeah, absolutely. So there's a couple of things we're going to

00:03:12.140 --> 00:03:20.000
talk about. And the first one really want to dig into is Python's journey at Microsoft, because Microsoft's one of

00:03:20.000 --> 00:03:26.880
these companies that started out not super open in terms of its open source contributions, and its

00:03:26.880 --> 00:03:33.300
culture around open source, right? Famously, Balmer had not super positive things to say about Linux,

00:03:33.300 --> 00:03:40.740
for example. But I feel like Microsoft has really made this transformation. And you've written this

00:03:40.740 --> 00:03:47.140
really cool article essay, maybe on your personal journey that sort of charts some of those things

00:03:47.140 --> 00:03:53.280
as well. So maybe we'll start there. Yeah, it felt a bit like writing like the start of my memoirs,

00:03:53.280 --> 00:03:59.560
which I feel like I'm too young to start doing that. But, but no, I was I was invited. We've got

00:03:59.560 --> 00:04:05.420
a series of people's kind of open source journeys, open source stories coming out right now. We've already

00:04:05.420 --> 00:04:13.800
had one about C#, and rewriting the compiler for that one about the .NET foundation, if I'm

00:04:13.800 --> 00:04:19.680
recalling correctly. And yeah, now one about Python's journey. And it was, I was thrilled to get to write

00:04:19.680 --> 00:04:25.120
it, because it has been an exciting time. And it's taken a number of years. But it's one of those things

00:04:25.120 --> 00:04:29.920
where I've at least been able to observe it along the way and participate in a few places.

00:04:29.920 --> 00:04:35.600
Yeah, so where does this journey start? You know, traditionally, Microsoft's been a kind of,

00:04:35.600 --> 00:04:41.180
we're going to build it here, even if it exists, we're going to create our own version of the thing,

00:04:41.180 --> 00:04:46.940
right? There's famously Java and C#. In the early days, you know, C# was something or a

00:04:46.940 --> 00:04:51.440
reaction to Java, I think that was more of a legal issue on around Sun and Oracle than it was more

00:04:51.440 --> 00:04:56.880
the not invented here thing. But there has been that kind of stuff around, say, like source control and

00:04:56.880 --> 00:05:03.320
other things that in the early days, I feel like maybe made Python not as welcome as it could have

00:05:03.320 --> 00:05:07.920
been. Yeah, yeah, there's, there's certainly when a company gets to a certain size, there's always a

00:05:07.920 --> 00:05:13.320
lot of that going on, even within the company, like we still have teams all over the place inventing the

00:05:13.320 --> 00:05:17.260
same thing, not realizing that another team at the same company is doing it at the same time,

00:05:17.260 --> 00:05:22.140
which is really interesting. And one of the great things that I get to do, because I bounce around

00:05:22.140 --> 00:05:25.900
a lot of teams is actually connect some of these up and say, Hey, you know, this other team is doing the

00:05:25.900 --> 00:05:32.040
same thing as you. But we have, we're definitely a lot better at looking out than we used to be, because

00:05:32.040 --> 00:05:36.320
there was certainly a time that we'd hear about problems, or we'd recognize problems, or we'd have

00:05:36.320 --> 00:05:43.620
problems ourselves, and go, Well, the only people who can solve this is us. So let's build a solution. And

00:05:43.620 --> 00:05:48.860
that's dramatically changed. Like the first thing now is, let's look out and see how other people have

00:05:48.860 --> 00:05:56.560
solved it. And let's help them, you know, let's help our developers use it. Let's help developers

00:05:56.560 --> 00:06:02.960
outside the company be able to use it as well. And it's also just hiring. I mean, yes, people change at

00:06:02.960 --> 00:06:08.640
the company, people, you know, the older people leave, the younger people join. And they're coming in with a

00:06:08.640 --> 00:06:14.340
totally different experience these days. All the open source stuff is everywhere throughout academia,

00:06:14.340 --> 00:06:17.980
it's everywhere, obviously, throughout open source. And that's what people are coming into the company

00:06:17.980 --> 00:06:23.060
with, right? Even scientists are now, you know, doing pull requests and stuff on GitHub, right? Like,

00:06:23.060 --> 00:06:30.260
or writing Python code instead of MATLAB or Mathematica, like it's all aspects of academia seem to be

00:06:30.260 --> 00:06:33.340
embracing open source, not just the computer science side.

00:06:33.340 --> 00:06:39.620
Yeah, no, absolutely. And I mean, some of the biggest things that we look at and say, oh,

00:06:39.620 --> 00:06:43.320
this is a computer science thing were created by scientists in the first place. Like,

00:06:43.320 --> 00:06:48.500
Travis Oliphant is quite happy to stand up and say he's not a computer scientist. And yet,

00:06:48.500 --> 00:06:50.840
where did NumPy come from? Where did SciPy come from?

00:06:50.840 --> 00:06:56.440
Exactly. Exactly. And it's made such a massive, massive difference. Yeah. So I guess some of the

00:06:56.440 --> 00:07:01.860
things that I see that sort of highlight some of this contrast, and I only know from the outside,

00:07:01.980 --> 00:07:06.760
right? I don't bounce around inside the engineering teams there, are things like Microsoft created

00:07:06.760 --> 00:07:12.420
CodePlex as sort of an alternative to places like GitHub. I don't know exactly the timing of when

00:07:12.420 --> 00:07:16.420
GitHub came out versus CodePlex, but then eventually I decided, you know what, everybody's at GitHub.

00:07:16.420 --> 00:07:23.480
Let's just move things like the .NET open source projects over to GitHub properly, ASP.NET famously,

00:07:23.480 --> 00:07:28.320
for example. Then, well, let's just buy GitHub because that's like where the action is,

00:07:28.660 --> 00:07:35.540
right. And really embracing this place that is, you know, has plenty of Microsoft stuff there,

00:07:35.540 --> 00:07:41.820
but plenty of Python and other others as well. And, you know, there's other examples of those types of

00:07:41.820 --> 00:07:45.800
things where it's like, well, it started out maybe this private build your own thing, and now we're

00:07:45.800 --> 00:07:50.040
going to go do our, you know, another one is like the source control story, like TFS and all that,

00:07:50.040 --> 00:07:54.800
and now Git and GitHub and so on. So I think those are the outside changes I see.

00:07:54.800 --> 00:08:01.600
But, you know, on the inside, you've seen maybe, I guess one of the biggest stories is Python is

00:08:01.600 --> 00:08:06.700
really starting to gain true traction, right? It's like, it's starting to show up in lots of products

00:08:06.700 --> 00:08:12.200
and get some real legitimacy, not just the stepchild thing that we have to care for because some people

00:08:12.200 --> 00:08:12.880
demand it.

00:08:12.980 --> 00:08:18.380
Yeah, no, it absolutely is. One of the mandates that came down a little while back is that all of,

00:08:18.380 --> 00:08:24.980
like, one of the must support languages for APIs and, you know, anything that you might want to

00:08:24.980 --> 00:08:29.240
manage from outside the company, anything you might want to manage on Azure. So if you want to create

00:08:29.240 --> 00:08:34.780
virtual machines or create new storage accounts, be able to push, pull files from wherever, Python is

00:08:34.780 --> 00:08:40.660
one of the required languages. Like, you can't call your service ready until it has Python support for it,

00:08:40.980 --> 00:08:47.960
along with a handful of other languages. But that hitting that point was was really exciting. And also a little

00:08:47.960 --> 00:08:54.320
bit terrifying, because suddenly there were huge, huge code bases that were suddenly deemed not ready. And of

00:08:54.320 --> 00:09:00.620
course, we didn't pull anything, right? Yeah, it was it was an internal. The next thing you need to do is add your

00:09:00.620 --> 00:09:06.840
Python support, right? And so then all these teams with this complicated code base come to you and your couple

00:09:06.840 --> 00:09:13.820
colleagues and say, we need some help. Yeah. And luckily, a lot of luckily, a lot of them did. There was certainly early on a few that that

00:09:13.820 --> 00:09:21.000
would happily just go their own way. And, you know, we'd find out later on, this hasn't happened for a few years now. But we did

00:09:21.000 --> 00:09:26.620
occasionally get emails saying, Have you seen this JavaScript code that someone wrote in Python and then published and called it

00:09:26.620 --> 00:09:51.240
Microsoft? We're like, Well, we'll go find them. Thank you. That hasn't happened recently. But there's definitely some transition to get there. And it has reached the point, like I mentioned in that story on medium, the Azure CLI team, actually coming to us and saying, We think Python's a good choice for this. Now, for writing a command line tool, that is going to get huge. And we have to be able to manage everything.

00:09:51.240 --> 00:10:05.240
Yeah, that's really great. And in your story, you talk a little bit about, you know, they expected this to take a long time. And you're like, No, no, we can knock this out pretty quickly. And they're like, Uh huh, sure. Well, just give it a try. Right? Like, you want to talk about that bit?

00:10:05.240 --> 00:10:33.240
Yeah. Well, I mean, any, I feel like most people who've built a command line in Python using argpars, not going to be too surprised at how quickly you can get something working. But I think they were previously doing it in Node.js. And they'd started it much, much earlier on and basically ended up with their entire command line parsing library written by themselves. And so by comparison to that, being able to take something out of the standard library for a language and just use it, and be able to use it so dynamically,

00:10:33.240 --> 00:10:48.240
like they have thousands and thousands of commands, which Python can handle because you haven't statically declared everything like you haven't had to write them all out in full, you can loop over a list of files and read stuff out of it and build up your command line that way.

00:10:48.240 --> 00:10:53.800
And so it was, I mean, it was, it was an intense few days. Like, don't, don't get me wrong.

00:10:53.800 --> 00:10:54.320
Of course.

00:10:54.320 --> 00:11:04.960
But, but yeah, it, the first version came out, you can dive back into the Git history and, and see my commits. And I was basically going straight to the repo from the start. So they're all there.

00:11:05.620 --> 00:11:11.160
And yeah, it came out with something that had kind of a scattering of features that they said they were interested in.

00:11:11.580 --> 00:11:26.060
And then got to stand up and present it and hand it off, which is a really nice feeling to, to be able to give some code to a team and, and walk through it and spend an hour showing off the bits and pieces and have them say, you just saved us months of work.

00:11:26.060 --> 00:11:27.020
Thank you so much.

00:11:27.020 --> 00:11:28.020
Yeah.

00:11:28.020 --> 00:11:32.060
Like that, that's definitely one of the best feelings that I've had in, in what I've gotten to do with Microsoft.

00:11:32.060 --> 00:11:43.220
Yeah. That's a crazy, crazy experience. And I, you know, you've heard stories about Python having this effect in different ways, but that's, that's a pretty stark contrast right there. That's great.

00:11:43.220 --> 00:11:49.880
Is it still in the Azure CLI? Is that still in Python? Is that shipping now?

00:11:49.880 --> 00:11:56.680
Yeah, it's absolutely still in Python. It bears, I think there's maybe two lines of code that are still there from what I wrote because they've

00:11:56.680 --> 00:12:07.020
gone and, and rewritten all of it. They've actually eventually migrated off argparsebecause it turns out that when you are getting up to thousands of sub commands that has some performance problems.

00:12:07.020 --> 00:12:16.320
Whoever wrote that is like, Oh, what's the upper limit? Like what's the most commands you've ever seen in an app? 24. Okay. Well, this is going to be fine. Right?

00:12:16.320 --> 00:12:26.060
I stumbled into a conversation at a conference between two people working on other command line libraries. I forget exactly what the libraries were, but they were

00:12:26.340 --> 00:12:38.180
discussing how their performance was kind of falling off around the 50, 60 sub commands level. And, you know, was that okay? That's probably okay. And I'm like, Hey guys, we've, we've got a few more than that.

00:12:39.120 --> 00:12:54.340
But the library they, they went and turned it into, as I said, Microsoft is full of amazing software engineers. And a few of them learned Python, didn't necessarily know it to begin with and created what was eventually refactored out into this library called knack.

00:12:54.900 --> 00:13:08.540
That is a highly scalable command line parser. The amount of work it takes to get started is higher than argpars. But the result that you get is much, much better.

00:13:08.540 --> 00:13:11.020
How interesting. I'd never heard of knack actually. That's cool.

00:13:11.020 --> 00:13:21.520
I hadn't actually heard of it either until I was going to the team and saying, Hey, there are people who would love your command line library. Have you thought about refactoring out? And they just sent me a GitHub repo back and said, here it is.

00:13:21.520 --> 00:13:51.500
Oh, beautiful.

00:13:51.500 --> 00:13:53.320
thousands of sub commands. Wow.

00:13:53.320 --> 00:14:03.380
And I don't actually know that he realizes how much of an achievement he's already made to get it down that far with something so big.

00:14:03.380 --> 00:14:10.700
That is so cool. I mean, it's projects like that, that kind of push the boundaries and make it better for everyone. If that gets pushed back into CPython.

00:14:10.700 --> 00:14:11.720
I, yeah, I hope so.

00:14:11.720 --> 00:14:39.620
Yeah. Let's talk about the early days. I guess the very first recollection I have of Python and maybe similarly Ruby making its way around Microsoft is first. I heard of iron Ruby and then iron Python. I think that was the order they were created. And then maybe shortly after that was Python tools for visual studio. This is not VS Code that people know today, right? This is like traditional.

00:14:39.620 --> 00:14:44.060
So that's the original Windows only visual studio. You had these tools you could plug in for Python, right?

00:14:44.060 --> 00:15:10.200
Yeah. So that's basically the timeline there as well. I came in just after Python tools for visual studio was getting started. I think I was there for the 1.0 ship party. I don't remember exactly. I can probably go back and figure that out, but I was definitely there for a party, which was one of the earliest releases. But yeah, before that, there was a project to make the CLR, the common language runtime work well for dynamic languages.

00:15:10.200 --> 00:15:29.200
The thing that .NET runs on top of to make it work well for dynamic languages, which have, you know, a whole different style of, you know, when you, you have the code and you compile it into this intermediate language, but you still don't know what your name lookups are going to find, which we're totally used to in Python.

00:15:29.200 --> 00:15:37.840
But the common language runtime was not initially designed for that. And so there was this project to make a dynamic dynamic one that could handle all of that.

00:15:38.200 --> 00:16:02.760
And IonPython and IonRuby were kind of the test cases for let's show this working on real languages. And both of them got to very good states in terms of supporting the language at that point in time. IonPython is still going in bits and pieces. I do occasionally bump into people at quite large companies with very serious uses of it still. They're actually quite happy with it.

00:16:02.760 --> 00:16:17.540
That's cool. There's some interesting integration cases, right? Like you can do things like the UI framework WPF in Python on top of Iron Python, I think. And stuff like that that you just, you're like, wait, you can put these technologies together?

00:16:17.540 --> 00:16:44.320
Yeah. And it works fairly nicely provided you're happy to live in that 2.7 world with severely restricted library support at this point. There were a number of projects to try and bridge that gap, which have largely fallen away, unfortunately. So the alternate implementations besides CPython are a little bit weaker in terms of extra library support, which is fairly off-putting. The ecosystem is just so critical to Python.

00:16:44.320 --> 00:17:06.700
Yeah, without it, it's just not even hardly the same thing. As a core developer, what are your thoughts on these other runtimes, right? Like PyPy, Jython, Iron Python? Do you look at them and go, ah, the energy would be better placed just like trying to make CPython itself better or they're good experiments? What are your thoughts?

00:17:06.700 --> 00:17:30.320
My thoughts are very big and complicated on that. And I think we'll probably go a little bit deeper in the next topic when we get to it. But certainly there's been a huge amount of value come out of the fact that they exist and that people have worked on them and that there's been collaboration between the reference implementation, CPython, and the other implementations. Everyone has gotten better as a result of that.

00:17:30.700 --> 00:17:42.880
Now, whether the energy is better redirected, honestly, there's not that much energy in the other projects, which is a real shame. But it also means that redirecting it doesn't necessarily add a lot.

00:17:42.880 --> 00:18:05.320
And I'll point out for the dynamic language runtime, Microsoft did two proof of concepts because having Iron Ruby and Iron Python flushed out things that just having one would not. And in the same way, I think having CPython, Jython, PyPy, Iron Python flush out issues in what Python the language is that you would never find out if you only had one implementation.

00:18:05.540 --> 00:18:14.000
Oh, that's interesting. So, you know, if they say run the test suite, you're like, actually, this part's really vague, we don't actually know what this means. That could help make CPython better for sure, right?

00:18:14.000 --> 00:18:40.840
Yeah, there were a lot of language semantics that were clarified, particularly while Iron Python was growing up. PyPy has been coming up a little bit later. It has a lot more support. There's a lot more people working on it than the other projects, but they're trying a lot harder to be compatible with whatever CPython does. And so there isn't as much kind of push and pull on what does the language actually mean here as when the other languages were really going.

00:18:40.840 --> 00:18:41.280
Interesting.

00:18:41.280 --> 00:18:41.460
Interesting.

00:18:41.460 --> 00:19:02.380
This portion of Talk Python To Me is brought to you by Linode. Are you looking for hosting that's fast, simple, and incredibly affordable? Well, look past that bookstore and check out Linode at talkpython.fm/Linode. That's L-I-N-O-D-E. Plans start at just $5 a month for a dedicated server with a gig of RAM.

00:19:02.380 --> 00:19:22.820
They have 10 data centers across the globe, so no matter where you are or where your users are, there's a data center for you. Whether you want to run a Python web app, host a private Git server, or just a file server, you'll get native SSDs on all the machines, a newly upgraded 200 gigabit network, 24-7 friendly support even on holidays, and a seven-day money-back guarantee.

00:19:22.820 --> 00:19:24.480
Need a little help with your infrastructure? Need a little help with your infrastructure?

00:19:24.480 --> 00:19:29.200
They even offer professional services to help you with architecture, migrations, and more.

00:19:29.200 --> 00:19:35.200
Do you want a dedicated server for free for the next four months? Just visit talkpython.fm/Linode.

00:19:35.200 --> 00:19:48.360
Let's talk a little bit about your being a core developer and Microsoft employing you, so basically, in some sense, being a sponsor of your open source contributions in that sense.

00:19:48.480 --> 00:19:55.100
Maybe I'll kick it off with a joke th@brianokken over on Python Bytes showed me.

00:19:55.100 --> 00:19:57.940
So it was, you know, it's around Christmas time.

00:19:57.940 --> 00:20:02.760
So there was this cartoon, and there was like one of these Santas at the mall, right?

00:20:02.760 --> 00:20:06.340
And people, kids go and sit on the lap of the Santa, and the Santa asks, what do you want for Christmas?

00:20:06.340 --> 00:20:08.380
And the Santa says, now, what do you want for Christmas, girl?

00:20:08.380 --> 00:20:10.360
Come on, be realistic.

00:20:10.360 --> 00:20:14.340
He says, I want to know if enough donations that I can just work on my open source project.

00:20:14.340 --> 00:20:17.680
He goes, okay, what color do you want your dragon to be?

00:20:17.680 --> 00:20:30.160
But in a sense, right, like you found this place in the world where, and there's other companies as well, but where Microsoft is basically supporting your work to make CPython better, right?

00:20:30.160 --> 00:20:30.600
Yeah.

00:20:30.600 --> 00:20:36.600
And the story of that coming about is very different from what everyone gets now.

00:20:36.600 --> 00:20:46.060
I feel like now people just, they are going through the interview process to get a job at Microsoft and just kind of tack on, hey, you should give me, you know, one, two days a week to work on this open source project that matters.

00:20:46.060 --> 00:20:47.460
And we just go, okay, sure.

00:20:47.460 --> 00:20:52.300
Back when I did it, I spent weeks negotiating with lawyers.

00:20:52.300 --> 00:20:57.080
I had patent attorneys basically interviewing me to find out what I was going to do.

00:20:57.640 --> 00:20:57.800
Yeah.

00:20:57.800 --> 00:21:06.520
Would you somehow taint all of Microsoft by like interacting with open source code and you've seen some GPL and then all badness breaks loose, right?

00:21:06.520 --> 00:21:06.960
Yep.

00:21:06.960 --> 00:21:11.040
A whole lot of advice on, you know, don't have other code up on the screen while you're working on it.

00:21:11.340 --> 00:21:17.460
And just the, like the risk management that we were doing at the time makes total sense.

00:21:17.460 --> 00:21:18.460
It really does.

00:21:18.460 --> 00:21:31.220
And when you understand that the aim of our legal team is not to help us win the eventual court case, it's to avoid it in the first place, you can see why they're so protective.

00:21:31.220 --> 00:21:39.080
Because basically once any kind of IP litigation goes to court, everyone's already lost, really.

00:21:40.160 --> 00:21:42.480
A couple of lawyers have won, but they're not our lawyers.

00:21:42.480 --> 00:21:44.540
So they're not interested in it going to court.

00:21:44.540 --> 00:21:47.460
They, this is other people get paid for that time.

00:21:47.460 --> 00:21:54.940
So really it's, it's take as much care as possible so that it, whatever comes up just gets thrown out as quickly as possible.

00:21:54.940 --> 00:21:57.040
And so there was a lot of caution there.

00:21:57.040 --> 00:21:57.280
Yeah.

00:21:57.280 --> 00:21:58.340
But again, it's risk management.

00:21:58.340 --> 00:22:09.880
So those of us who were doing that kind of work at that time, and we had at like the same time I was doing that, we had MS Open Tech still, which was the totally separate company.

00:22:09.980 --> 00:22:16.820
That just happened to work on the same campus and some of the same buildings as Microsoft and beyond the same email network and everything.

00:22:17.300 --> 00:22:25.880
Who were legally allowed to do to work on these projects because there, there was some legal boundary where you couldn't sue all of Microsoft if you decided to sue them.

00:22:26.300 --> 00:22:28.140
So I was doing it from within the company.

00:22:28.140 --> 00:22:34.140
Like people like Dino Veland had already done before me and a handful of others.

00:22:34.140 --> 00:22:34.360
Yeah.

00:22:34.360 --> 00:22:35.420
I guess it's worth mentioning.

00:22:35.420 --> 00:22:39.660
There's more than just you as the core developer at Microsoft, right?

00:22:39.660 --> 00:22:41.540
There's Brett Cannon and who else?

00:22:41.540 --> 00:22:44.960
Barry Warsaw is at LinkedIn, but we count him because we like LinkedIn now.

00:22:44.960 --> 00:22:51.320
Eric Snow and Trent Nelson is starting whenever immigration gets their act together, I believe.

00:22:51.320 --> 00:22:53.000
Right on.

00:22:53.060 --> 00:22:55.500
The fun of being an Australian working at Microsoft in the US.

00:22:55.500 --> 00:22:56.060
Yeah.

00:22:56.060 --> 00:22:57.940
A lot of hoops to jump through, right?

00:22:57.940 --> 00:22:58.860
But yeah.

00:22:58.860 --> 00:22:59.060
Okay.

00:22:59.060 --> 00:23:01.820
So it's, it's not just you who have found this place.

00:23:01.820 --> 00:23:04.120
There's this group of folks inside the company.

00:23:04.120 --> 00:23:16.020
And again, the transition within Microsoft over the last six years has gone to the point where we aren't forcing open source contributors to sit down with lawyers to describe what they're going to do ahead of time.

00:23:16.020 --> 00:23:17.380
That's really nice.

00:23:17.380 --> 00:23:19.600
It means we have a lot more contributors to projects.

00:23:19.600 --> 00:23:25.360
We're able to support and help develop projects in, in ways that benefit.

00:23:25.360 --> 00:23:26.920
Obviously they benefit us.

00:23:26.920 --> 00:23:31.620
Otherwise, you know, we wouldn't get work time to do it, but they, they benefit the community as well.

00:23:32.080 --> 00:23:44.600
So it's, we're trying to find that balance between kind of straight up altruistic development on open source projects and being able to resolve the issues that, that people are having that are kind of specific to us.

00:23:44.600 --> 00:23:44.920
Yeah.

00:23:44.920 --> 00:23:48.040
Well, like some of this, you know, it's kind of like basic scientific research.

00:23:48.040 --> 00:23:51.820
Like some of this comes back to help the world in nice ways.

00:23:51.820 --> 00:24:00.360
Like for example, the Git stuff that you all contributed back to Git so that you can move windows to Git because kind of like command line argument thing.

00:24:00.360 --> 00:24:08.960
Like the number of files, like the virtual, basically virtualizing the Git file system had to be done for windows to be part of Git, right?

00:24:08.960 --> 00:24:09.860
Or to be on Git.

00:24:09.860 --> 00:24:10.240
Yeah.

00:24:10.240 --> 00:24:17.640
We had a number of repositories internally that were just the working directory was being counted in gigabytes of code.

00:24:17.720 --> 00:24:18.900
And we had one.

00:24:18.900 --> 00:24:25.800
So one of the repositories I worked on through the transition from TFS to get the, like one checkout.

00:24:25.800 --> 00:24:32.000
When I first checked out that repository, it was something like 180 gigabytes, just the working directory.

00:24:32.000 --> 00:24:33.080
Just the main branch.

00:24:33.080 --> 00:24:34.000
Just the main branch.

00:24:34.000 --> 00:24:37.420
And not even the history, like just the latest up to date.

00:24:37.420 --> 00:24:39.320
We had a whole lot of stuff checked in that didn't need to be.

00:24:39.460 --> 00:24:40.060
Yeah, yeah, sure.

00:24:40.060 --> 00:24:46.320
Some massive work went on and I think we got it down to six gig and said, that is the best we can do.

00:24:46.320 --> 00:24:53.020
And part of that six gig is a script that downloads the other hundred gig of tools into a different directory.

00:24:53.020 --> 00:24:53.640
So.

00:24:53.640 --> 00:24:53.920
Wow.

00:24:53.920 --> 00:24:54.440
That's incredible.

00:24:54.440 --> 00:24:54.840
Yeah.

00:24:54.840 --> 00:25:02.040
There's some really, really huge code bases that a lot of the big corporations have inside that simply never get heard of outside.

00:25:02.360 --> 00:25:09.240
Like open source community run projects are never going to consider those cases because they sound utterly ridiculous.

00:25:09.240 --> 00:25:16.320
And yet all of these really big companies have nothing but utterly ridiculous cases.

00:25:16.320 --> 00:25:17.760
Yeah.

00:25:17.760 --> 00:25:20.520
And they've been around for so long and so many people have worked on it.

00:25:20.520 --> 00:25:20.820
Yeah.

00:25:20.820 --> 00:25:22.640
It's, it's pretty insane.

00:25:22.640 --> 00:25:26.000
Just the number of developers continuously contributing to that stuff.

00:25:26.380 --> 00:25:32.240
So I feel like one of the big turning points in Python's journey probably hinges around Azure.

00:25:32.240 --> 00:25:33.260
What do you think?

00:25:33.260 --> 00:25:45.520
Certainly Azure has created a place for Microsoft to really care about developers in, in a way that is kind of different from when it was, when Windows was the main thing.

00:25:45.520 --> 00:25:45.800
Right.

00:25:45.800 --> 00:25:52.980
Because it used to be, well, long as it works well on Windows and we get that app on Windows and then, you know, like that was the key.

00:25:53.680 --> 00:26:02.020
But it's hard to say, well, all the Python developers are going to come over and run on Windows potentially because maybe they're hosting on Linux or whatever.

00:26:02.020 --> 00:26:07.040
But, but now if they're all going to run on the internet, right?

00:26:07.040 --> 00:26:08.280
Exactly.

00:26:08.280 --> 00:26:12.040
If they're running in our data centers, then we're happy to have them.

00:26:12.040 --> 00:26:18.880
And, you know, we want to do whatever we can to roll out the red carpet for whichever developers want to come in and work with us.

00:26:18.880 --> 00:26:24.820
And it's so much less about the end user platform because that's essentially the browser.

00:26:24.820 --> 00:26:27.140
Now it's like everyone's hiding behind the browser.

00:26:27.140 --> 00:26:28.560
The browsers work everywhere.

00:26:28.560 --> 00:26:29.560
The browser and the phone.

00:26:29.560 --> 00:26:30.020
Yeah.

00:26:30.020 --> 00:26:30.460
Yeah.

00:26:30.800 --> 00:26:41.080
So, yeah, it really has created that opportunity to do production apps in Python, which, which I mean, Python was never really designed for rich user facing applications.

00:26:41.080 --> 00:26:43.560
That's never been one of its use cases.

00:26:43.560 --> 00:26:44.860
It can certainly do it.

00:26:44.860 --> 00:26:47.880
There's other languages that have been designed for that kind of thing.

00:26:47.880 --> 00:27:02.200
And so Python's place has always really been more developers and command line, happy people and adding the web to that and being able to run your code on Azure enables Python developers in a way that want to encourage.

00:27:02.200 --> 00:27:07.060
And I don't want to say it's forced us because most of us are happy about it.

00:27:07.060 --> 00:27:12.400
It's broken down the barriers or the walls to pursuing the stuff you'd like to do anyway, maybe.

00:27:12.400 --> 00:27:12.840
How's that?

00:27:12.840 --> 00:27:13.720
That's basically it.

00:27:13.720 --> 00:27:17.440
We have somewhere to put our Python developers that we want to be working with now.

00:27:17.440 --> 00:27:22.840
Which is come and run your stuff on Azure or, you know, multi-cloud, hybrid cloud, whatever it is.

00:27:22.840 --> 00:27:28.760
But we have somewhere for Python developers to run production code, which Windows never really had a place for before.

00:27:28.760 --> 00:27:29.480
Yeah, for sure.

00:27:29.480 --> 00:27:33.980
So when I go to Azure these days, I feel like it's like opening up an encyclopedia.

00:27:33.980 --> 00:27:36.360
It's just like there's so much stuff in here.

00:27:36.360 --> 00:27:37.280
What do I even do?

00:27:37.280 --> 00:27:39.180
How many services are there?

00:27:39.180 --> 00:27:52.980
This actually came up while I was writing the story because I have a mention in there of Laurent as well, who manages the Azure management SDK in Python, which basically has modules for every single Azure service there is.

00:27:52.980 --> 00:27:56.180
And when I drafted it, I'm like, it's probably around 50.

00:27:56.180 --> 00:27:57.100
I'll write around 50.

00:27:57.100 --> 00:28:00.260
And then at one point I went and checked and it's over 100.

00:28:00.260 --> 00:28:00.700
Wow.

00:28:00.760 --> 00:28:01.540
And so I checked again.

00:28:01.540 --> 00:28:02.840
I'm like, no, it's still over 100.

00:28:02.840 --> 00:28:05.960
So I asked him and he's like, yeah, it's definitely over 100.

00:28:05.960 --> 00:28:07.420
I don't even know what they all are.

00:28:07.420 --> 00:28:07.700
Yeah.

00:28:07.700 --> 00:28:10.000
I feel like there needs to be a song about it.

00:28:10.000 --> 00:28:13.440
Like there's that song that all the US kids learn in school for all the states.

00:28:13.440 --> 00:28:13.760
Yeah.

00:28:14.620 --> 00:28:20.220
Like we need that song for all the Azure services just so we can remember what's going on.

00:28:20.220 --> 00:28:22.280
So bizarre how music does that.

00:28:22.280 --> 00:28:22.540
Yeah.

00:28:22.540 --> 00:28:22.880
Yeah.

00:28:22.880 --> 00:28:28.960
No, in terms of getting started, what I think was a really good move, and I'm not sure who made it, but I'll go give them a high five at some point.

00:28:28.960 --> 00:28:32.500
There's basically a set of free services on Azure.

00:28:32.500 --> 00:28:37.480
There's like a free sign up, credit card verification or like .edu email address verification.

00:28:37.480 --> 00:28:39.420
And you get like a limited set.

00:28:40.020 --> 00:28:46.040
And the thing about it being a limited set is there's one obvious choice for whatever you want to do within that set.

00:28:46.040 --> 00:28:47.960
And so it's like, I want to push up a website.

00:28:47.960 --> 00:28:49.120
It's like, well, app services here.

00:28:49.120 --> 00:28:50.540
That's the obvious choice.

00:28:50.540 --> 00:28:58.620
You don't have six things that you could make work that are all subtly different and serve different use cases better, which, you know, you can grow up into.

00:28:58.620 --> 00:29:05.520
Once you're at the point where it's like, this is a really serious workload that my company or my users are depending on.

00:29:05.520 --> 00:29:09.080
Let me optimize it for certain use cases.

00:29:09.220 --> 00:29:10.060
It really just is.

00:29:10.060 --> 00:29:11.620
You get one choice.

00:29:11.620 --> 00:29:12.720
You know, you want a virtual machine?

00:29:12.720 --> 00:29:14.060
Have a virtual machine.

00:29:14.060 --> 00:29:23.760
Oh, you want to choose from the 50 different sized virtual machines and different drive options and different GPU options and different memory options.

00:29:23.760 --> 00:29:26.700
Step up to the paid thing for that.

00:29:26.700 --> 00:29:30.900
But having that kind of filter on this is what you need to just do something.

00:29:30.900 --> 00:29:36.180
Taking away that choice is one of those things that I think is actually a really good user experience at time.

00:29:36.400 --> 00:29:38.620
Yeah, sometimes less is more in that sense.

00:29:38.620 --> 00:29:40.800
That's pretty cool that the free tier starts that way.

00:29:40.800 --> 00:29:46.560
So let's spend just a little bit more time on this journey, maybe focus on the areas that you've worked on.

00:29:46.560 --> 00:29:55.280
I don't know how many listeners out there know, but you actually, you know, when you interact with Python on Windows, they are interacting with a lot of your work, right?

00:29:55.280 --> 00:30:01.140
Yeah, so I actually, the way I got into core development was I was at PyCon.

00:30:01.140 --> 00:30:09.000
I think I was at my second PyCon ever, PyCon US, that is, and had some annoyances with the installer for Windows.

00:30:09.000 --> 00:30:12.580
And so I basically found some of the core developers and said...

00:30:12.580 --> 00:30:16.600
It was cool that it was there, but it definitely was a little bit not...

00:30:16.600 --> 00:30:18.720
Tell us some of the challenges that you ran into because of it.

00:30:18.720 --> 00:30:20.620
It was a simple installer for a simpler time.

00:30:20.620 --> 00:30:29.560
Because there was a time on Windows where you just drop files anywhere and they run, and anyone who has physical access to the machine can do whatever they like.

00:30:30.020 --> 00:30:35.740
Then the internet happened, and it turns out that even people who don't have physical access to the machine have access to the machine.

00:30:35.740 --> 00:30:38.660
And we really ought to be locking things down a little bit better.

00:30:38.660 --> 00:30:57.100
And that sent off the whole security kick that eventually brought us to where we are today, where we get all the way to store packaged apps that are managed in a really, really, really secure and isolated way that didn't exist at the time when the previous Python installer was written.

00:30:57.100 --> 00:30:59.940
Which I think came straight out of the DOS days.

00:31:00.380 --> 00:31:05.940
So it absolutely served a very important purpose for making Python available on Windows.

00:31:05.940 --> 00:31:11.020
But it just hadn't grown up at the same rate as Windows was growing up.

00:31:11.020 --> 00:31:17.320
So I basically found some core developers and said, hey, I would like to contribute some work to improve this.

00:31:17.320 --> 00:31:18.680
Who do I talk to?

00:31:18.680 --> 00:31:22.600
How do I go about suggesting ideas, proposing the work, contributing the work?

00:31:22.600 --> 00:31:27.060
And they basically said, you know, give us your contact, we'll chat and we'll get back to you.

00:31:27.720 --> 00:31:36.680
And they got back to me the next day because it happened that the person who was currently maintaining Windows, Martin Von Lois, had decided to step away and they were looking for a replacement.

00:31:36.680 --> 00:31:37.700
Good timing.

00:31:37.760 --> 00:31:39.280
Very good timing.

00:31:39.280 --> 00:31:42.700
And so I was basically co-op.

00:31:42.700 --> 00:31:48.400
I'm one of the rare people who basically gets dropped into core development without having contributed.

00:31:48.400 --> 00:31:51.500
That's not the usual way people become core developers.

00:31:52.040 --> 00:31:57.200
But I feel like it turns out when you're very much in a minority like that.

00:31:57.200 --> 00:32:00.880
I'm such an affirmative action hire for the CPython core development team.

00:32:00.880 --> 00:32:05.220
It's not even funny because there's very few Windows developers there.

00:32:05.220 --> 00:32:11.180
And so if someone comes along and says, I would like to do Windows and I'll let you all worry about Linux.

00:32:11.180 --> 00:32:12.100
I don't care about it.

00:32:12.100 --> 00:32:13.020
I just want to do Windows.

00:32:13.020 --> 00:32:15.320
They're like, are you even real?

00:32:15.420 --> 00:32:15.900
Yeah.

00:32:15.900 --> 00:32:16.760
Who's this person?

00:32:16.760 --> 00:32:19.060
They're like, yes, please come and help.

00:32:19.060 --> 00:32:27.580
And so I basically just got to rewrite it as I wanted to do things like moving the default install so that it's not writable by anyone on the machine.

00:32:27.980 --> 00:32:38.480
Because it was installing and Python 2.7 still installs to this default directory where any user on the machine can add files that the administrator will then automatically run.

00:32:38.480 --> 00:32:41.980
And it's something out of place like c colon backslash Python, right?

00:32:41.980 --> 00:32:42.360
Yeah.

00:32:42.360 --> 00:32:43.880
It's unconventional now.

00:32:43.880 --> 00:32:45.520
At the time, it certainly wasn't.

00:32:45.520 --> 00:32:47.440
It made a total sense originally.

00:32:47.440 --> 00:32:54.840
But these days, yeah, programs go into program files or they go under the user app directory if it's restricted to the user.

00:32:55.080 --> 00:32:57.720
And that helps keep things secured.

00:32:57.720 --> 00:33:02.420
It sets the access control right so that people aren't modifying files that other people are going to use.

00:33:02.420 --> 00:33:10.580
And being able to put it in a per user location means that we don't have to require users to be administrator to install anymore.

00:33:10.580 --> 00:33:18.900
And I feel like that's been the most exciting feature about the new installer for people is enabling that don't make me elevate.

00:33:18.900 --> 00:33:21.720
For a lot of people, it's don't make me hit yes on that box.

00:33:21.720 --> 00:33:25.040
But for others, it's don't make me go get the administrator to type in their password.

00:33:25.040 --> 00:33:26.420
Yeah, exactly.

00:33:26.420 --> 00:33:30.420
When you're at home and it means the UAC kicks in and you click OK, like that's OK.

00:33:30.420 --> 00:33:32.800
That's I mean, it's not super, but it's not the end of the world.

00:33:32.800 --> 00:33:41.900
But if you're at a company, like some of the places where it would make a lot of sense at like investment places and whatnot, like they don't even get to ask.

00:33:41.900 --> 00:33:43.080
It's just the answer is no.

00:33:43.080 --> 00:33:43.660
All right.

00:33:43.660 --> 00:33:44.740
No, you don't get to install stuff.

00:33:44.880 --> 00:33:46.080
No, you don't get to raise permissions.

00:33:46.080 --> 00:33:46.300
No.

00:33:46.300 --> 00:33:46.940
Yeah.

00:33:46.940 --> 00:33:52.900
And then pip install something becomes challenging as well if it's trying to install into like restricted directories.

00:33:52.900 --> 00:34:00.420
And so you basically made it so you can all that stuff potentially happens into your user profile where you can control almost all the time.

00:34:00.540 --> 00:34:09.700
Yeah. And at this point with 3.7, I made the final change, which was to do with the C runtime that would still try and install as an admin if you needed it.

00:34:09.700 --> 00:34:12.380
We're down to very few people who needed that already.

00:34:12.560 --> 00:34:24.880
So 3.7, I just kind of cut that cord completely and said, if you're still running on a Windows 7 machine that isn't receiving updates for whatever reason, then we won't try and install this properly.

00:34:24.880 --> 00:34:30.640
We'll just give it to you so it works and assume that you're not ever going to run into the trouble that we were concerned about previously.

00:34:30.980 --> 00:34:40.980
But I think my favorite story about people enjoying this feature, I had a teacher who spends a lot of time teaching Python throughout Africa come up to me.

00:34:40.980 --> 00:34:50.940
It was actually at Europython this year and absolutely gushing with praise and thanks for making it possible for his students who are coming into like evening classes.

00:34:50.940 --> 00:34:52.900
They've borrowed their mom's work laptop.

00:34:52.900 --> 00:34:54.840
It's a 10-year-old laptop.

00:34:54.840 --> 00:34:55.780
The keyboard's broken.

00:34:55.960 --> 00:35:03.700
So they've got a USB keyboard plugged into this laptop because they don't get to buy the latest MacBook Pro every year, right?

00:35:03.700 --> 00:35:04.460
Yeah, of course.

00:35:04.460 --> 00:35:09.360
And it's running Windows 7 and it's totally locked down and they can still install Python on it.

00:35:09.360 --> 00:35:09.860
That's great.

00:35:09.860 --> 00:35:10.880
Because of that change.

00:35:10.880 --> 00:35:20.840
It's a little mind-blowing that that change, which feels like it's technically a good thing to do, but it also enables kids in Africa to learn to program.

00:35:20.840 --> 00:35:22.420
And that's just mind-blowing.

00:35:22.420 --> 00:35:23.640
Yeah, that is so awesome.

00:35:23.640 --> 00:35:24.360
That's great.

00:35:24.360 --> 00:35:31.460
And I think actually that sort of speaks a little bit to the reach of talking about Python on Windows and this journey.

00:35:31.460 --> 00:35:38.700
I mean, we talked about, like I talked about, say, oh, Azure is what's like unlocking a lot of the sort of commercial side of the things around Python for you.

00:35:38.700 --> 00:35:47.340
But if you look at who is using Python, like what operating system people are using Python on, you go to the conference, maybe it doesn't feel this way.

00:35:47.440 --> 00:35:53.340
But general studies and surveys say the majority of them is like massively on Windows, right?

00:35:53.340 --> 00:36:02.340
I've actually been going around speaking to this at conferences recently, which I think we'll have a link to at least one of the recordings of the talks in the notes for this.

00:36:02.500 --> 00:36:16.700
But my recent presentation I've been giving at conferences is called Python on Windows is OK, actually, which is a bit of a bait and switch title, to be honest, because most of the talk is not actually about Python on Windows because Python on Windows is OK.

00:36:17.360 --> 00:36:20.740
And one of the best things when I joined the team was it was already OK.

00:36:20.740 --> 00:36:23.120
I think this installer actually makes a big difference.

00:36:23.120 --> 00:36:27.020
I think getting it installed properly is a big deal.

00:36:27.020 --> 00:36:28.820
It's the first experience, right?

00:36:28.820 --> 00:36:33.720
And the first experience matters because if you have a bad first experience, then you may never look twice.

00:36:33.720 --> 00:36:47.360
The big kind of controversial thing in that talk and a lot of the reason that I wanted to give it was I started looking at the stats and, you know, I looked around the conferences and was like, yeah, the entire Python world is using Macs mostly.

00:36:47.360 --> 00:36:51.980
And occasionally people that have taken a Windows laptop and installed Linux on it instead.

00:36:51.980 --> 00:36:58.500
And then I'd get up to give a talk and I've got a Windows machine and I can look out and just see all these glowing fruits staring back at me.

00:36:58.500 --> 00:36:59.940
Taunting you.

00:37:00.060 --> 00:37:15.860
Yeah. And but then we start looking at actual numbers and it's so skewed in the other direction, looking at things through downloads from PyPI, downloads from Conda, usage in the PSF survey, usage in IDEs like VS Code and PyCharm.

00:37:15.860 --> 00:37:17.580
So PyPI accepted.

00:37:17.580 --> 00:37:24.000
Windows is a huge, if not the biggest chunk or just a true majority of Python usage.

00:37:24.000 --> 00:37:26.280
Yeah. So let me just just to put some numbers behind this.

00:37:26.320 --> 00:37:36.760
Let me just read the operating system usage numbers from the JetBrains and PSF survey, the one that you just mentioned, which was not promoted by JetBrains.

00:37:36.760 --> 00:37:38.080
They just happened to do some of the analysis.

00:37:38.080 --> 00:37:40.560
So promoted by PSF.

00:37:40.560 --> 00:37:44.420
We've got other operating systems, whatever that means, at 17 percent.

00:37:44.420 --> 00:37:47.880
We have macOS at 15 percent and Linux at 19 percent.

00:37:47.880 --> 00:37:48.320
Right.

00:37:48.400 --> 00:37:53.140
I feel like everyone has a Mac at the conferences, but 49 percent people said Windows.

00:37:53.140 --> 00:37:56.580
Right. That's like not just a little bit more than others.

00:37:56.580 --> 00:38:00.240
That's more than Mac and Linux combined.

00:38:00.240 --> 00:38:04.460
Yeah. And as you say, it was not promoted by JetBrains so much.

00:38:04.460 --> 00:38:07.340
It was like this was promoted in regular Python channels.

00:38:07.340 --> 00:38:19.280
And so regular Python users are seeing and responding to this and doing it from a Windows machine, which we just don't have that kind of perception in the community that that's the case.

00:38:19.440 --> 00:38:21.140
And so it was really exciting.

00:38:21.140 --> 00:38:28.520
Like my favorite part about giving talks in general is looking at the tweets afterwards and the things that people have been most shocked by or most excited by.

00:38:28.960 --> 00:38:39.900
And the amount of photos of me standing in front of the slide with the 14 million downloads per month of Python for Windows is one of my favorite photos now.

00:38:39.900 --> 00:38:41.460
That's a ridiculous number of downloads.

00:38:41.460 --> 00:38:41.940
That's awesome.

00:38:41.940 --> 00:38:42.860
It's huge.

00:38:42.860 --> 00:38:46.180
And a lot of that is going to be CI systems, to be fair.

00:38:46.180 --> 00:38:49.300
But most of the downloads from Python.org are CI systems.

00:38:49.300 --> 00:38:55.120
We don't actually think there are 14 million Python developers anywhere in the world right now.

00:38:55.120 --> 00:38:57.220
I think the estimates are closer to six to seven million.

00:38:57.680 --> 00:38:59.240
So there's automatic downloads.

00:38:59.240 --> 00:39:04.140
At the same time, if people are automatically downloading Python for Windows, that means they're running Python code on Windows.

00:39:04.140 --> 00:39:04.440
Yeah.

00:39:04.440 --> 00:39:10.160
So it doesn't actually spoil the point at all for it to be, you know, oh, it was a script.

00:39:10.160 --> 00:39:13.400
It's like, well, every single download of the source for Linux was a script.

00:39:13.400 --> 00:39:17.400
Every single download of the Mac installer was also a script.

00:39:17.400 --> 00:39:26.400
That's just the reality is no matter how you slice the numbers, I describe it as at least half of the Python community is actually running on Windows.

00:39:26.400 --> 00:39:30.100
And we just don't realize that.

00:39:30.100 --> 00:39:32.140
And a lot of time we don't act like it.

00:39:32.140 --> 00:39:44.600
So the rest of the talk is basically around things that Python projects or Python packages do or have done that make them not work well on Windows.

00:39:44.600 --> 00:39:45.080
Likewise.

00:39:45.300 --> 00:39:50.300
Things like using string dot split on forward slash to separate paths.

00:39:50.300 --> 00:39:52.440
So paths on Windows use backslashes.

00:39:52.440 --> 00:39:55.200
And we have a couple of great modules for this.

00:39:55.200 --> 00:39:59.160
So I strongly recommend pathlib, which is in the standard library as of 3.4.

00:39:59.160 --> 00:40:02.940
But we've had OS dot path module for dealing with paths.

00:40:03.160 --> 00:40:07.840
And yet people will still just use the normal string functions to split on a forward slash.

00:40:07.840 --> 00:40:14.500
And if you do that to a path on Windows, you get one element with the entire path still in it because there's no forward slashes anywhere.

00:40:14.500 --> 00:40:15.760
It's a weird root directory.

00:40:15.760 --> 00:40:16.860
And it's things like that.

00:40:16.860 --> 00:40:24.880
And my big concern with that is you get, you know, these 50% of Python developers will install your package, try and use it, and it won't work.

00:40:25.580 --> 00:40:28.220
And it's just a very unwelcoming experience.

00:40:28.220 --> 00:40:29.480
Yeah, it can be.

00:40:29.480 --> 00:40:31.500
And, you know, it can be indirect as well.

00:40:31.500 --> 00:40:35.540
Like, for example, Sanic is a cool async web framework, right?

00:40:35.540 --> 00:40:35.960
Python.

00:40:35.960 --> 00:40:43.180
But I believe right now, at least last I heard, it didn't work on Windows because UVlib doesn't work on Windows.

00:40:43.180 --> 00:40:45.040
It doesn't install properly on Windows.

00:40:45.040 --> 00:40:47.060
And there's just these little glitches just like, ah.

00:40:47.060 --> 00:40:47.400
Yeah.

00:40:47.400 --> 00:40:55.560
And there's always a very long and reasonable chain of things that would have to work perfectly for the final result to work, which I understand.

00:40:55.560 --> 00:40:55.800
Yeah.

00:40:55.800 --> 00:40:56.480
Which I understand.

00:40:56.480 --> 00:41:01.780
This portion of Talk Python To Me is brought to you by us.

00:41:01.780 --> 00:41:06.040
Have you heard that Python is not good for concurrent programming problems?

00:41:06.040 --> 00:41:11.620
Whoever told you that is living in the past because it's prime time for Python's asynchronous features.

00:41:11.620 --> 00:41:21.300
With the widespread adoption of async methods and the async and await keywords, Python's ecosystem has a ton of new and exciting frameworks based on async and await.

00:41:21.400 --> 00:41:26.440
That's why we created a course for anyone who wants to learn all of Python's async capabilities.

00:41:26.440 --> 00:41:28.860
Async techniques and examples in Python.

00:41:28.860 --> 00:41:34.740
Just visit talkpython.fm/async and watch the intro video to see if this course is for you.

00:41:34.740 --> 00:41:37.220
It's only $49 and you own it forever.

00:41:37.220 --> 00:41:38.260
No subscriptions.

00:41:38.260 --> 00:41:41.020
And there are discounts for teams as well.

00:41:42.940 --> 00:41:53.900
Part of the problem is that of those 50% of Python users on Windows, I'm pretty well convinced that 49% of them work at places where they're not actually allowed to release their libraries.

00:41:53.900 --> 00:41:54.280
Yeah.

00:41:54.400 --> 00:42:05.900
I know for a fact that they have fast networking libraries, that they've re-implemented things for their platforms that make Python run great, that would be amazing to have out there in the community.

00:42:05.900 --> 00:42:13.480
That'd be a real enabler for a lot of the rest of us that are not kind of restricted to within these companies, but they just can't do it.

00:42:13.480 --> 00:42:22.220
Yeah, it's probably built to drop a half a millisecond on a trading engine for algorithmic trading or something like this where it's like, well, this is our advantage.

00:42:22.220 --> 00:42:23.140
We're not putting this out.

00:42:23.140 --> 00:42:23.780
Yeah, absolutely.

00:42:23.780 --> 00:42:29.200
And that advantage is worth a few millions of dollars per second as well for a lot of these places.

00:42:29.200 --> 00:42:33.080
Yeah, it's not unreasonable for them from a self-interested perspective.

00:42:33.080 --> 00:42:34.280
But yeah, I hear you.

00:42:34.280 --> 00:42:52.280
I would actually say it in reverse, maybe, like how many of the core developers or maybe not even the core developers, let's just say people working on popular packages like UVlib, not calling them out in particular, really, but just those types of things where the developer doesn't even own a Windows machine.

00:42:52.280 --> 00:42:54.160
So it just doesn't bother.

00:42:54.160 --> 00:43:03.360
And ultimately, I think that's the biggest point is when you don't have the machine to run and test on, then you stop thinking about these things.

00:43:03.360 --> 00:43:17.560
And works on my machine is, for volunteers, I think totally reasonable by asking people to invest in new hardware, invest in a new software installation, especially the matrix of things that you have to worry about.

00:43:17.560 --> 00:43:24.380
Even for Linux, I would suspect that most Python packages are not tested on more than one variation of Linux.

00:43:24.380 --> 00:43:25.680
Yeah, for sure.

00:43:25.680 --> 00:43:26.780
For exactly the same reason.

00:43:26.780 --> 00:43:34.640
Like no one, very few people are going to be running on FreeBSD and Ubuntu, for example, to make sure that it works on both.

00:43:34.640 --> 00:43:38.840
So that is a really valid reason for not doing it.

00:43:38.840 --> 00:43:47.020
But at the same time, it doesn't take a lot of work these days to set up a CI system that is going to run on Windows for you.

00:43:47.140 --> 00:43:48.880
And it doesn't actually take any money at all.

00:43:48.880 --> 00:43:58.400
Azure Pipelines is a fairly recent one that a lot of people are really excited about, myself included, because the open source thing.

00:43:58.400 --> 00:44:10.380
So if you mark your project as public so that anyone can see the results, you automatically get unlimited build time, up to 10 concurrent builds, and Windows, Mac, Linux, a choice.

00:44:10.380 --> 00:44:15.940
So you can choose, I think there's two Ubuntu images and two Windows images right now.

00:44:15.940 --> 00:44:21.920
It has Docker support if you want to be running builds inside containers of whatever container you like.

00:44:21.920 --> 00:44:22.480
That's pretty cool.

00:44:22.480 --> 00:44:24.600
Can I plug it into like my GitHub repo?

00:44:24.800 --> 00:44:25.420
Yeah, absolutely.

00:44:25.420 --> 00:44:27.380
You could do that before we bought GitHub.

00:44:27.380 --> 00:44:30.560
In fact, that's always been one of the primary uses for it.

00:44:30.560 --> 00:44:31.860
Yeah, that's pretty awesome.

00:44:31.860 --> 00:44:34.520
We have CPython builds running on that.

00:44:34.520 --> 00:44:37.180
Those are not the binding required checks yet.

00:44:37.180 --> 00:44:39.800
Pandas has switched over most recently.

00:44:39.800 --> 00:44:41.340
NumPy is there.

00:44:41.540 --> 00:44:45.740
Condor Forge is investigating moving all of their builds over to it.

00:44:45.740 --> 00:44:46.820
Oh, that would be huge.

00:44:46.820 --> 00:44:50.900
I mean, I know some people know what Condor Forge is, but maybe not everyone.

00:44:50.900 --> 00:45:01.540
And I think they're really interesting in enabling Windows to not have these problems of like building these weird tools because they deliver binaries on the right OS.

00:45:01.540 --> 00:45:05.040
Maybe tell people real quick what Condor Forge is so they'll appreciate that if they don't know.

00:45:05.040 --> 00:45:10.360
So Anaconda is a distribution and normally how people think about getting to the Condor tool.

00:45:10.520 --> 00:45:15.320
So you'll get this big fat installer from Anaconda and install that and get a whole lot of packages.

00:45:15.320 --> 00:45:18.820
And you have this Condor tool for creating environments, installing packages.

00:45:18.820 --> 00:45:20.880
But Condor is really just a pip equivalent.

00:45:20.880 --> 00:45:28.320
And you can, rather than getting the packages from the Anaconda channel, you could get them from other people's channels as well.

00:45:28.320 --> 00:45:34.420
Condor Forge is a community organization that basically fills in the gaps.

00:45:34.420 --> 00:45:42.220
Like they have something like 5,000 GitHub repositories that are basically the build steps for 5,000 packages.

00:45:42.920 --> 00:45:49.760
And they have a set of packages that are built for, I think they're doing five different platforms now.

00:45:49.760 --> 00:45:53.420
Just automatically building all of these packages.

00:45:53.420 --> 00:46:10.440
So if you've ever run into like trying to pip install something and there's no wheel, Condor Forge has probably done some of the work or someone's contributed the steps to build the pre-built Condor package so that you can Condor install a whole lot of stuff from there that would otherwise not be available.

00:46:11.160 --> 00:46:24.560
And they've been doing that through this incredible system of basically abusing as many free CI providers as possible for the various Mac and Windows and Linux builds.

00:46:24.560 --> 00:46:26.840
We're doing it for the good of the community.

00:46:26.840 --> 00:46:28.380
Yeah.

00:46:28.380 --> 00:46:33.340
And I think we timed things right when I mentioned this to one of the Azure Pipelines team members.

00:46:34.160 --> 00:46:39.740
And I don't actually know exactly what our financial motivation would be, but I don't really care.

00:46:39.740 --> 00:46:51.560
If they're excited to move as many thousands of builds onto one system that does everything that they want and the Condor Forge team is excited to do that and everyone's just excited to do it.

00:46:51.560 --> 00:46:57.560
I don't actually know what anyone's getting out of it, but Condor Forge is getting builds, users are getting packages.

00:46:57.560 --> 00:46:58.440
And so I'm happy.

00:46:58.440 --> 00:47:06.780
It can at least make the story for installing some of these packages on Windows better by ensuring they have a good infrastructure and it's not too much work, right?

00:47:06.780 --> 00:47:08.380
So there's maybe something there, yeah?

00:47:08.380 --> 00:47:11.420
And it makes installing the packages on Mac and Linux better as well.

00:47:11.720 --> 00:47:17.420
Anaconda's popular on Windows, but not like 90% of their usage.

00:47:17.420 --> 00:47:18.560
Yeah, yeah, it's true.

00:47:18.560 --> 00:47:30.540
They still have really significant use on Linux and Mac because they'll give you far more reproducible installs across various platforms than using Building From Source or even Wheels on pip are going to give you.

00:47:30.540 --> 00:47:34.360
Sure, I've used it on my macOS because something wasn't installing right.

00:47:34.360 --> 00:47:36.580
I'm just like, I'm just going to get the pre-built version.

00:47:36.700 --> 00:47:40.740
Let me ask you a question about something that's bugged me for a while.

00:47:40.740 --> 00:47:46.820
And as somebody who creates courses and tutorials, I often have to have like a couple of steps.

00:47:46.820 --> 00:47:55.380
Like, okay, here's the command that you type when you register this package so you can develop it in development mode on Mac or Linux.

00:47:55.380 --> 00:47:59.820
And, oh, here's the separate command that you run on Windows so that you can do that.

00:47:59.820 --> 00:48:03.900
So, like, Python 3 set up PY develop, right, for example.

00:48:03.900 --> 00:48:06.340
But, well, Python 3 was not a command on Windows.

00:48:06.340 --> 00:48:08.040
And that can be super annoying.

00:48:08.040 --> 00:48:14.280
You're like, well, on Windows, what you do is you say Python, but you've got to make sure which Python it is in the right path so that Python 3 comes first.

00:48:14.280 --> 00:48:16.940
And you and I, we've had these conversations before, right?

00:48:16.940 --> 00:48:17.280
Yeah.

00:48:17.280 --> 00:48:19.080
So this, I think, is one of those things.

00:48:19.080 --> 00:48:23.560
But you actually have like a super early announcement of something that's amazing around this.

00:48:23.560 --> 00:48:24.100
Is that right?

00:48:24.100 --> 00:48:30.660
It's not exactly a secret project, but it's, you know, if you're not watching the Python bug tracker, then you won't have noticed.

00:48:30.660 --> 00:48:33.620
But hopefully when we start promoting this a little bit more, you'll see it.

00:48:33.620 --> 00:48:40.040
With Windows 10, we actually added, so Microsoft added a store, an app store, basically.

00:48:40.040 --> 00:48:41.440
Oh, and in fact, it came in Windows 8.

00:48:41.440 --> 00:48:47.520
But in Windows 10, some of the recent updates, it's been growing the ability to install regular old apps.

00:48:47.880 --> 00:48:56.700
So a lot of people think of app stores like, you know, you get your Apple approved iPhone app that only runs on the phone and can't actually do interesting stuff beyond what the app does.

00:48:56.700 --> 00:49:02.380
Like you can't install a full Python interpreter that can access everything on your system through any of the app stores.

00:49:02.380 --> 00:49:08.820
What's being added to the Microsoft app store, the Windows app store, is the ability to install those.

00:49:08.820 --> 00:49:12.940
So paint.net is actually one of the earlier ones to do this.

00:49:13.060 --> 00:49:17.780
You can install paint.net through the store and it installs really quickly.

00:49:17.780 --> 00:49:20.240
It's properly isolated between users.

00:49:20.240 --> 00:49:29.120
And what we're doing, or at least experimenting with at this point, is adding Python to that store so that when it comes out,

00:49:29.120 --> 00:49:34.200
and the aim is to do this with Python 3.7.2 initially and then keep updating it from there,

00:49:34.800 --> 00:49:41.240
is be able to, rather than going to python.org and downloading the installer and running through all of those steps,

00:49:41.240 --> 00:49:45.320
you'll be able to just go to the Microsoft store and say, get it free.

00:49:45.320 --> 00:49:48.200
Obviously, it's going to be a free app and install it.

00:49:48.200 --> 00:49:53.720
And that will give you Python on your machine, managed by the store, automatic updates.

00:49:53.720 --> 00:49:54.760
It'll do automatic updates.

00:49:54.760 --> 00:49:55.580
Beautiful, yeah.

00:49:55.580 --> 00:49:56.680
It'll do automatic updates.

00:49:56.680 --> 00:49:58.460
I don't know how much trouble that's going to cause yet.

00:49:58.460 --> 00:50:00.240
That's why this is still a little bit experimental.

00:50:00.240 --> 00:50:08.900
I had to make some significant improvements to VN to make it be able to handle that, but I believe it can handle that now.

00:50:08.900 --> 00:50:09.500
Oh, that's great.

00:50:09.500 --> 00:50:17.220
But the really cool thing that comes with store apps that isn't available to regular apps is being able to manage path properly.

00:50:17.220 --> 00:50:25.100
So one of the problems with the path environment variable on Windows, or just one of the differences from POSIX-like systems,

00:50:25.100 --> 00:50:31.180
which I actually talk about in my Python Windows talk, is that on POSIX-like systems, you kind of have,

00:50:31.180 --> 00:50:36.980
here's a directory for my user commands, and any commands I put in there are available to the current user.

00:50:36.980 --> 00:50:41.240
Here's a directory for my machine commands, and any commands in there are available for the whole machine.

00:50:41.780 --> 00:50:42.680
And Windows hasn't had that.

00:50:42.680 --> 00:50:49.620
Windows has, let me add a directory to the path, and then go look in that when you're searching for anything.

00:50:49.620 --> 00:50:53.360
And so you get apps fighting over who's first on the path.

00:50:53.360 --> 00:50:59.380
There's no good way to automatically manage it and say, well, 3.7 should come before 3.6.

00:50:59.380 --> 00:51:01.480
But you can't automatically do that.

00:51:01.480 --> 00:51:06.160
And so that's always been messy, to the point where that's one of the controversial things I did with the new installer,

00:51:06.160 --> 00:51:11.660
was disable setting that up by default, because it just leads to breakage and confusion that we can't manage.

00:51:11.660 --> 00:51:18.280
What the store apps let us do is specify aliases for these commands.

00:51:18.280 --> 00:51:20.980
And those get managed by the operating system.

00:51:20.980 --> 00:51:24.940
There's like a nice GUI for switching which app gets to have that alias.

00:51:24.940 --> 00:51:30.140
And they go in a proper directory, that is, here are my user commands that are available to my user,

00:51:30.140 --> 00:51:32.100
and it's on path automatically.

00:51:32.100 --> 00:51:32.940
That's awesome.

00:51:32.940 --> 00:51:37.740
So this store package, when it comes out, is going to have python.exe on path,

00:51:37.740 --> 00:51:41.600
managed properly so that you'll get that one, you'll get the right one.

00:51:41.960 --> 00:51:47.680
And then when you update the store package, you'll get the latest version because it goes in the same place,

00:51:47.680 --> 00:51:50.360
and the newer version wins over the older version.

00:51:50.360 --> 00:51:55.940
But because it's so cheap and easy to do this, I thought, hey, why not put python3.exe in there?

00:51:55.940 --> 00:52:00.120
Well, since we're here, why not put 3.7.exe in there as well?

00:52:00.600 --> 00:52:03.240
So now you have all of those on path with this package.

00:52:03.240 --> 00:52:05.560
So that python3 command is going to work.

00:52:05.560 --> 00:52:08.280
And then I thought, well, this is still pretty easy.

00:52:08.280 --> 00:52:09.140
Why not keep going?

00:52:09.140 --> 00:52:10.420
Let's put pip in there.

00:52:10.420 --> 00:52:11.920
Let's put pip3 in there.

00:52:11.920 --> 00:52:13.380
Let's put pip3.7 in there.

00:52:13.380 --> 00:52:14.660
Let's put idle in there.

00:52:15.100 --> 00:52:20.580
And so it's very easily got to a point where when you install this package,

00:52:20.580 --> 00:52:25.680
you can go to any command prompt and type idle, and it will start running idle in that directory.

00:52:25.680 --> 00:52:30.520
You can type, when you end up with multiple ones, idle 3.7 will give you the 3.7,

00:52:30.640 --> 00:52:34.120
and eventually idle 3.8 will give you the later version of it.

00:52:34.120 --> 00:52:35.500
It's just there.

00:52:35.500 --> 00:52:36.660
That is such good news.

00:52:36.660 --> 00:52:37.640
Thank you.

00:52:37.640 --> 00:52:38.660
Thank you for doing that.

00:52:38.660 --> 00:52:39.880
That is really, really cool.

00:52:39.880 --> 00:52:41.300
So excited to hear it.

00:52:41.300 --> 00:52:43.700
Today, as we're recording, the first preview one is up.

00:52:43.700 --> 00:52:46.880
And so there's some more kind of private testing going on.

00:52:46.880 --> 00:52:53.720
But my hope is that with the 3.7.2 RC, when that is out, which is probably a week or two ago,

00:52:53.720 --> 00:52:56.300
by the time you're listening to this, it'll be up there publicly.

00:52:56.300 --> 00:52:58.780
So you can just go to the store and type python,

00:52:58.780 --> 00:53:01.080
and we should be showing up somewhere near the top of the list.

00:53:01.080 --> 00:53:06.120
And you can use that to install Python on your Windows box.

00:53:06.120 --> 00:53:08.160
And the cool thing is that doesn't require admin,

00:53:08.160 --> 00:53:11.500
but it installs in a way that only installs it once for the whole machine,

00:53:11.500 --> 00:53:12.360
which is really nice.

00:53:12.360 --> 00:53:16.820
So if you have 100 users on a machine, it installs it once

00:53:16.820 --> 00:53:20.440
and then isolates just the differences per user.

00:53:20.440 --> 00:53:24.180
So it doesn't take up 100 times the disk space or anything crazy like that.

00:53:24.180 --> 00:53:27.700
That is so cool because you see so many tutorials and stuff

00:53:27.700 --> 00:53:32.060
where people just put the POSIX variant up there, right?

00:53:32.060 --> 00:53:38.220
The type pip3 here or, you know, you do this and it's like, well, that's not exactly what you need.

00:53:38.220 --> 00:53:44.460
So I think this, so you're thinking that this Windows 10 store app of Python 3

00:53:44.460 --> 00:53:47.820
is going to be the way to get Python going forward.

00:53:47.940 --> 00:53:48.380
I hope so.

00:53:48.380 --> 00:53:51.560
There's certainly going to be some cases where it won't be the best option.

00:53:51.560 --> 00:53:54.660
And so the old installers aren't going to go away anytime soon.

00:53:54.660 --> 00:53:55.000
Sure.

00:53:55.000 --> 00:53:58.260
But we'll just have to see how people use it and what people need.

00:53:58.260 --> 00:54:00.980
There's already cases where the existing installers aren't ideal.

00:54:00.980 --> 00:54:04.280
And this is why we have other distros out there.

00:54:04.280 --> 00:54:06.940
This is why WinPython exists, Anaconda exists,

00:54:06.940 --> 00:54:10.500
because they offer things that, you know, Python.org can't offer.

00:54:10.580 --> 00:54:12.440
So this is another thing to be offered.

00:54:12.440 --> 00:54:15.380
And we can track usage.

00:54:15.380 --> 00:54:17.740
One of the really nice things is we can get all the crash reports.

00:54:17.740 --> 00:54:23.860
So when Python is breaking badly, there's a dashboard for store apps that basically says,

00:54:23.860 --> 00:54:26.260
here's how reliable your app is over time.

00:54:26.260 --> 00:54:29.920
And so we'll start getting those reports for Python, which will be really interesting.

00:54:29.920 --> 00:54:30.840
Yeah, for sure.

00:54:30.840 --> 00:54:34.300
And you also were able to recompile Python with the later tools,

00:54:34.300 --> 00:54:39.280
like move seven years or five years into the future on the compilers, right?

00:54:39.280 --> 00:54:41.780
As you, as part of your project of taking over all this.

00:54:41.780 --> 00:54:47.120
Yeah, that was, that was actually one of the first things I did was let's move off Visual Studio 2010

00:54:47.120 --> 00:54:50.940
since it's now 2015 or whenever I started doing that.

00:54:50.940 --> 00:54:51.480
Exactly.

00:54:51.480 --> 00:54:54.140
I've had a number of people thank me profusely for that.

00:54:54.140 --> 00:54:57.160
Mostly other people who are trying to build Python itself.

00:54:57.160 --> 00:54:59.240
Yeah, because if you're on Windows and you're a developer,

00:54:59.520 --> 00:55:04.080
you probably have the latest version of Visual Studio or the Windows Platform SDK or whatever,

00:55:04.080 --> 00:55:09.220
but none of those are likely like a seven or eight year old version of the compiler.

00:55:09.220 --> 00:55:12.560
It gets harder and harder over time to get the old compilers.

00:55:12.560 --> 00:55:18.340
Microsoft supports the latest and try to build in the compatibility that makes it

00:55:18.340 --> 00:55:21.160
viable for people to move forward as soon as possible.

00:55:21.160 --> 00:55:27.000
Around the Visual Studio 2015 point is where we actually made some big dramatic changes to make that easier.

00:55:27.520 --> 00:55:36.020
And so we are at a point now where VS 2015 or 2017 or 2019, which is recently first preview was released,

00:55:36.020 --> 00:55:38.080
will all work.

00:55:38.480 --> 00:55:45.100
and they can all kind of work across each other so that even Python 3.5, which was compiled with the old one,

00:55:45.100 --> 00:55:48.940
can use extensions built with the newest compiler because we've got that.

00:55:48.940 --> 00:55:50.700
I think we finally got the compatibility right.

00:55:50.700 --> 00:55:53.900
We went through about six or seven different ideas on how to make it right.

00:55:54.620 --> 00:56:05.260
And this time I think we've got it in a way that really just helps developers not have to pull all sorts of crazy tricks with their setup to make things work.

00:56:05.260 --> 00:56:06.380
That's also really good news.

00:56:06.380 --> 00:56:09.460
So I think there's so many positive things around here.

00:56:09.460 --> 00:56:11.480
This Python on Windows is okay, actually.

00:56:11.480 --> 00:56:15.200
I mean, it did highlight some of the issues, but also some of the great things.

00:56:16.060 --> 00:56:24.480
So maybe let's close out this conversation around the journey of Python at Microsoft by talking about this Python Summit at Microsoft.

00:56:24.480 --> 00:56:26.140
What's the story on this thing?

00:56:26.140 --> 00:56:27.160
I've only just barely heard of it.

00:56:27.240 --> 00:56:32.340
There's rumors floating around, and we've put up a blog post with a bit of information.

00:56:32.340 --> 00:56:40.120
Being at Microsoft gives me an opportunity to speak to and hear from people at companies that don't normally interact with the community.

00:56:40.120 --> 00:56:52.020
And so through that, we get to hear about problems and solutions that various people have tried for fixing things in Python or just in their Python projects.

00:56:52.020 --> 00:56:55.980
And we start hearing some commonalities across a lot of different places.

00:56:55.980 --> 00:57:04.380
And I'm talking like the Facebooks, Instagrams, through to various big banks who are all heavily invested in Python.

00:57:04.380 --> 00:57:10.180
We start hearing so much in common, but also not actually being aware of everyone else.

00:57:10.180 --> 00:57:18.880
And we started feeling like, hey, maybe we're in the privileged place where we can see all the overlap, and maybe it's on us to do something about it.

00:57:18.880 --> 00:57:25.040
So we basically put out the call to the people that we knew in these places.

00:57:25.040 --> 00:57:29.200
And so this wasn't like a public open call for content.

00:57:29.440 --> 00:57:41.920
This was specific invites to companies we know heavily invested in Python and say, hey, come, let's get in a room together and basically talk through what our struggles have been, what our efforts look like.

00:57:42.220 --> 00:57:46.240
You know, it's like half conference, half counseling session almost.

00:57:46.240 --> 00:57:46.520
Yeah.

00:57:46.520 --> 00:57:55.200
And, you know, do it in a place where we can have kind of an informal NDA going on about, you know, this is not, we're not broadcasting this on the internet.

00:57:55.200 --> 00:58:01.140
So if you want to actually talk about problems that you're having, you don't have to worry that it's going to be a news headline tomorrow.

00:58:01.140 --> 00:58:16.420
And people came and we had a room filled with trillions of dollars worth of Python development, which is something that probably makes absolutely no sense to most of the community because we look around and see the same kind of Python developer all around us at most of the conferences.

00:58:16.980 --> 00:58:42.840
And then in this room, we had the Dropboxes, the Amazons, Facebook, Google show up along with the big banks that are like actually making huge amounts of money using Python to the point where if Python disappeared or if Python fell away, they would be losing like billions of dollars of income or being able to move money around like and investments and stuff.

00:58:42.840 --> 00:58:49.760
So it was just a great opportunity to hear directly from them, hear that and basically find out that, yeah, all of our problems are the same.

00:58:49.760 --> 00:58:52.440
And a lot of them are shared with the community.

00:58:52.440 --> 00:58:56.000
A lot of packaging concerns affect everyone.

00:58:56.000 --> 00:59:06.440
A lot of performance concerns and like the C API for CPython is a hot topic right now, both in the community and in the corporate world.

00:59:06.440 --> 00:59:10.240
And everyone kind of has different ideas about what to do and how to fix it.

00:59:10.240 --> 00:59:13.440
And so it was a lot of people said it was a great time.

00:59:13.440 --> 00:59:18.340
They were very excited that there was a chance to share some of this.

00:59:18.340 --> 00:59:23.220
As I said, it's a little bit private kind of thing.

00:59:23.220 --> 00:59:27.800
So we're not putting out reports on specific things from specific companies.

00:59:27.800 --> 00:59:33.540
But certainly there are those of us like Microsoft who are kind of that channel, right?

00:59:33.540 --> 00:59:39.460
So, you know, we're a little bit of gatekeepers, I guess, which is not ideal.

00:59:39.460 --> 00:59:52.460
But at the same time, if it's the only way for like if the only way for the community to hear what the corporations are worried about is for us to kind of aggregate it and say, we spoke to corporations that won't publicly talk about this stuff.

00:59:52.460 --> 00:59:56.900
And, you know, you'll just have to trust that they said it's a problem to us.

00:59:56.900 --> 01:00:05.540
And we're kind of repeating anonymously, which the interactions between the corporate world and the community of open source are actually really, really complicated.

01:00:05.540 --> 01:00:06.680
Yeah, I'm sure they are.

01:00:06.680 --> 01:00:10.940
Most of what we've talked about today has been how complicated it can be.

01:00:11.060 --> 01:00:16.580
It's still so complicated, still so much uncertainty about how to interact.

01:00:16.580 --> 01:00:25.540
There's a lot of investment that these corporations want to make into community projects like Python, like Pip, like NumPy.

01:00:25.540 --> 01:00:30.560
And really, there's a huge amount of uncertainty about how to actually do it.

01:00:30.560 --> 01:00:32.220
And there's a lot of reasons for that.

01:00:32.220 --> 01:00:34.720
And probably most of them are solvable.

01:00:34.720 --> 01:00:38.920
But the only way to solve them is by having some amount of open dialogue.

01:00:39.660 --> 01:00:46.760
And we're trying to open that up and make it possible for companies to say, hey, this is what we're worried about.

01:00:46.760 --> 01:00:47.900
This is our concerns.

01:00:47.900 --> 01:00:49.060
This is what we can offer.

01:00:49.060 --> 01:00:55.880
And try and link up with the community to make some of that actually happen.

01:00:55.880 --> 01:00:56.180
Yeah.

01:00:56.180 --> 01:00:59.240
So apart from the corporations, we did have some community people in the room as well.

01:00:59.240 --> 01:01:06.380
I think we had seven Python core developers in the room, mostly employees of these companies, but they were there.

01:01:07.280 --> 01:01:16.180
You know, we were looking around going, did that guy get up and give an entire talk about using Python and not say any dollar amount under a million dollars?

01:01:16.180 --> 01:01:21.920
Like it's like every single, not the only number less than a million was the version number of Python he was talking about.

01:01:21.920 --> 01:01:22.440
Yeah.

01:01:22.440 --> 01:01:22.940
Yeah.

01:01:22.940 --> 01:01:23.920
Did that really happen?

01:01:23.920 --> 01:01:25.060
Does Python get used for that?

01:01:25.060 --> 01:01:25.680
Yeah, it does.

01:01:25.680 --> 01:01:33.620
But how do you turn that like millions and billions of dollars into community contributions is a really difficult problem.

01:01:33.620 --> 01:01:34.840
Yeah, it sounds like it is.

01:01:34.920 --> 01:01:48.400
And I think if you guys could unlock that somewhat, that would be so amazing because there's so many projects that are underfunded or run by one person in their spare time that are probably critical to the work that these other folks are doing.

01:01:48.960 --> 01:01:55.920
If you could somehow get that support to flow the other way or get at least their concerns to help make CPython better, that'd be great.

01:01:55.920 --> 01:01:57.660
There's been a few attempts to do this.

01:01:57.660 --> 01:02:06.200
There's a number of companies out there that are set up for kind of the purpose of redirecting corporate funding into open source projects.

01:02:06.200 --> 01:02:09.260
Anaconda, originally Continuum, is one of those.

01:02:09.260 --> 01:02:13.500
Travis Elephant's new venture, QuantSight, is trying to do the same thing.

01:02:14.340 --> 01:02:16.780
Tidelift is also following a similar thing.

01:02:16.780 --> 01:02:18.020
Yeah, Tidelift is really cool.

01:02:18.020 --> 01:02:31.840
It largely comes down to corporations will give out money for exactly one reason, and it's to get a return, which means you either have to convince them that altruism is going to get them some good PR or actually sell them a product.

01:02:31.840 --> 01:02:43.760
And Tidelift and Anaconda and QuantSight are all selling the corporations a product and turning that into open source funding, which is something that individual projects have really struggled to figure out.

01:02:43.760 --> 01:02:44.480
Yeah, absolutely.

01:02:44.480 --> 01:02:48.980
Like the Patreon model or the PayPal donate button, you know.

01:02:48.980 --> 01:02:49.780
Oh, yeah.

01:02:49.780 --> 01:02:50.220
Good luck.

01:02:50.220 --> 01:02:50.640
Good luck.

01:02:50.640 --> 01:02:50.960
Yes.

01:02:50.960 --> 01:02:56.840
You can't convince, you know, a vice president to, like, to give money to Patreon.

01:02:56.840 --> 01:03:00.480
It just, it's not how any of these big companies work.

01:03:00.480 --> 01:03:04.800
Yeah, they're just like, well, it doesn't make even any sense to us how this works.

01:03:04.800 --> 01:03:05.000
Yeah.

01:03:05.000 --> 01:03:05.400
Yeah.

01:03:05.400 --> 01:03:07.200
And you have a scale problem as well.

01:03:07.200 --> 01:03:15.700
We actually face this with trying to sponsor meetups is finding the person that can approve, you know, a $1,000 sponsorship is actually kind of hard.

01:03:15.700 --> 01:03:19.660
Like, because no individual engineer can approve that expense because that's too big.

01:03:19.920 --> 01:03:24.780
But what if the only people you can find look at that and say, why are you bothering me with something so small?

01:03:24.780 --> 01:03:25.280
Yeah.

01:03:25.280 --> 01:03:27.060
Come back when it's six figures or more.

01:03:27.640 --> 01:03:39.320
And it's actually this really hard spot to get anything done in a large company when it's too big to go unnoticed but too small to be, like, to deserve a decision maker.

01:03:39.320 --> 01:03:41.680
Yeah, it's a weird no man's land in the middle there.

01:03:41.680 --> 01:03:42.480
Indeed.

01:03:42.480 --> 01:03:46.620
Well, it sounds to me like this summit you guys had is a real positive thing.

01:03:46.620 --> 01:03:47.780
So, yeah, keep that up.

01:03:47.780 --> 01:03:48.700
And thanks for the report.

01:03:48.700 --> 01:03:48.940
Yeah.

01:03:48.940 --> 01:03:49.360
Yeah, indeed.

01:03:49.760 --> 01:03:55.260
All right, well, I think as much as I have a million more questions for you, I think we have to call it an episode on that.

01:03:55.260 --> 01:03:58.000
So, let me ask you the two final questions before you go.

01:03:58.000 --> 01:04:00.200
And maybe we can touch on something else as well.

01:04:00.200 --> 01:04:03.300
So, if you're going to write some Python code, what editor do you use these days?

01:04:03.300 --> 01:04:06.500
Probably not Visual Studio Tools for Python these days, right?

01:04:06.500 --> 01:04:07.960
That was my project for a long time.

01:04:07.960 --> 01:04:09.780
So, I have a lot of sympathy.

01:04:09.780 --> 01:04:10.860
I do still use it.

01:04:10.860 --> 01:04:12.780
I wouldn't say it's my primary editor.

01:04:12.780 --> 01:04:15.420
Like, Visual Studio Code right now is probably my primary editor.

01:04:15.420 --> 01:04:17.000
That was going to be my guess, yeah.

01:04:17.220 --> 01:04:27.380
Yeah, well, but the main reason for that is I got up on stage at a conference recently to do demos of it and just failed miserably at a number of things because I had not spent enough time using it.

01:04:27.380 --> 01:04:34.400
It's a really cool editor, but it's one of those you kind of have to live in because there's a lot of non – there's not lots of buttons and stuff.

01:04:34.400 --> 01:04:37.140
You've got to kind of know how to interact with it, right?

01:04:37.140 --> 01:04:43.920
Yeah, and I know Visual Studio inside out, so I can use that comfortably, but I don't feel like I'm learning anything.

01:04:44.580 --> 01:04:53.400
So, I'm living in Visual Studio Code these days to learn and get a good feel for it because, you know, it's rightly or wrongly, people expect me to know how it works.

01:04:53.400 --> 01:04:58.300
And, you know, it would be helpful for that to actually be true when they ask me questions about it.

01:04:58.300 --> 01:05:00.680
So, that is where I'm living right now.

01:05:00.680 --> 01:05:02.360
But also the extension for that.

01:05:03.600 --> 01:05:08.800
So, the Python extension, when we adopted that, oh, gee, a year, two years ago?

01:05:08.800 --> 01:05:09.640
Two years ago, yeah.

01:05:09.640 --> 01:05:12.200
It was like a one-person part-time thing.

01:05:12.200 --> 01:05:16.560
It now has like 100 million people working on it or something.

01:05:16.560 --> 01:05:18.140
I can't even keep track of the team.

01:05:18.140 --> 01:05:18.440
Yeah.

01:05:18.760 --> 01:05:23.920
We have six people based in Vancouver, most of whom I've never actually met in person.

01:05:23.920 --> 01:05:33.460
Like, it's moving along so quickly that it's at that really exciting time where every month when the update comes in, there's new and exciting stuff.

01:05:33.460 --> 01:05:34.960
And it feels totally fresh.

01:05:34.960 --> 01:05:36.600
And it's just exciting to be using right now.

01:05:36.600 --> 01:05:37.600
So, that's also fun.

01:05:37.600 --> 01:05:37.880
Awesome.

01:05:37.880 --> 01:05:41.560
Yeah, it definitely seems like there's a ton of energy around Visual Studio Code.

01:05:41.560 --> 01:05:46.160
And I suspect most people know, but obviously, it runs on all the platforms, right?

01:05:46.160 --> 01:05:49.960
It's not just on Windows, even though it shares the Visual Studio name.

01:05:49.960 --> 01:05:51.400
Yeah, Windows, Mac, Linux.

01:05:51.400 --> 01:05:56.380
Having .NET Core support for all of those platforms is, I feel like that's the limiting factor right now.

01:05:56.380 --> 01:05:58.460
Obviously, it's electron-based.

01:05:58.460 --> 01:06:00.140
So, anywhere that can run, it'll work.

01:06:00.140 --> 01:06:04.800
But some big pieces, especially the Python support, are running in .NET Core.

01:06:04.800 --> 01:06:05.940
So, that's about the limit.

01:06:06.300 --> 01:06:08.040
That's still a really big limit.

01:06:08.040 --> 01:06:14.920
Like, you've got some old versions of Linux distros that we support because it works there.

01:06:14.920 --> 01:06:15.500
Yeah.

01:06:15.500 --> 01:06:18.260
Where really most people would not dream of using them.

01:06:18.260 --> 01:06:20.420
But you can run VS Code there.

01:06:20.420 --> 01:06:21.360
Like, it's a tool.

01:06:21.360 --> 01:06:24.340
It's got to run where people want to use it.

01:06:24.340 --> 01:06:27.080
We can't force you to upgrade your operating system just for the editor.

01:06:27.080 --> 01:06:27.960
Yeah, that's great.

01:06:27.960 --> 01:06:28.980
All right.

01:06:28.980 --> 01:06:31.520
I definitely think there's a ton of energy around VS Code.

01:06:31.520 --> 01:06:36.200
And I think that whole Python extension editor side of the story deserves.

01:06:36.200 --> 01:06:41.920
Like, sort of mention in the Python's journey at Microsoft, even though it, you know, didn't get it to the very end here.

01:06:41.920 --> 01:06:42.700
All right.

01:06:42.700 --> 01:06:44.280
And then, notable PyPI package?

01:06:44.280 --> 01:06:44.660
Yeah.

01:06:44.780 --> 01:06:47.100
So, one that I'm really excited about right now.

01:06:47.100 --> 01:06:51.860
The Azure Machine Learning team recently came out with their new product.

01:06:51.860 --> 01:06:54.280
And the name is very similar to a lot of things we've had before.

01:06:54.280 --> 01:06:57.440
So, it can be tricky to track through exactly what it is.

01:06:57.580 --> 01:07:05.660
But Azure Machine Learning Service is basically an entire service for being able to do your machine learning tasks.

01:07:05.660 --> 01:07:08.980
Everything from data cleaning to model management and deployment.

01:07:08.980 --> 01:07:16.360
In a way that works really nicely balanced between your local editor and running stuff and pushing stuff in the cloud.

01:07:16.360 --> 01:07:22.040
So, it's got a whole lot of nice functionality for, hey, run this job in a cluster that is this big.

01:07:22.040 --> 01:07:23.420
Let me know when you're done.

01:07:23.420 --> 01:07:24.740
Nice.

01:07:24.740 --> 01:07:26.960
And run that training job and then store the model somewhere.

01:07:26.960 --> 01:07:30.980
You get a good history of all the models that have ever been published.

01:07:30.980 --> 01:07:35.640
And that's one of the products that has come out with only support for Python.

01:07:35.640 --> 01:07:37.740
Like, if you want to use that, you're using Python.

01:07:37.740 --> 01:07:39.780
There's actually no other options right now.

01:07:40.220 --> 01:07:41.900
So, I'm excited about that.

01:07:41.900 --> 01:07:47.140
But one particular part of that package that's really cool is the Azure ML Data Prep package.

01:07:47.140 --> 01:07:48.580
So, this is one part of it.

01:07:48.580 --> 01:07:55.220
But it's the part for I have a file on disk or I have a set of files on disk that have some raw data in it.

01:07:55.220 --> 01:07:57.120
And I need to preprocess.

01:07:57.120 --> 01:07:57.920
I need to clean.

01:07:57.920 --> 01:08:02.540
I need to extract data from certain columns and split them into more.

01:08:02.540 --> 01:08:03.740
I need to add more columns.

01:08:03.740 --> 01:08:05.180
I need to remove rows.

01:08:05.180 --> 01:08:06.400
I need to replace missing values.

01:08:06.400 --> 01:08:08.120
It's a library for doing that.

01:08:08.300 --> 01:08:10.280
It does a couple of really cool things.

01:08:10.280 --> 01:08:17.740
So, if anyone's used Flash Fill in Excel, which is this cool feature, you can try to put in a list of people's names.

01:08:17.740 --> 01:08:20.900
Just first name, last name, however many names in one column.

01:08:20.900 --> 01:08:23.340
And then start splitting them out manually.

01:08:23.340 --> 01:08:27.280
You put a few examples of type the first name in the next column, then the last name.

01:08:27.280 --> 01:08:28.660
And do that for two or three.

01:08:28.660 --> 01:08:34.400
And it will suggest splitting up every single row using those examples and figure out how to do it.

01:08:34.400 --> 01:08:35.560
So, that's been in Excel for a while.

01:08:35.560 --> 01:08:36.820
It's also in this package.

01:08:37.240 --> 01:08:44.060
So, you can take a raw data file that has, you know, maybe it has the date and time in those same column.

01:08:44.060 --> 01:08:47.100
And you say, okay, here's a couple of examples of how I want it split out.

01:08:47.100 --> 01:08:50.740
Put, you know, the date here and the time in a separate column.

01:08:50.740 --> 01:08:52.060
And it will figure out how to do that.

01:08:52.060 --> 01:08:55.160
And it can, it will use those examples.

01:08:55.160 --> 01:08:59.680
And then you can run it, stream it over a huge, huge file that won't fit in memory.

01:08:59.760 --> 01:09:04.680
And it will do it to the whole thing and write out a new file that's been pre-processed.

01:09:04.680 --> 01:09:05.720
It's changed together.

01:09:05.720 --> 01:09:06.420
Yeah, it's cool.

01:09:06.420 --> 01:09:09.040
Yeah, it's really, really neat.

01:09:09.040 --> 01:09:11.640
I guess there'll be a link in the show notes to the info on that one.

01:09:11.900 --> 01:09:19.200
But that, when we look at how far Microsoft has come in terms of Python usage, and it seems, this feels like a great ending to this show, actually.

01:09:19.200 --> 01:09:19.720
Yeah.

01:09:20.220 --> 01:09:20.900
This is the new point.

01:09:20.900 --> 01:09:25.720
We have some amazing technology here that is just Python.

01:09:25.720 --> 01:09:26.900
It's just Python.

01:09:26.900 --> 01:09:29.000
Like, it's not even .NET first and then Python.

01:09:29.000 --> 01:09:31.040
It's not even .NET second right now.

01:09:31.040 --> 01:09:32.180
It's Python only.

01:09:32.180 --> 01:09:32.760
It's just Python.

01:09:32.760 --> 01:09:33.640
It's Python all the way.

01:09:33.640 --> 01:09:34.380
That's beautiful.

01:09:34.380 --> 01:09:35.280
And that's so exciting.

01:09:35.280 --> 01:09:39.180
And, you know, I can't wait to see what comes next.

01:09:39.180 --> 01:09:42.740
Yeah, well, it sounds like there's a ton of momentum going there.

01:09:42.740 --> 01:09:45.460
And you've done a lot to get it going in the right direction.

01:09:45.460 --> 01:09:46.380
So that's wonderful.

01:09:46.380 --> 01:09:46.840
All right.

01:09:46.840 --> 01:09:48.060
So people are excited.

01:09:48.060 --> 01:09:49.660
Maybe they want to learn more about this history.

01:09:49.820 --> 01:09:50.640
Some more of the details.

01:09:50.640 --> 01:09:51.520
Things like that.

01:09:51.520 --> 01:09:52.100
What do they do?

01:09:52.100 --> 01:09:52.760
Final call to action.

01:09:52.760 --> 01:09:54.880
I'd say go read the Medium post.

01:09:54.880 --> 01:09:59.380
And that's got links scattered throughout to interesting things.

01:09:59.380 --> 01:10:02.420
And down the end, there's links to all the current things.

01:10:02.420 --> 01:10:09.660
So once you've seen where we came from, where we've gotten to, then I'm trying to make it easy to see what we have on offer today.

01:10:09.660 --> 01:10:09.900
Yeah.

01:10:09.900 --> 01:10:11.180
Well, a lot of cool stuff.

01:10:11.180 --> 01:10:14.000
And thanks for doing this historical journey story.

01:10:14.000 --> 01:10:14.340
It's great.

01:10:14.340 --> 01:10:14.600
Yeah.

01:10:14.600 --> 01:10:15.020
Thanks.

01:10:15.020 --> 01:10:15.400
You bet.

01:10:15.400 --> 01:10:16.080
Talk to you later.

01:10:16.080 --> 01:10:16.440
See ya.

01:10:16.440 --> 01:10:19.780
This has been another episode of Talk Python To Me.

01:10:19.780 --> 01:10:22.680
Our guest on this episode was Steve Dower.

01:10:22.680 --> 01:10:26.660
And it's been brought to you by Linode and us at Talk Python Training.

01:10:26.660 --> 01:10:30.920
Linode is your go-to hosting for whatever you're building with Python.

01:10:30.920 --> 01:10:34.480
Get four months free at talkpython.fm/Linode.

01:10:34.480 --> 01:10:36.380
That's L-I-N-O-D-E.

01:10:37.020 --> 01:10:38.580
Want to level up your Python?

01:10:38.580 --> 01:10:43.420
If you're just getting started, try my Python Jumpstart by Building 10 Apps course.

01:10:43.420 --> 01:10:51.580
Or if you're looking for something more advanced, check out our new async course that digs into all the different types of async programming you can do in Python.

01:10:51.580 --> 01:10:56.240
And of course, if you're interested in more than one of these, be sure to check out our Everything Bundle.

01:10:56.240 --> 01:10:58.140
It's like a subscription that never expires.

01:10:58.140 --> 01:11:00.420
Be sure to subscribe to the show.

01:11:00.420 --> 01:11:02.840
Open your favorite podcatcher and search for Python.

01:11:02.840 --> 01:11:04.060
We should be right at the top.

01:11:04.060 --> 01:11:13.040
You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm.

01:11:13.560 --> 01:11:15.140
This is your host, Michael Kennedy.

01:11:15.140 --> 01:11:16.640
Thanks so much for listening.

01:11:16.640 --> 01:11:17.700
I really appreciate it.

01:11:17.700 --> 01:11:19.460
Now get out there and write some Python code.

01:11:19.460 --> 01:11:19.620
Thank you.

01:11:19.620 --> 01:11:39.560
Thank you.

01:11:39.560 --> 01:12:09.540
Thank you.