#476: Unified Python packaging with uv Transcript

00:00 A couple weeks ago, Charlie Marsh and the folks at Astral made another big splash with a major

00:04 release of UV called UV Unified Python Packaging, which has many far-reaching features. We had to

00:11 have Charlie on the show to give us an inside look into this development. Let's get to it.

00:16 This is Talk Python to Me, episode 476, recorded September 3rd, 2024.

00:21 Are you ready for your host, Darius?

00:24 You're listening to Michael Kennedy on Talk Python to Me.

00:28 Live from Portland, Oregon, and this segment was made with Python.

00:31 Welcome to Talk Python to Me, a weekly podcast on Python. This is your host, Michael Kennedy.

00:39 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython,

00:45 both accounts over at fosstodon.org. And keep up with the show and listen to over

00:50 nine years of episodes at talkpython.fm. If you want to be part of our live episodes,

00:55 you can find the live streams over on YouTube. Subscribe to our YouTube channel over at

00:59 talkpython.fm/youtube and get notified about upcoming shows.

01:03 This episode is sponsored by Posit Connect from the makers of Shiny.

01:08 Publish, share, and deploy all of your data projects that you're creating using Python.

01:13 Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quattro, Reports, Dashboards, and APIs.

01:20 Posit Connect supports all of them. Try Posit Connect for free by going to talkpython.fm/posit.

01:24 And it's also brought to you by us over at Talk Python Training. Did you know that we have over

01:33 250 hours of Python courses? Yeah, that's right. Check them out at talkpython.fm/courses.

01:40 Charlie, welcome back to Talk Python to Me. How are you doing?

01:43 I'm good. Thanks so much for having me back. It's always a pleasure to be on here.

01:47 Well, you are certainly making waves in the Python space these days with all of your projects

01:53 and their fastness. I think you're known as the fast Python guy these days.

01:58 Oh, that's not fast enough. We got to use something from Astral.

02:01 The fast Python tooling guy, maybe. Yeah.

02:05 Yeah, exactly. Exactly. So interesting. We spoke, I think you've been on twice before. Is that right?

02:11 We talked about Ruff once.

02:12 Came on once to talk about Ruff, and then I came on in March shortly after we did the first public

02:18 release of UV. Right, right. And UV at that time, I'll let you tell people what UV is, but for those

02:25 who know, it was kind of in the, it has pip-like features. And now it's got another layer to the

02:32 onion or another shell to it. And that's actually why I invited you back because there's a lot of

02:37 interesting things to talk about there. Yeah. Maybe like several more layers too.

02:41 Okay, well, let's hear about them. So first of all, I guess, not everyone, believe it or not,

02:50 not everyone listened to that first Ruff interview, R-U-F-F interview and those kinds of things. So I

02:58 suppose maybe quick introductions to you, Astral, how'd you get to where you are?

03:05 Totally. Yeah, yeah, yeah. So I've been working on Python tooling full-time for a little under

03:13 two years at this point. And my journey, I've been writing Python professionally for a long time,

03:19 but my journey with Python tooling started with Ruff, which is a very fast Python linter,

03:27 code formatter and code transformation tool. And the sort of foundational idea of Ruff was

03:35 can we make really fast Python tooling? And so Ruff is written in Rust, but you don't need to

03:41 have Rust installed or know anything about Rust to use it. You can just install it with pip or

03:45 with whatever you're using. And that whole sort of story for me started from just working with

03:52 Python a lot and asking if we could build sort of different Python tooling by using languages like

04:00 Rust. So the first version of Ruff was very minimal, but it's grown from then to have,

04:06 I don't know, maybe something like 800 plus rules. If you've used tools like Lake Eight or Pylint in

04:11 the past, Ruff can do similar things. So discover issues with your code without running it. It can

04:17 also fix them automatically. And then if you've used tools like Black in the past for code

04:23 formatting, Ruff can do a similar thing too, where it automatically formats your code.

04:27 So that was how I first got started in Python tooling. And also how I first ended up on

04:31 Talk Python was people were very interested in this when it came out.

04:34 And I think it broke a lot of people's minds in the sense that it really changed what was possible.

04:41 Maybe an analogy. So if you have unit tests, but to run the unit test takes 30 minutes,

04:48 there's no one who doesn't just want to kill time. Who's going to say, I made a small change.

04:53 Let me run the test and see how that change did. But if the test run in 50 milliseconds,

04:58 you could just set up a thing to just run the test every time code changes and just have a

05:03 green, red, white. And I feel like Ruff did that for linting and formatting.

05:09 Like for me, for example, I have it just run Ruff on, if I ask, I use PyCharm,

05:15 ask PyCharm to reformat the code. It just, it uses Ruff to do that. Or, and you can check the

05:19 box to just say, run it when I hit save, because you won't notice a difference. You know what I

05:23 mean? Yeah. Yeah. I think that's been one of the really interesting things about working on

05:27 tooling with the focus on performance has been like, if you build things that are way,

05:32 way faster than what currently exists, like not just like a little bit, but like significantly

05:36 faster, the ergonomics of the tool just change a lot. So maybe, maybe things that you only ran

05:41 in CI before you can now run locally and maybe things that you now run locally, you can run.

05:46 Yeah. On save as opposed to on, you know, on commit or on explicit command. So that's been a

05:52 big part of you know, what we've seen with Ruff. And, you know, the other big piece apart from

05:58 performance that we've been, that I think we sort of accidentally made a lot of progress on with

06:03 Ruff was just trying to make things simpler by bundling more stuff together. So Ruff can do like

06:10 a lot of different things. And so you don't need as many to chain together as many tools to get the

06:14 kind of setup that you might want to have in your Python projects. And so, you know, a lot of the

06:19 things we've built, you know, with Ruff and with UV, they've had these kind of dual goals of,

06:23 you know, being much faster, you know, hopefully bringing in some interesting new features and

06:28 functionality, but also trying to reduce a little bit of the, you know, the fragmentation and

06:34 tooling and make it easier for people, especially, you know, for newcomers who are writing Python for

06:38 the first time to kind of get started. And we have, you have announcements. I have things to

06:44 discuss with you about them, about making that even easier. So that's why some of the big features

06:49 of UV, right. For example, what if I don't have Python? Not a problem. So that's pretty, that's

06:54 pretty exciting, I think. Yeah. Yeah. And we'll get into that, but I do want to just take a, not

06:59 get too far past it because even though this episode is not about Ruff, two things. One,

07:06 a theme of Astral is we're going to take some tools or some kind of concept that's been around

07:11 for a long time, redo it in Rust, but also not just, well, we ported over the code, but you know,

07:17 rethink some of the algorithms and some of the assumptions. And I think that's also a lot of

07:21 what makes it faster, like the way UV works with caching and things like that. Yep. And there's

07:26 been a lot of excitement and positivity about that, but I think there's also been a little

07:32 suspicion of, well, wait a minute. If, if we write this in Rust, is it really for Python people,

07:38 by Python people? Like what's, what's your take? I don't, I have, I have some thoughts that are,

07:43 I don't see it as a negative, but I think, I think it's interesting just to discuss,

07:47 you know, where does Rust actually belong here? Yeah. I mean, I think like the way I view it is

07:53 that you know, just like Python itself is, is implemented in C, like it makes some sense for

08:00 like the underlying tooling that people are using to be written in a different language. If it means

08:05 that you can ultimately build, you know, tools that are better, you know, at least on, on some

08:11 metrics that people care about. So like for me, I sometimes view what we're doing as, you know,

08:16 like we put all this investment into this tooling. We write all this stuff in Rust, which, you know,

08:22 in theory incurs some costs for us. Maybe, you know, maybe we move a little bit more slowly,

08:27 maybe it's harder. And in doing so, we kind of like give all these people who are running these

08:32 commands, you know, hundreds of times a day, a big boost in how they work. And they can keep

08:38 writing Python, but have a better and better experience writing Python. So, you know, I don't

08:43 think it actually really says anything personally. I don't really think it says anything about Python

08:48 that we write the tooling in Rust. Cause you know, we get, sometimes people will criticize

08:52 Python for that, but like, you know, different languages are useful and good for different

08:56 things. And Python is an incredibly productive language. It's like growing more and more every

09:01 year. People love writing it. But maybe it's not the best language to write, you know, a super high

09:07 performance package manager. And that's totally fine. Cause like, we're going to do that. And

09:11 everyone who's writing Python can benefit from it and can leverage it. Yeah. I do think that

09:16 one interesting piece here that I got, you know, we got a lot of skepticism around early on was like,

09:23 you know, how can people in the Python community like contribute to this if it's not written in

09:27 Python? You know, how do you like have a sustainable contributor base? And the interesting

09:32 thing for me is like, I actually think we've had a really strong contributor base over time. Like

09:37 we have, I don't know, like somewhere between four and 500 contributors total, maybe in Rust.

09:42 You know, most of those are like one or two small, you know, one or two commits, maybe just one or

09:46 two time committers. You know, but a lot of people have come to Rust and said, you know, I want to

09:51 learn Rust. I come from the Python ecosystem. There's a part of this that's sort of native to

09:55 what I know, which is the Python language, the Python semantics. And it's an entry point for

09:59 me to learn Rust, you know, from other people who come from Python. So, you know, I think there is

10:04 certainly a cost to engaging and a cost for contributors because most, you know, most of

10:09 our users don't know Rust. But I don't think it's as steep as people would expect. And I think,

10:17 you know, over time we have been able to grow a pretty healthy contributor base, like many people

10:22 who have never written Rust before. Yeah, I agree. And I was thinking the same thing that you

10:26 pointed out that Python itself is written in C. Right. Generally speaking, of course, there's

10:32 multiple runtimes and so on and so on. But CPython, the one that we generally use is written

10:37 in C. And I mean, I don't see why there should be an allegiance to C over Rust or anything. And a

10:43 lot of the data science tools are written in C or Fortran even, you know, something really wild

10:49 like that. And Python is a way to kind of expose them and make them more accessible, more useful,

10:54 make people more productive on top of those, that's a native code. Yeah, yeah. A third example

11:00 would be Jupyter, JupyterLab, Jupyter Notebook, all those things. When you talk to the contributors of

11:04 those projects, they're down in TypeScript and JavaScript most of their day. So the data science

11:10 folks don't have to write TypeScript and JavaScript. They just work in the tooling that's on the front

11:15 end. And so it's, you know, it's kind of like, to me, it's kind of like, let's take one for the

11:20 team. Yeah. We'll do this. So we all benefit, but you don't have to, you don't have to do it.

11:24 I actually think it's one of the kind of, one of the real strengths of Python is the

11:30 interoperability around like native code and extension modules. It's like a big,

11:35 it's actually like a big part of Python and it's a big part of why Python has succeeded.

11:42 And I think that will actually become more, I mean, this is sort of like going off on a tangent

11:46 a little bit, but I think that will become a bigger and bigger part of like what Python looks

11:50 like and what it looks like to work with Python. Because, you know, and I think Rust can actually

11:56 play an interesting role here because at least from my perspective, like I do not have a systems

12:00 programming background. Like I've not, I've definitely not written any C professionally.

12:06 I don't think I've ever really written any C++. And Rust was my first sort of foray into that

12:13 kind of programming. And I think at least for me personally, I found it much more accessible.

12:19 And so I think Rust and Python actually have this interesting story where Rust to some degree

12:25 is this kind of like superpower low level layer that you can drop into and then expose with a

12:30 Python CLI forever in a consume. So anyway, my prediction is I think that will only become more

12:35 and more of a part of what Python looks like. And that's actually, I think it's actually a good

12:40 thing. Yeah, I do too. I do too. Jake out in the audience, Jake Callahan, who says, and another

12:45 comment, "Great timing. I just released a video on these changes yesterday." So that's awesome.

12:50 But said, "The progress made in Ruff and UV sensor introductions clearly demonstrates that

12:54 there's a strong contributor base, even when it's based on Rust, right?" Yeah. Yeah. I think another

13:00 thing that it- I'm not trying to dismiss that it's hard to learn Rust because I actually think

13:03 it is pretty hard to learn Rust. And I think the learning curve is pretty steep. But I think the

13:08 thing we found is there are people who are interested and who want to do it. Sure. Sure.

13:12 Agreed. I think another thing that it shows, regardless of Rust or not, is if it's your

13:18 full-time job and there's funding and a team, rather than part-time work, it can go many times

13:25 faster towards the goal than before. I'm sure you've felt that way. Yeah. I mean, we're in a

13:30 very unique, I mean, not completely unique, but fairly unique position whereby we can fund people

13:37 full-time to work on this stuff. And it's actually just kind of wild to look at how much, not to sort

13:44 of talk up our own work, but just how much you can accomplish in a relatively short period of time

13:49 with that level of when people are kind of unlocked to work in that way and work on those kinds of

13:54 problems. So just thinking, it's like the amount of time we've been able to put into Ruff and UV

14:02 is sort of hard to fathom for some open source projects. It's like, wow, that's incredible.

14:06 But from the perspective of a company, it's actually quite reasonable. And in fact, not even

14:11 like that much. So I think it's pretty, like the leverage that you have with open source,

14:17 like when people are able to invest in it and really focus on it is pretty amazing because

14:23 we work on these things, we improve these things and they ship out to, I don't know,

14:26 millions of users or whatever it is, and really can provide a huge boost to the ecosystem.

14:32 So yeah, I mean, being able to work on this stuff full-time, that was actually something that I

14:35 decided pretty early on with Ruff was the project was growing more and more popular. And I was like,

14:41 I don't know if I can keep up with this if I don't work on it full-time. And so I'm at least going

14:46 to see where that goes. And if it's possible. - It's possible, right? And here you are.

14:52 - I mean, it is, but yeah, it's like, I mean, I just feel like I just have a lot of respect for

14:57 people who, you know, commit significant amounts of their time to open source alongside everything

15:02 else because yeah, it's, I mean, it's an amazing thing, but it's also very challenging and it's-

15:08 - Oh, it definitely is. - It's 24/7.

15:10 - Well, I think it's really positive that there's a group of people putting their full effort into,

15:17 especially the UV side of things. That's going to make a dent.

15:22 - Yeah, I mean, I think the other thing that's been cool about, you know, building this as a

15:27 company is that we've been able to bring in a lot of people from like outside of Python to come and

15:34 work on Python tooling. And I think that's like a really healthy thing for the ecosystem because

15:39 not only are these people, you know, I think like really talented and really great, but they also

15:44 bring like very different ideas and experiences. And so we have a lot of cross-pollination on the

15:48 team. The team's kind of a mix of people who fall everywhere on the spectrum of like, or axes,

15:54 I guess, probably not a spectrum of like how much Rust experience, how much Python experience they

15:58 have. You know, some of them have mostly worked in like TypeScript and the JavaScript ecosystem. So

16:03 that was sort of an intentional thing for me when thinking about like building a team to work on

16:06 this stuff was I want to have like some people on the team who are like super deep Python experts.

16:12 And I want to have some people who like bring a totally different perspective to the problems.

16:16 So that's been another like very cool piece of putting together a team to work on this stuff.

16:20 - I think that's a great perspective. This portion of Talk Python to Me is brought to you by Posit,

16:26 the makers of Shiny, formerly RStudio, and especially Shiny for Python. Let me ask you

16:32 a question. Are you building awesome things? Of course you are. You're a developer or data

16:37 scientist. That's what we do. And you should check out Posit Connect. Posit Connect is a way

16:42 for you to publish, share, and deploy all the data products that you're building using Python.

16:47 People ask me the same question all the time. Michael, I have some cool data science project

16:52 or notebook that I built. How do I share it with my users, stakeholders, teammates? Do I need to

16:58 learn FastAPI or Flask or maybe Vue or ReactJS? Hold on now. Those are cool technologies, and I'm

17:05 sure you'd benefit from them, but maybe stay focused on the data project. Let Posit Connect

17:09 handle that side of things. With Posit Connect, you can rapidly and securely deploy the things

17:14 you build in Python. Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Ports, Dashboards,

17:20 and APIs. Posit Connect supports all of them. And Posit Connect comes with all the bells and

17:26 whistles to satisfy IT and other enterprise requirements. Make deployment the easiest

17:32 step in your workflow with Posit Connect. For a limited time, you can try Posit Connect for free

17:36 for three months by going to talkpython.fm/posit. That's talkpython.fm/POSIT. The link is in your

17:44 podcast player show notes. Thank you to the team at Posit for supporting Talk Python.

17:49 One thing before we move on is talk about UV proper. Closing the loop here. You mentioned

17:56 those rules, those 800 Lint rules. And this resource you have over at docs.ashville.sh/ruff/rules

18:04 is awesome. So even if you don't use Ruff, you know, you care about say PEP8 naming or something,

18:10 you get a warning from Flake 8 or whatever, and it says, "Under function name." Like,

18:15 what is this error, right? What is this thing? And then you've got each one of these rules has a,

18:21 what is this? Why is it bad? How should you change it? Here's the bad version. Here's the

18:26 good version. And so on. And I think this is just a really good resource. I know you all put a ton

18:30 of time and energy into it. - I appreciate you calling that out. Yeah, it takes a lot of time.

18:34 I mean, once you cover all the rules, maintaining it incrementally is slightly easier. But,

18:38 you know, when we started this, we had a few hundred rules. And so, you know,

18:41 it was a lot of contributors who contributed us getting to full coverage here.

18:45 - It's almost like the Wikipedia of Lint rules or something.

18:50 - How to Python, yeah. Yeah, I guess maybe one thing I'll say is like, we have a lot of rules,

18:55 but you don't have to use them all. And like the default rule set is actually pretty small.

18:59 So even when I work on projects, I don't necessarily enable all of the rules. I typically

19:03 enable like a couple of subsets. So I just think sometimes it can be intimidating to imagine 800

19:07 rules being applied over your code base. So, you know, the default rule set is pretty small. It

19:12 covers things like unused imports, you know, things that are pretty common and relatively

19:17 unobjectionable. And then we have all these different categories for ways that you can

19:20 kind of level up the level of coverage. - Yeah. I take pretty much the defaults

19:24 as well on all my projects, except for wider lines of code, more columns.

19:30 - Yeah. Yeah. - On a 32 inch monitor, 80 columns is just a little corner. - We actually took that out of the defaults.

19:38 - Did you? Oh, beautiful. - Yeah.

19:40 - I don't need a ruff.toml or whatever it was I was putting in there for a configuration.

19:45 - Yeah. - Because the thinking there is like, if you have long lines, they should generally be handled by a formatter, not a linter. So by default,

19:53 we don't enforce that. Yeah. - Okay. Interesting. And then the single quotes versus double quotes, which we discussed on GitHub, I remember. I think we talked about

20:01 last time. - Yeah, probably.

20:02 - Maybe as well. Yeah. - I talked about that with everyone.

20:05 - Yeah. And Miguel also agrees here. Amazing docs. The rules are top-notch. Nice work.

20:12 - I appreciate that. - Okay. So I know we're going to talk EV, but Kushar does have a good question out there. And I don't know if speak to it or don't

20:19 speak to it. Are there plans for Ruff to become a Pylint alternative? So a language server of some

20:26 sort. Is this anything that you're willing to talk about?

20:29 - I would say, I mean, yeah, I think that's like an interesting thing that we'll consider. It's

20:34 not something that we are committed to doing or committed not to doing, like building a language

20:39 server. I would say that's probably like a few steps away from where we are today. Because before

20:45 we would build a language server, we would probably build something like a type checker,

20:52 like something that could deal with type inference and better understand the relationships between

20:56 different parts of your code. I view that as like, it's not strictly a prerequisite to building a

21:01 language server, but I view that as like a little bit more of a path.

21:03 - More Mypy. - Yeah, more Mypy. I think there's like a lot of interesting stuff around building a, well, first of all, of course, there's like

21:12 building a type checker, but there's also building what I would call like a type aware linter. Like

21:18 you could imagine that Ruff itself could actually get a lot more powerful if it could do all sorts

21:22 of type inference. Like we have some rules that are only supposed to activate on dictionaries

21:26 and we do some sort of bad local type inference and heuristics to figure out if we think a variable

21:32 is a dictionary. Like if you call not get on it, it might be a dictionary, right? So if we knew,

21:38 there's just a lot of interesting things we could do. We could build a much better tool. So it's not

21:41 just about building a type checker. It's also about how can we build a much better linter and

21:46 sort of like set of tools. But yeah, we're thinking about that a lot. There's a lot of

21:50 discussion happening about that, both internally and publicly on the repo and the Discord. We're

21:56 thinking a lot about sort of type inference and how to evolve Ruff in that direction. So yeah,

22:01 language server may be something we can do eventually. It's obviously a huge undertaking

22:05 and we want to make sure that like anything we decide to build, we feel a lot of conviction,

22:11 that it can be great because it doesn't necessarily make sense for us to try and build

22:16 something that we think is like 5% better than what's out there. It's very hard to get users to

22:20 switch to something that's 5% better. So when we look at the opportunity set, we're kind of thinking

22:24 about where can we have a big impact? What do users really want? Cross-referenced against where do we

22:30 think there's opportunity to build something that would be an improvement in ways that users care

22:33 about? So yeah, very interesting. Not something we're focused on right now, the language or piece

22:38 at least. Yeah. You guys are going to need at least a few weeks to get to that, huh?

22:41 We're going to need a few weeks to get to language server.

22:43 All right. Ivan out there in the audience says, "Excellent tool. Ruff and UV have changed my life."

22:49 I've talked about Ruff.

22:50 That's very nice.

22:51 Yeah. UV, maybe a quick summary for what it is and then we could talk about your post,

22:57 which adds layers of features and capabilities as we...

23:01 Yeah, totally.

23:02 So I came on the show back in March and we talked a bit about what UV was at the time.

23:08 We released UV in mid-February, so that was a couple of weeks after.

23:11 And UV is our Python packaging tool. And it's built under a lot of the same

23:20 design goals as we had for Ruff. So we wanted it to be extremely fast. We wanted it to try and

23:30 bundle a few more things together so that you could use fewer tools to work with Python and

23:34 be productive with Python. And we wanted it to be really easy to adopt. That was another principle

23:39 that we've thought about a lot. So when we released UV in February, it was framed as a PIP

23:47 alternative. So the idea was if you've used pip before, pip install, pip uninstall, or tools like

23:55 pip compile or pip sync, it should be really familiar to you. So the API that we started with

24:00 was UV pip install, UV pip uninstall, UV pip compile. And it was meant to mirror the pip API.

24:07 And the idea there was we wanted it to hopefully be immediately obvious to people how this tool

24:13 works and what it does, and also, I guess, to some degree, what it doesn't do. So we released

24:18 that in February, and it grew extremely fast. It's just the adoption, both in terms of raw

24:27 numbers and just the companies that we talk to that are using it very quickly. A lot of people

24:32 were using it. I consider it a big success. I'm really happy with how that went. But it was also,

24:37 I think, a pretty small... It was sort of a small portion of what we wanted to build for packaging.

24:43 Ultimately, we weren't trying to build just a faster PIP. I think that's a really good starting

24:49 point for a couple of reasons. One, much easier for people to adopt. Two, it required us to build

24:57 a lot of the fundamental pieces that you need for Python packaging. In order to build a PIP

25:02 alternative, we had to be able to resolve Python dependencies, install Python packages,

25:08 understand virtual environments, manipulate all those things. We had to implement a bunch of

25:12 standards around introspecting, build distributions and source distributions and virtual

25:17 environments. So we had to invest in all these things that I view as fundamental primitives

25:23 of working with Python packaging. But the ultimate goal for us was always,

25:28 we want to build something that's a single static binary that you download and install,

25:32 that just gives you everything you need to be productive with Python. And from that perspective,

25:37 a pip alternative is pretty low level. And we wanted to build something that was a little bit

25:42 more high level, something that can install Python for you, something that can manage the

25:46 virtual environment for you, something where you just do run this file and it figures out the

25:51 dependencies, installs them, it creates the environments and it runs this file in the

25:54 environment. That is what we wanted to build. So that's what we've been building up to over

25:58 the past couple of months. Yeah. Amazing. Of course, you've got to have the UV pip features as a foundation, right? Because even if you don't,

26:07 kind of like we talked earlier, even if you don't expose it directly as a CLI thing,

26:12 that from a project management perspective, it's needed, right?

26:16 Yeah. And it's part of meeting people where they are to a certain degree. Everyone has workflows

26:23 that are built around PIP. Even people that have other packaging workflows, often those

26:29 workflows are actually built around pip commands. And like I said, I view those as a little bit low

26:33 level. With PIP, the operations are kind of like, install this package into this environment,

26:37 as opposed to here are my dependencies, make sure the world is in sync with my dependencies.

26:42 That's what I would consider to be a little bit more high level. So we built and released that

26:47 pip interface in February. And since then, it's basically been hammered by users. So it's just

26:54 gotten better and better, right? Because they find problems with it and then report them and it just

26:59 gets better. It gets faster. It gets more feature complete. So we've had a few months to kind of

27:03 improve it and build on top of it too. Yeah. That's one of the benefits of such,

27:08 it taking, they can hold so much and people using it so much as they hit the little edge cases all

27:14 the time and it gets smoothed out quick. Yeah. A lot of edge cases. Yeah. I bet.

27:18 Yeah. But it's super fun set of problems to work on, by the way, and really different from Ruff

27:26 too. Yeah. Ruff is more like a compiler. I would say we have a parser and a lexer,

27:35 all that kind of stuff. And with UV, it's a lot more, how do we do IO really fast? We have a lot

27:42 of networking. We're reading and writing from disk a lot. There's a lot of interfacing with

27:46 standards, a lot of interfacing with the registry, everything's async. So it's just a very different

27:54 set of problems. And it's fun to get to kind of work on both of them because they're just really

27:58 different. Yeah. I'm sure they're both fun projects and there's some really good ideas

28:02 and questions in the audience, but we're not ready for them. So I think when we spoke back in March,

28:09 one of the things we discussed is how you structured the CLI API in a way to leave space.

28:16 So for example, it's not UV install package, it's UV pip install package. And it's not UV

28:23 compile --update for dependence. It's UV pip file or whatever the command is. I have it

28:29 alias to a couple of letters and I never think about it again, but something that you've left

28:34 space in the CLI with these sub commands. And I think now we're starting to see why you left space.

28:41 Yeah. That was pretty controversial at the time and users really hated it.

28:46 I mean, they still used it, but it's so much better. A lot of people complained about it.

28:53 I even found myself saying, "Maybe we should stop doing this. Maybe we should just give up."

28:59 People were complaining and I was annoyed by it and stuff. But yeah, it's either a huge,

29:05 obviously, it's either a terrible mistake or it was genius. I don't actually know.

29:09 But what it did was, one effect, which I said before, was it immediately conveyed the rough

29:16 shape of the commands, like UV pip install. And then if you've used pip install, you know that

29:20 the next argument is a package name or a requirements file. But the other thing it did is

29:25 it left space in the CLI for these new APIs that are first-class UV APIs. So instead of doing UV

29:34 install when we released in February, it was UV pip install, which meant that today, this new set

29:38 of APIs that we launched, let's see, two weeks ago, I think today. Scroll up. Is there a date?

29:44 August 20th. So this new set of APIs that we released are UV lock, UV sync, UV run. They can

29:53 all be top level because they're really the first-class UV workflows. The pip API, by the way,

30:00 still 100% there and we're going to invest in it a lot and continue maintaining it because that is

30:07 how most people use UV. And maybe eventually we want that to be less true. We hope more and more

30:13 people use the new stuff, but we're going to keep building that. But for projects that can, we now

30:18 have this new interface that if you can fit into it, it's just sort of more powerful and does more

30:23 for you. So we saved space in the CLI exactly for this reason, which is we had a bunch of stuff we

30:29 knew we wanted to build. And if we polluted, well, not polluted is the wrong word, but if we polluted

30:34 the interface with UV install-

30:36 Clogged. If you clogged it.

30:39 Yeah. If we lost all that space to those existing commands, it would make things harder. So I think

30:44 it's worked. It's done. It's had the intended effect for us. It kept the space open. It let

30:50 people use it, conveyed how it works. And now they kind of exist alongside each other. And the pip API,

30:56 you can kind of use them together in certain ways. And when you use them together, you sort of

30:59 realize that the pip API is more low level. It's like the new APIs, you have a PyProject.toml file

31:05 with your requirements in it, and you just run UV sync and it will resolve them, create a lock file,

31:10 install them in a virtual environment. That's like one command. You could then modify the virtual

31:15 environment. You could do UV pip install, blah, blah, blah, and start making edits to it. That

31:20 kind of illustrates what I mean by it's like a low level pip APIs where you're kind of manually

31:24 manipulating things versus these higher level APIs where you tell us what your dependencies are,

31:29 and then we figure everything out for you.

31:30 Yeah, that's excellent. Just one comment on the, it's too many words or it's too long or whatever.

31:36 Like I hinted at before, I just have aliases for these. So for example, I just type PIR,

31:42 and that means UV pip install dash R requirements dot TXT. And like, I don't care if it's UV pip

31:49 that or it's pip install dash, like whatever. I don't want to type any of those. So it's just

31:52 three PIRs, let's go. And I've got three or four of those, one to make virtual environments with

31:58 UV and a couple of things like that. And one to compile the changes. And you look up for me,

32:03 I look up the docs, I go, okay, well, here's how I'm going to shorten that to something I never

32:08 think about again. So it doesn't matter.

32:10 That's totally fine. Right. That's totally fine.

32:12 But you do got to consider, I guess, the workflow for new people.

32:16 Yeah.

32:16 Maybe that's a good place to talk about the features here. So one of the things that really

32:20 so there's a lot here. One of the things that even though it's number three on the list of

32:24 four things is the Python command. So UV Python install UV Python list, all of those kinds of

32:33 things. I mean, maybe we should start there because without that we don't have any Python.

32:38 Although, yeah, so that runs sort of implicitly under some circumstances, right?

32:42 Correct. Yeah. So part of what we wanted to do with this release was make UV

32:47 sort of self bootstrapping, which sounds complicated. But the idea there is

32:53 if you don't have Python installed, UV doesn't depend on Python. It's just a binary.

32:58 So you download it. And then when you run a command like UV VM and you give us a Python

33:03 version, if you don't have any Python installed, if you don't have that version installed,

33:06 we will download and install it for you. So you can sort of seamlessly run these

33:10 commands without worrying about how do I install Python? Where's my Python,

33:13 et cetera, et cetera. Which means that we can achieve this really cool experience

33:18 where the whole set of commands you need to create a Python application are like,

33:24 you curl install UV and then you just run like UV init --Python. And we create a project

33:29 for you. We download and install that version of Python. You can just start running things.

33:32 You don't have to think about how do I install Python? How do I get it on my machine? Do I have

33:36 the versions I need? Anything like that. So you can obviously turn this stuff off, right? And

33:40 you can use your own... It also respect Pythons that you have on your machine already. But part

33:45 of the goal was we wanted to get to this, I won't call it the holy grail, but this experience of you

33:50 just download the binary, you do UV run and everything just works. So that was a big part

33:56 of it. And if you look at the full list of things that we put in this release, there's a lot of

34:03 stuff, right? There's four bullets of things that each of those could be a big release.

34:08 One is the Python installation. One is we have this sort of tool install, kind of like PIPX.

34:14 So if you want to install Ruff globally, you can do UV tool install Ruff and we'll install it for

34:20 you. It'll get put in your path, everything like that. We have these project management APIs that

34:23 I've hinted at a little bit throughout this conversation, like UV run, UV lock, UV sync.

34:27 Yeah, we'll get into it.

34:29 And we have this thing around script execution. So for a long time, I thought that these would

34:34 actually all be separate releases, that we would do a staggered series of releases. And I was

34:41 excited about that because, well, first of all, that's always fun. You kind of have a bunch of

34:44 things and you're shipping one week, one week, one week, and everyone's like, "Oh my gosh,

34:48 we're shipping so much stuff." But the thing we found was they're all kind of interconnected in

34:52 subtle ways. And the project management and the tool API, they're just not as interesting if you

34:57 don't have the Python. They are interesting, but the fact that we have the complete story of you

35:02 do UV run and we install Python, we've resolved your dependencies, we install your dependencies,

35:06 we run the command in the environment, that whole picture, everything's a little bit

35:10 interconnected. So that's how this release came to be so much stuff, is we had this vision for

35:15 what we wanted the full stories to be, and they all became somewhat interconnected. And in the end,

35:19 they're all individually, I think, really powerful things, but they kind of come together to give

35:25 this, what we want this Python experience to be. And we're not totally there yet. We're missing

35:29 stuff for sure. But this was our attempt to capture kind of a couple of different workflows

35:34 that are intertwined in different ways. Yeah, I see how they all interconnect,

35:39 but they are also interesting on their own. Yes.

35:43 The Python installation thing is interesting. And usable on their own, by the way.

35:47 Yeah, exactly. They're all independently. You just had one of those, you'd be like,

35:52 that's pretty cool. Okay, I like that. That's pretty cool. So let's talk just a bit more about

35:57 the Python one. So there's analogies here to other tools that people may know, and the closest one

36:04 for this would be pyenv, do you think? Yeah, probably. It's like, there are

36:09 obviously differences, but that's a very popular one that a lot of people know.

36:14 Yeah, yeah. I'm not saying they're identical, but it fulfills a role that a lot of people were

36:20 solving with it. Now, when I tried to do IENV, I don't remember what I was doing wrong, but this

36:25 was the early days of Apple Silicon. And maybe I had the Intel x64 version of Homebrew, but I had

36:33 the build tools for ARM. I can't remember, but I could not get it.

36:37 Trying to do something under Rosetta or something. Yeah, and I couldn't get it to install anything

36:42 with pyenv, because it would download it, it would try to compile it, then there'd be some

36:47 weird compiler bug. And I'm like, you know what, I'm not debugging the source of, I'm just not

36:51 debugging this. I'm out. I'm just going to go download it. And also, I compile Python for some

36:58 of the server stuff I'm doing, and it takes a while. It's not that fast. And when I get it from

37:04 you guys, if I say, "UV Python install 3.12" or something, boom, I get it. In your standard style,

37:11 it's fast. How is it fast? Yeah, so that comes down to a fundamental difference between what we're doing and what Py... I don't know if it's pyenv or pyenv,

37:20 by the way. I think it's pyenv, by the way.

37:22 Okay, so I'm just going to say that, because that's what it means too.

37:24 Let's go with that. If I try to do the other, I will mess up.

37:27 So the way that pyenv typically works is, it's a great tool. A lot of people use it.

37:32 It's building Python from source. And there are trade-offs around this, but the idea is

37:36 it'll build the version of Python that you need. What we're doing is we're building a top of

37:41 project called Python Build Standalone, originally created by Greg Sork. And that's a project that

37:48 we've started contributing to. So I think I did the last two releases maybe. But the idea here is

37:54 we have these standalone, easily redistributable Python builds. And when we do a release,

37:59 it will create Python builds for a bunch of Linux architectures for macOS, for Windows,

38:06 on all the different Python versions. And it will also run a bunch of optimizations over it.

38:11 So it'll do... These acronyms, it's not important that you understand them, but it can do link time

38:16 optimization or LTO and profile guided optimization, PGO. So we build all the libraries-

38:22 That's pretty impressive.

38:24 Yeah, we do. That all happens in advance. So if you go to the releases page here,

38:29 like scroll up a little bit and just click the latest release. You'll see all the... Yeah,

38:36 like 773 artifacts. That's like 773 Python builds. So it's a lot of different Pythons.

38:43 Yeah, yeah. It's a lot of different-

38:44 Yeah, 773 different-

38:46 So we build all this stuff. And then when you run Python install, we figure out the correct...

38:51 See, we have like arm, we have all this stuff. So then we figure out the correct Python for

38:54 your machine, we just download and unzip it.

38:56 I see. The same way you get a wheel is the way you get this.

38:59 Yeah. I mean, it's actually the same code pretty much under the hood for us. It's like

39:03 we stream and unzip it down into disk. And that's great. It's super fast.

39:08 It's also... They're already optimized. PyEnv by default does not compile with optimizations.

39:14 You can compile with optimizations, but it's not what it does by default.

39:18 So these will be noticeably faster than what you would get by default with PyEnv.

39:23 There are some downsides. Some people like to build from source, right? They like to have

39:28 effectively the full chain of reproducibility in the build. And now you're kind of trusting

39:32 us to give you your Pythons. So that is a downside to some people.

39:35 Yeah. But then the next thing you do is you pip install something that has a binary wheel anyway.

39:41 Yeah. No, that is true. There are also a couple of quirks with these builds

39:48 that we're working on a little bit. Some of it has to do with licensing, the fact that you want

39:53 to have kind of a statically... You want to have a self-contained Python. And there are some things

39:57 in there that require licensing changes. So it uses all... There are some slight deviations from

40:04 what you would get with building from source from CPython. And those are documented in Python

40:08 build standalone. But the general idea is we pre-build and pre-optimize these Pythons. And

40:13 then we download them on demand and it makes it really fast.

40:17 Awesome. When I go to python.org and download a DMG package or an MSI for Windows or whatever,

40:25 it runs this installer process that takes a good long while. And Jake just pointed out

40:30 similar experience I had. It says it took me less than two seconds to install 3.10.

40:33 Same thing for 3.12. So what is... Are we missing anything compared to running a

40:40 proper installer that seems to take 20 seconds even on a fast computer?

40:44 I think those installers also have to do some operations from source, most likely, because they're not shipping... I could be wrong about that. I haven't looked at

40:55 them closely. But Python in general, like Python.org, does not distribute these kind of

41:03 pre-built binary distributions for all these different architectures. And there's interest

41:09 in doing that. And I'm also interested in doing that. But there are some things that need to be

41:16 decided, standards that potentially need to be set. So it's possible that eventually we can just

41:22 grab these basically from python.org. I think our PyPI downloads come from python.org,

41:31 if I'm not mistaken. PyPI, that is. Or it actually might come from pypy.org. Now I can't remember.

41:38 Okay. Anyway, those come from some sort of official source. Yeah. Yeah. Yeah. Yeah.

41:43 So, yeah. Maybe eventually. But for now, yeah, we're using Python build standalone and we've

41:48 just been trying to grow our familiarity with the project too.

41:51 Right. Okay. So let's give people a sense here. They do not have Python at all,

41:56 or they can't count on having Python or the right version of Python. They can run a single shell

42:01 command to get UV, which UV can then manage itself. They could pipx install the UV. Can you homebrew?

42:08 Yeah. What other ways can I get?

42:09 Yeah, you can get UV on your machine however you want. Like UV doesn't need to be in the virtual

42:15 environment that it's manipulating. It can just be anywhere on your machine and it can operate

42:19 on any environment. External tool sort of deal. Yeah.

42:22 Yeah. Yeah. So once you have that.

42:23 You can install it with pip. We have our own standalone installers that you can curl.

42:29 It's on homebrew, right? You can install with pipx. You can install however you want.

42:33 And then ultimately it can operate on any environment on your machine. So I actually

42:37 don't recommend installing it in a virtual environment. I recommend installing it globally.

42:42 Yeah. That's how I have it as well.

42:43 Kind of working from there. Yeah.

42:44 I believe I've pipx installed it because anything that falls into that category for me goes under

42:48 pipx, at least for now. Yeah.

42:50 We'll talk about that in a minute.

42:52 I mean, one interesting thing is if you use our installers, like the curl installers,

42:57 then you get access to self updates. So you can run UV self update and we will update to

43:03 the latest version. That's right.

43:05 Because I tried to try that. We can't really do that if you, yeah, you can't do that if you installed through a different package manager because we don't

43:11 really know how it was installed. So if you install it through the installer, right, we

43:17 write a receipt that we understand where we installed it and all that kind of stuff.

43:21 Yeah. So that's one benefit, but it's not, you're not required to do that, of course.

43:27 Sure. I ran into that when I, I just wanted to see what it would do. I ran a self update

43:31 and it's, it said, you installed this from a package manager type thing. So go do that

43:35 thing. But that's fine. I have automation for like all those things. So it's not a big deal.

43:38 So to give people a sense, like once UV is on your system and in the path, you can say UV

43:44 VENV --Python and put up some variation of a Python version. And if you have it, great,

43:51 it'll use it right to create the virtual environment. If you don't have it, it will

43:55 then do this two second download install deal. Assuming you have fiber and then create a virtual

44:00 environment based on it. Right. Yep. That's right. And you can turn that off, but that is

44:04 the default behavior. Yeah. That's awesome. And you can also pass a commands like a managed only,

44:09 I think it is, or something like that, where you say, don't use the system Python, even.

44:14 Yeah. Yeah. Only use the ones that UV will install or like only use the Pythons that

44:19 are on my machine, ignore UVs, Pythons, or some customizations around that.

44:22 Right. Like you could do the opposite. What an advantage that is.

44:24 I will also say this version, by the way, this version format request thing is like so hard.

44:31 Yeah. It's just like, you'd just be shocked like how much engineering work has gone into like

44:37 understanding those requests and then discovering all the Pythons on your machine. And the

44:42 discovering all the Pythons in your machine part problem, by the way, is a problem that like a

44:47 bunch of tools have had to solve. And we've talked with like the VS Code team about fast Python

44:54 discovery, because there's just like so many different places that it could be. And like

44:58 on Windows, it's also like totally different than on Unix. So anyway, a lot of work has gone into

45:04 that, but it's kind of like hilariously flexible now. Like you can say, yeah, you can say CPython

45:11 greater than or equal to 3.12. Right. And then we'll look for CPython rather than PyPy or rather

45:15 than GrailPy. So anyway, a lot of work went into that. What if I just say UV, VENV and I have no

45:22 Python whatsoever? I think we would install the latest Python. Would you just go, yeah, just do

45:27 an, a latest, like maybe a, an implicit --Python space three. Yeah. I think that would

45:33 count as give me any Python from the list of downloads and then we sort by version. So I think

45:39 you would effectively get the latest compatible version, but now you're kind of quizzing me a

45:43 little bit. So I don't mean to be. I'm just kidding. All right. So we'll move on. I know

45:50 there's so much more and we're, we don't have a ton of time to get it, but I just pulled up my,

45:55 my warp terminal and I typed UV Python list and it shows me a bunch of options for arm 64 for MacOS

46:02 3.12.5, 3.12.0, 3.11.9, 3.10.3, 8 and so on that I could pick. However, I'm just wondering where

46:12 your cutting edge needle setting is going to go here. I don't see a three of 13. Yeah. And one of

46:24 the things that is tricky about any of these package manager stories is if I homebrew

46:29 it, I'll see that there's a new Python, especially a major release, but I might have to wait a week

46:34 before it's available on homebrew and, but homebrew auto updated. But if I install it from

46:38 the installer from python.org, it won't auto update it. And you know, there's just, there's

46:42 always a little drawback of it. And so when I saw this feature, I'm like, well, if this thing could

46:47 just really soon have the newest one, that would just be icing on the cake. And so what is your,

46:53 what is your policy on how quickly to adopt new things? How quickly to adopt release candidates

46:59 and betas and things like that? Yeah. Like with 3.13, I mean, I would like to have it out as

47:05 soon as there's an RC that that's what I would like to do. Our policy or our, our goal is to

47:14 make sure that it's out before the stable release. So we would like to not be lagging on the stable

47:19 release. And you know, we view like the, the minor releases in Python similarly in RUFF

47:26 historically, like we'll be like, okay, we want to make sure that we support all the 312 language

47:30 features before 3.12 is stable. And there were like a bunch of language features in 3.12. There

47:35 was like some new grammar, there were new typing features. And so the goal there again, was like,

47:40 the goal is to have it out by time it's stable. So that's typically like the contract we try and

47:44 uphold. And that's still our plan for 3.13. Okay. Awesome. Yeah. Good to know.

47:48 Jay says it would be, it would be nice to have an option to install pre-release builds.

47:54 And Tushar says the thing is Python build standalone builds 600 releases per version

47:59 number, having that run daily is too much, but if anyone could do some interesting caching,

48:04 surely the Astral folks could do some sweet caching.

48:06 It's kind of amazing though, that project runs, it's not like we don't, we don't own it. It's

48:11 not under our org. That project runs completely for free on GitHub action.

48:15 Wow. Isn't that kind of crazy, isn't it?

48:17 Yeah. GitHub is ridiculous.

48:20 Yeah. So anyway, that's kind of cool.

48:21 How much traffic it handles and yeah, how much data. Yeah. They're, they're pretty awesome.

48:26 Okay. So if you don't have Python, you can also, you don't have to do a VENV, you can just say UV

48:32 Python install, give it a version and then you'll just have that. Like that's what

48:36 Jake and I were referring to earlier, but you can also create a VENV. And this is sort of a

48:42 philosophy that I'm, I'm getting, and sorry, someone said this earlier, but it's scrolled

48:46 off the live stream comments that this prefer or require virtual environments by default,

48:52 rather than prefer to try to jam stuff into system Python or the core Python. And then

48:57 it could be a virtual environment if you really know the incantations.

49:01 Yeah. Yeah. That's probably like, you know, when we did the release in February,

49:05 we tried to steer pretty closely to the way pit behaves to make it easy for people. But we did

49:12 pick a few battles where we wanted to intentionally diverge. And this was, this was one of the bigger

49:18 ones, which is UV by default requires a virtual environment. So if you try and UV pip install,

49:23 and there's no, we can't find a virtual environment in the current directory or an

49:26 active virtual environment, we'll throw an error. And you can opt out of that by passing a system

49:31 flag, but unlike pip, right, it's opt out. So you have to, by default, it uses a virtual environment

49:37 and you can escape from that if you want to intentionally. So we're trying to like with

49:44 these APIs too, like these are all virtual environment. Well, the tool stuff is, it is,

49:50 but in a different way, like UV run, UV lock, UV sync, these all create a virtual environment

49:54 in the project directory where the project is defined by a Py project. So if you run UV run

49:59 in that directory, it will look at the dependencies, resolve them, create a lock file,

50:04 create a virtual environment, install their locked versions into the virtual environment,

50:07 run the command. And it does that every time, but it's so fast that we will hopefully so fast

50:13 that we can do it. So like if the dependencies haven't changed, it will still make sure that

50:17 everything's up to date. So if you just use UV run, we try and keep the whole environment

50:21 in sync for you. And you don't have to think about how do I manage my environment? How do

50:25 I activate it? How do I install stuff? But we're pretty like, we're pretty pro virtual environment.

50:31 And one thing I want to do, and it's sometimes hard to hold these lines because you have users

50:38 who come to you and want something really different. And it can be hard not to say yes

50:44 to a lot of the things that users want. But some lines, I think we have to hold because

50:49 if we concede on them, sorry, I'm making it sound like a battle, but if we concede on them,

50:55 it just limits the things we can do in the future. Like if we make things too flexible,

51:00 it limits some of the possibilities of what we can do in the future. And like one thing that I

51:04 kind of want to change the perception around is, I think virtual environments are, they're here to

51:10 stay really. And so we embrace them, but I want to change some of how people think about virtual

51:18 environments. In particular, I want to change this whole idea of activating a virtual environment.

51:22 I want it to feel more like node modules, which people don't necessarily think highly of node

51:28 modules. I think when people think of node modules, they think of a huge node modules folder

51:31 with like a ton of bloat. But the idea there is like, when you're in a project, you just sort of

51:36 run commands and all the packages just get installed there. And it just runs in the correct

51:41 environment with the correct dependencies. And that's actually what I kind of want. That's

51:44 actually what I want Python to feel like is, yeah, there's a virtual environment, but it's

51:47 actually just kind of like the directories. It's kind of just like full of packages, full of your

51:51 dependencies. And you just use the right one at the right points in time. So that's part of the

51:56 shift that I want us to move towards a little bit. And it's part of why we're a very virtual

52:01 environment first and why we put the virtual environment right in the project in a specific

52:05 known place, because we want to get away a little bit from this idea of where's the environment?

52:10 How do I activate it? Do I have the wrong one activated? I want it to feel a little bit more

52:14 like it is just sort of native to how the project works. If you're in the context of that thing,

52:20 then it just uses the right one. Do you have intentions to do something? I can't remember

52:24 the path, but something to the effect where if you're in a directory and there's no PyProject

52:29 TML or virtual environment, but if you went up one or two, if we're both, would it fall to that

52:33 one or would it just go? We do that with UV run. So with UV run, we sort of find the root of

52:40 the project. We'll look up at parent directories. But if you just run Python, blah, blah, blah,

52:48 obviously we don't hook into anything there. And so, like with a lot of other things, like with UV

52:53 run, UV lock, UV sync, we create a virtual environment. If you use our commands, it's

52:57 kind of just like everything just works. But you can also just activate the virtual environment and

53:01 do whatever you want, because we're kind of trying to embrace the fact that virtual environments are

53:05 ever knows, not ever knows how to use them, but people are familiar with them. Editors and tooling

53:11 revolve around them. So things that try to eject from virtual environments tend to cause a lot of

53:19 problems for editors and workflows. So part of embracing virtual environments, I think again,

53:23 is meeting the ecosystem where it is a little bit and being like, okay, the fact that we use

53:28 virtual environments and they're just in your project means that you don't need a special

53:32 editor integration to use UV. PyCharm will just let you use that virtual environment,

53:37 just a normal virtual environment, in this perspective. So that's, again, part of what

53:41 we're trying to do is build things that just work for people and try and embrace the standards and

53:47 the conventions while also maybe influencing or changing parts of them that we think could be

53:53 improved or could be made more user-friendly. Yeah. I don't know why I didn't book you for

53:58 a two and a half hour podcast, because that's what we're going to need. So Vincent from Calm

54:02 Code says, "UV run is such a game changer. He may work on a Jupyter runner for that this week.

54:08 Feels like such a game changer," which is awesome. Cool. Yeah. There's a lot of cool ideas too around

54:12 this sort of... Sorry, go ahead. Well, I was going to say with that in mind, let's dive into

54:17 this whole project management stuff here. Right? Like this is, I think this is one of the three

54:23 big pieces of functionality that comes through. And you've been talking about this sort of,

54:28 some of the features like UV run and so on, but let's talk through, let's talk through...

54:32 Yeah. I want to create a project. I don't have the Python installed. It's a new project. I want

54:40 to create a pyproject.toml, but I don't remember the format for that. Let's go. How's that work?

54:46 So we have a series of commands that kind of manage the project lifecycle for you. So you

54:51 can run UV init to create a new project. And it has a couple of flags based on whether you want

54:57 to build a library or a command line application, whether you want it to be structured as a Python

55:02 package or just kind of, maybe it's just an application that you're running on your machine.

55:07 But UV init will create the project. You can then run UV add, flask, uv add, FastAPI, whatever,

55:12 to add dependencies. And when you run UV add, it will automatically generate the lock file

55:17 and sync the dependencies to your system. So we do have commands like uv lock to generate the

55:23 lock file and UV sync to download the correct dependencies into your environment. But you can

55:28 also just kind of use these lifecycle commands and UV will make sure that everything's in the

55:32 right state as you go. So you can run UV add, UV run. If you do UV run and then Python, blah, blah,

55:38 blah, what happens behind the scenes is we resolve the dependencies, we make sure the environment's

55:44 up to date, and then we run that command in the environment. So the whole lifecycle of working

55:48 with your project can be managed through UV init, UV add, UV remove, UV run. There's a lot that goes

55:55 into that. And there's also a lot of kind of powerful features built in here too. Like when

56:01 you run UV run, it accepts a flag --with. So you can do UV run --with and put a

56:07 dependency there. And it will basically just download that, put it in an ephemeral virtual

56:13 environment and make it available for the command. So you can just pull in one-off dependencies as

56:18 you need to run things. Like I never actually think about activating environments anymore.

56:23 Like most of the time when I'm running Python commands to run one-off tooling, I'm running

56:28 UV run --with requirements and I give it a requirements file. And then I pass the name of

56:33 the file I want to run. And it just builds the thing you need and runs the command and then

56:38 throws it away. And because you have, again, we were talking at the beginning of this episode

56:42 about how really fast tools change the ergonomics. Part of what we're trying to build here is if we

56:48 have this packaging system, that's really fast, especially for like repeated operations. Like

56:53 I need to keep pulling in, I've downloaded this package before and you download it again,

56:58 use it for this one command. Like we can just like throw away virtual environments and create

57:01 new virtual environments in like in milliseconds. Yeah. If this was a Python dash M V E and V sort

57:08 of, it would be way too much overhead. Yeah. Yeah. And so we can just like, even like a bunch of

57:16 dependencies, like as they've been installed in your machine already, it's really, really fast

57:19 just to pull these things together, use them and throw them away. So there's just a lot, there's a

57:26 lot of things that we can do. And I think even like looking forward to the future, there's a lot

57:30 of things we can do because we have this like performance foundation. Like, you know, I'm

57:35 thinking about what if you want to test code on like a bunch of different Python versions. Yeah.

57:39 And maybe you don't want to like have to keep destroying and recreating your virtual environment

57:44 or something like that. Like there's just, there's a whole lot of stuff that we can build

57:48 that would be really hard to imagine if you didn't have a really fast tool.

57:51 Yeah. It's a chain. It definitely changes what's possible.

57:54 Too sure out there, put it on an example that it's very near and dear to my heart.

57:58 You know what would be great? If UV could build standalone binaries and embed Python into them.

58:04 Yeah. It's not something we've built anything around, but yeah, I know it's being described

58:10 there. Yeah. I think one of the, what I've seen most of the things do is we'll do something like

58:16 zip up a virtual environment or site packages or something and then put that in a binary. And then

58:22 when you run it, it'll spit that out or maybe do a memory mapped sort of magic to try to map into

58:28 that and unzip it into memory or all sorts of weirdness. And I think really for this to be

58:33 easy, some somewhere along the way, I could be wrong, not a core developer, but I feel like

58:38 Python's import behavior needs to be able to import from an embedded resource in a compiled binary

58:47 rather than sticking to binary and have the binary do like weird things where it fakes out the path.

58:52 Cause there's always like something that falls apart. But if the pip itself, like, okay, we can

58:56 import from a memory stream and not just from file system, then these things would be like, oh, well,

59:02 if that's the case, here's how you just embed those. And off it goes, you know, it'd be more

59:05 like go or say, here's your one thing you run it. You know what I mean?

59:08 Yeah. Yeah. There's definitely interesting stuff to explore there. And there's some

59:14 interesting tools that exist already to try to facilitate some of those things.

59:17 Yeah.

59:18 It's also, we've invested time in yet though.

59:20 Yep. All right. Well.

59:21 Yeah. I was thinking PyApp is similar. Yeah.

59:23 Yeah. PyApp is definitely, I've used PyApp before. Actually I have an app running in my system right

59:27 now. It's in a little menu bar thing. That's a PyApp app and it works well. It does work well.

59:32 Yep.

59:33 Okay.

59:34 So the project API is kind of like, you know, if you've used like poetry or PDM,

59:40 like these tools that are oriented around working on a project and you have a PyProjectTOML that

59:45 defines your dependencies and you might have some sort of structure. It's based around that kind of

59:50 workflow of working on a project, which, you know, is that's like one way that people work with

59:55 Python, but there's a lot of ways, there's a lot of things that people do with Python, which is

59:58 why we have a couple of different APIs. Right. So we also have this tool API, like UV tool install,

01:00:05 UV tool run, which we've aliased to UVX. These are for running.

01:00:10 This is interesting. Yeah.

01:00:12 Yeah. These are for running like one-off, not one-off commands, but like things that you want

01:00:16 installed globally. So like you might want rough installed globally on your machine so you can run

01:00:21 it on a bunch of different projects. We call those like tools and you install them, you know,

01:00:26 similar if you've used PipX before, we install them into dedicated environments

01:00:30 and put them on your path. And there's a lot of really cool stuff here. Like I now use UVX all

01:00:37 the time when I want to run things. So I'll do like UVX rough check and that translates to UV

01:00:43 tool run rough, you know, run the check command. And behind the scenes, what it does is it finds

01:00:47 latest version of rough, it creates an environment with that version, and then it runs the command

01:00:52 using the rough binary in there. And there's really no overhead to that if you're kind of

01:00:58 running it repeatedly. So, you know, this is oriented around like a different way of working

01:01:05 with Python, where you have these tools that you need to run on your machine and you use them for

01:01:09 different things. So we have like the project interface for working on a project. And then

01:01:14 we have the tool interface for running Python, you know, command line tools.

01:01:18 Yeah. So UVX run, well, UVX is the run version. You've also got the install. I have some thoughts

01:01:26 of when I might do one or the other, but UV tool install seems like something you might want to run

01:01:31 and have around and have auto-complete in your shell. So I could type R U tab and get rough or

01:01:37 I could UV tool install.

01:01:39 I think for things you're going to run repeatedly, it makes a lot of sense to install them.

01:01:43 Also, if you have like, if you ran UV tool install rough, and then you ran like UVX rough,

01:01:50 we would use the installed version. So if you already have the tool installed, we'll still

01:01:54 pull the installed version, assuming it's compatible with the version request that you made.

01:01:59 So like, yeah, I generally like UV tool install things that I'm going to run, you know, multiple

01:02:04 times. The other really nice thing about UV tool install is we have UV tool upgrade. So you can do

01:02:11 like UV tool upgrade --all, and we'll upgrade all the installed tools. So you don't

01:02:15 have to think about like, which tools do I have installed? Like what versions are they at? Are

01:02:19 they out of date? So it kind of gives you a way to manage all the Python tools that you're using

01:02:24 at a kind of global level. So, you know, we have a lot of commands like in our docs,

01:02:29 we have a lot of like automation tooling built around Python, like, you know, rough and UV

01:02:33 are written in Rust, but a lot of our like documentation tooling, like we have a lot

01:02:36 of scripting in Python. And all of our docs now in the doc string at the top have like the UVX

01:02:43 command that you run to like run the script. Nice. And so I just like copy that and paste that and

01:02:48 then the script just runs with all of its dependencies. So it's like, it's just super.

01:02:53 It's just very, very convenient. I think like all these are API's and the way they come together.

01:03:00 And there's a lot I know it's a lot of different stuff. And that was actually my biggest fear with

01:03:06 this release was it's hard to succinctly explain what this release is, because it's a lot of

01:03:12 different things. And it's not like when we released UV in February, when we released UV

01:03:16 in February, it's like, this is a new tool. If you know pip, you understand what this is, right?

01:03:21 It was much easier to explain, I think, in a succinct way, like what we're doing. And in this

01:03:27 release, it's like, it's actually like a lot of different stuff that comes together to tell what

01:03:30 I think is a very coherent story. But it's hard to succinctly explain exactly what we did. And

01:03:39 I was very, like the response to it, I was really heartened to see that people were like getting it

01:03:48 and they were getting different parts of it. Like different parts of it were speaking to different

01:03:51 people, because I actually think different parts of it are like relevant to different ways that

01:03:55 people work with Python. So a lot of people were excited about the lock files, the UV run. A lot

01:04:01 of people were excited about the fact that we have this like scripting API, so you can like have

01:04:05 these standalone scripts where the dependencies are embedded in them. And if you do UV run that

01:04:09 script, we just resolve the dependencies, download them and run the script with those

01:04:16 dependencies available. So you can have these like hermetic Python scripts. We had the Python

01:04:21 install API. So there were just different things that were resonating with people, which I think

01:04:24 is what made me happiest in the response to it. Did you read, I'm sure you read Armen's,

01:04:29 Rye and UV, August is harvest season for Python packaging. Yeah. Yeah. That's a pretty positive

01:04:35 take and a pretty wide ranging recommendation there. It's pretty interesting. Yeah. The

01:04:41 relationship with Rye is something we get a lot of questions about. And I think like when we took

01:04:48 over Rye, the goal was always like, we want to build UV up until to be a suitable replacement

01:04:55 that people can migrate from Rye to UV. And we're getting further along that path, but there's still

01:05:02 like things we're missing. We don't plan on like deprecating Rye anytime soon, but all of our like

01:05:07 new feature development is really happening in UV and Rye it's mostly like bug fixes, like keeping

01:05:11 things up to date. So, Rye has actually benefited a lot from UV because it uses UV under the hood.

01:05:18 So like Rye has just gotten like there's new features or it's gotten faster. But our goal

01:05:25 is really to like find the things that are missing from Rye and make sure we can support them in UV.

01:05:29 Yeah. Last one we've got to cover here is, Yovheni points out single file scripts with

01:05:35 dependencies. That's the last one, the script execution. So yeah, this is a huge problem.

01:05:40 One of the reasons I think Tushar was suggesting it, and I'm also really excited about it is if you

01:05:46 could bundle your code into an application, one of the benefits is you don't have to have a

01:05:51 conversation with everyone that runs it. How to do virtual environments, how to do package management,

01:05:56 all this kind of stuff. You just say run, click it or whatever. And somewhat short of that is I

01:06:01 want to give you just a script of some sort, but and just say run this, right? Here's the script

01:06:08 of app, you have an example.py to run that. But if you put some sort of magical incantation, which

01:06:14 it's backed by a PEP to say what it actually requires, then UV --script something,

01:06:21 it'll just see that and go, okay, or UV run rather. It'll see that, install the pieces

01:06:25 ephemerally, right? And run. That's pretty cool. Yeah, exactly.

01:06:28 So as our kind of our last thing before we run out of time.

01:06:31 It's really cool. So this is backed by a pep. I think it's 723, if I recall correctly. So this

01:06:36 is like a standardized, this syntax at the top of the file is standardized. Might not be mentioned.

01:06:42 Somewhere. Yeah. There's a bunch of PEP conversations, but not the one.

01:06:45 Yeah. We sometimes try and like shy away from talking about user facing features in terms of

01:06:51 peps. Yeah.

01:06:53 It's because-

01:06:54 There you go, 723, that's it.

01:06:55 Yeah. Okay. Well, there it is. But anyway, the, so the idea here is like,

01:07:00 you can declare the dependencies for the script, right in the script. And then when you do UV run

01:07:05 that script, we read that dependency data and we run the script in an environment with those

01:07:09 dependencies. So the other, I think really cool piece here is I talked before about UV add. So

01:07:19 if you're in a project with a PI project, Tom, you can do like UV add best API or whatever,

01:07:25 but you can also do that for scripts. So if you scroll, if you look for like UV add dash dash

01:07:30 script, so you're referred to this like magical incantation at the top. We will actually like

01:07:34 create that for you. So if you do UV add --script, and then the script name and the

01:07:39 dependencies, like we'll actually just like put that stuff in there for you. So you don't actually

01:07:43 have to like remember how to do that. Like you can just do UV run --script, and we will

01:07:48 actually like populate that field. So you can create a script, you can manage the dependencies

01:07:53 in it with UV, you can run it. This is like super useful. There's some interesting conversations

01:07:58 happening now too around like, what if we want to have a lock file for these? Because it's a little

01:08:04 bit different, right? This is the list of dependencies. It's not the list of resolved

01:08:08 versions. So if you wanted to have like full reproducibility, you probably want a lock file.

01:08:12 And not just these things versions, but they're the transitive closure of all the dependencies.

01:08:19 Everything they depend on. Yeah. So, you know, we do that for projects. Like if you do UV lock

01:08:27 in a project, we create this lock file. That's all the transitive dependencies. It locks the

01:08:31 exact URLs, the hashes, the versions, everything. We don't do that right now for scripts just because

01:08:36 there's no, there isn't really a place to put it. But we're kind of thinking about.

01:08:40 Have you considered the Windows registry? I mean, you could always just like integrate that and

01:08:44 import that over to Mac and Linux. That'd be fine. Yeah. Just put everything in the Windows registry.

01:08:49 Yeah. That's a winning idea. No, I totally get it.

01:08:54 Yeah. I mean, the other thing that's interesting too is like the notebooks, like people are

01:08:56 thinking about how can we add this kind of functionality to notebooks and you know,

01:09:02 being able to declare the dependencies for a notebook, right in the notebook, being able to

01:09:06 lock them too. It's a lot of cool stuff that we can build now that we have all these primitives

01:09:10 connected. Yeah, for sure. I mean, you could do something like take a hash of the file and then

01:09:14 use that hash as the name and as long as the file doesn't change, it'll run the same. I don't know.

01:09:18 Yeah. Yeah. I'll leave that to you all.

01:09:21 So I think, I mean, we didn't even talk about the speed for some of these things and there's,

01:09:26 there's a whole lot of stuff here, but. There's a lot of good stories to tell about

01:09:30 how we made some of these things. The problems we had to solve to like really some of this stuff,

01:09:35 but they'll make for, you know, if I dream, they'll make for good blog posts. In reality,

01:09:40 they'll probably make for good Twitter threads. Absolutely. All right. Well, Charlie, thanks for

01:09:46 being here. Maybe give us a final thoughts and words for folks who are interested in this,

01:09:51 this new layer of UV. Yeah. I mean, I think like this is, this release is really different from

01:09:58 the previous release in part because we designed and built a lot of stuff that's pretty new for

01:10:02 people and also requires people to, to change their workflows in a lot of cases. Like the

01:10:08 previous release, it was like, this is a lot like pip, so kind of just drop it in. And now it's like,

01:10:12 here's a bunch of ways to use this tools. If you were, you, it does require you to work a certain

01:10:17 way and kind of think a certain way, but we think it's really powerful. And that's all stuff we're

01:10:21 like kind of trying to figure out. Right. So we're very eager for, you know, I'm sure, I'm sure all

01:10:26 this stuff will change and I'm just eager for like, like every release. I'm just excited to have it

01:10:31 out, have people testing it, have people using it so we can get feedback. And so, you know, I, I

01:10:36 would love for you to try it out. I'd love you even more for you to come file issues about the

01:10:40 things that don't work so we can make it better and better. But I think we have a really cool

01:10:47 foundation here. We can build a lot of things that hopefully make working with Python a lot,

01:10:51 you know, a lot easier in the sense that the tools kind of get out of your way rather than

01:10:57 getting in your way. So yeah, that'll be our focus for the next, for the next while is, is seeing how

01:11:02 people are using this stuff and seeing where we can make it better. Well, I think it's pretty

01:11:06 awesome what you all are up to. Big fan of UV. So happy, happy to see the project. Thank you.

01:11:11 Progress here and always happy to have you on. So thanks. Thank you. Yeah. See you later. Bye.

01:11:15 This has been another episode of Talk Python to Me. Thank you to our sponsors. Be sure to check

01:11:22 out what they're offering. It really helps support the show. This episode is sponsored by Posit

01:11:27 Connect from the makers of Shiny. Publish, share, and deploy all of your data projects that you're

01:11:31 creating using Python. Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards,

01:11:39 and APIs. Posit Connect supports all of them. Try Posit Connect for free by going to

01:11:44 talkpython.fm/posit. Want to level up your Python? We have one of the largest catalogs of Python

01:11:52 video courses over at Talk Python. Our content ranges from true beginners to deeply advanced

01:11:56 topics like memory and async. And best of all, there's not a subscription in sight. Check it

01:12:01 out for yourself at training.talkpython.fm. Be sure to subscribe to the show. Open your

01:12:06 favorite podcast app and search for Python. We should be right at the top. You can also find

01:12:11 the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on

01:12:18 talkpython.fm. We're live streaming most of our recordings these days. If you want to be part of

01:12:23 the show and have your comments featured on the air, be sure to subscribe to our YouTube channel

01:12:28 at talkpython.fm/youtube. This is your host, Michael Kennedy. Thanks so much for listening.

01:12:34 I really appreciate it. Now get out there and write some Python code.