Algorithms for high performance terminal apps

00:00 In this episode, we welcome back Will McGugan, the creator of the wildly popular Rich Library and the founder of Textualize.

00:07 We'll dive into Will's latest article on algorithms for high-performance terminal apps

00:12 and explore how he's quietly revolutionizing what's possible in the terminal.

00:17 From smooth animations to dynamic widgets and full-fledged 2E, or should we say GUI, frameworks.

00:23 Whether you're looking to supercharge your command line tools or just curious about how Python

00:28 can push the limits of text-based UIs. You'll love hearing how Will's taking a modern,

00:32 web-inspired approach to old-school terminals. This is Talk Python to Me,

00:37 episode 498, recorded February 4th, 2025.

00:42 Are you ready for your host, please?

00:44 You're listening to Michael Kennedy on Talk Python to Me.

00:48 Live from Portland, Oregon, and this segment was made with Python.

00:54 Welcome to Talk Python to Me, a weekly podcast on Python. This is your host, Michael Kennedy.

00:59 Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython,

01:05 both accounts over at fosstodon.org, and keep up with the show and listen to over nine years of

01:11 episodes at talkpython.fm.

01:12 If you want to be part of our live episodes, you can find the live streams

01:16 over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get

01:21 notified about upcoming shows. This episode is sponsored by Posit Connect from the makers of

01:27 Shiny. Publish, share, and deploy all of your data projects that you're creating using Python.

01:32 Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards, and APIs.

01:39 Posit Connect supports all of them. Try Posit Connect for free by going to talkpython.fm

01:44 slash posit, P-O-S-I-T. Will, welcome back to Talk Python To Me. It's great to talk with you.

01:49 Thank you. It's good to be here again.

01:51 It's going to be really fun.

01:52 You know, I've built some really cool apps with Rich, and we're going to talk about Rich and Textual.

01:58 Probably a little more on the Textual side this time around, but still a very, very cool revolution you're bringing to the terminal.

02:07 Yeah, a very quiet but pretty revolution.

02:10 I like it.

02:11 Indeed.

02:12 Okay, so we want to talk about Textual.

02:15 You wrote a cool article called Algorithms for High-Performance Terminal Apps,

02:19 which I thought would be fun to dive in, but we're going to also keep it more of a broad-ranging conversation.

02:24 At the start of all that, you know, it's been a little while since I've had you on the show.

02:29 Again, tell people who maybe haven't listened to that episode or episodes

02:33 who you are.

02:35 Sure. My name is Wilmer Guggen.

02:36 I'm a software developer, Python developer, probably like most of your listeners.

02:41 I achieved a certain amount of infamy when I built this library called Rich,

02:47 which a lot of people use.

02:49 If you see some fancy output in a terminal, that might have been me.

02:53 And I founded a startup called Textualize, which is all about improving the developer experience

02:59 in the terminal.

03:00 And I'm sure we'll discuss Textual, which is my other Python library in this show.

03:05 Yeah, absolutely.

03:06 I'm super excited for it.

03:08 And is this close to the one-year anniversary for Textual?

03:13 It was an anniversary of some sort because we're going to talk about one of your,

03:17 one of your articles.

03:19 It's actually closer to three years since I founded Textualize.

03:23 Oh, no, you know, it's not, it's the 1.0 miles.

03:26 Oh, right, yeah.

03:27 1.0 year, the 1.0 version.

03:29 There we go.

03:30 That's what I was thinking.

03:32 Yeah, so Textual follows Semver, which I imagine people are familiar with.

03:37 When you're on the zero version, you have a bit of flexibility and you can make breaking changes.

03:42 We try not to, but you can make breaking changes and it gives you a bit of freedom

03:45 early on to play around with things.

03:48 But some people stick to zero ver just forever, which is fine if that's what you do.

03:54 But when you go to 1.0, it advertises to people that this is stable.

03:58 You can rely on the interface not changing too quickly.

04:04 And if there are any changes, it's well documented.

04:07 If you followed along, it's a very modest release from the last zero version

04:11 to the 1.0 version.

04:13 But it says to the world, hey, everything textualized is ready.

04:16 You can start using it in production, even though you could probably have been using it in production for a year.

04:22 I'm sure that you could have.

04:23 I'm sure you could have.

04:24 Are you familiar with zerover.org, the numeral zerover.org?

04:31 Have you seen this?

04:32 No, I didn't.

04:33 I thought it was just an aspect of Semver.

04:35 I didn't realize it was.

04:36 It is absolutely an aspect of Semver.

04:39 But this, I can't remember who put this together.

04:41 Does it say at the bottom?

04:43 Let's see about the About page.

04:45 published yeah that's what i thought i thought it was mamoud hashemi amazing yeah so mamoud put

04:50 this together and basically it calls out i think it's it's a little contrarian to you know there's

04:57 a lot of especially larger companies that have rules about what version you're allowed to use

05:03 and is software ready enough if it's a 0.3 we can't use it it's not ready not until it's a 1.0 or a 2.0

05:09 or something along those lines, right?

05:11 And so this is the popular projects that are still really, really low in their versions.

05:19 It says, welcome December 0.0.1.

05:23 And if you go down here, it says, like, look, React Native is 0.78.

05:27 Right, right, yeah.

05:28 Even though it's 10 years old.

05:30 Hugo, which I use for my blog and love it, it's got 77,000 GitHub stars.

05:35 It was first released 12 years ago.

05:38 More than a decade long.

05:40 It's still zero dot something.

05:43 Let's see if there's some that are really extreme here.

05:45 I think the problem is that non -software developers have a different idea of what versions mean.

05:52 We know that they're about compatibility.

05:56 But when you're marketing it, 1.0 is like you advertise it, you make a splash about it.

06:01 That's when it's actually out.

06:02 You can use it.

06:02 Otherwise, it's just like a toy, right?

06:05 And then, boom, 2.0.

06:07 Yeah.

06:07 Yeah, let's see.

06:09 NeoVim, 0.10.

06:11 scikit-learn, it's 15 years old, it's 0.17.

06:15 So I think I totally hear what you're saying, right, about like, yes, this, and I think we both agree, like, what this should mean.

06:22 If your thing is 0.17, probably that means there's a good chance that there might be some adaptations to the API.

06:31 at the same time if you're scikit-learn and you've had you've been around for 15 years yeah it might

06:37 be time to bump that to something over 0.17 you know what i mean what do you think it's

06:42 i think it's it's the um developer-led projects the ones which are just a bunch of developers releasing

06:49 at open source uh we developers don't like to make too much promises exactly if we were to put a 1.0

06:55 on it who's going to be on the hook for making that 1.0 and then they're stuck with it right

06:59 Exactly. We know that 1.0 is not going to be perfect and bug-free.

07:05 So we don't necessarily want to be on the hook for it not being perfect.

07:10 Yeah, I know. At the same time, you're on the hook to not change scikit-learn at this point anyway.

07:16 You know what I mean?

07:16 And I'm only picking on them because I know they're so well-loved in the community and they're very, very common.

07:22 Unlike, say, Ruff, which is on this list, but Ruff is pretty new.

07:26 Anyway, I think I'll put this in the show notes.

07:29 Zero over.

07:30 Thanks, Mommy.

07:30 That was really fun.

07:32 So let's talk about terminals.

07:35 Terminal revolution.

07:36 I think it's a pretty interesting time for terminals, both the application that runs your terminal

07:44 as well as closer to what you're doing, building applications that are like CLI apps or 2E apps,

07:52 as we'll coin the term.

07:54 No, you've coined the term.

07:55 We'll tell people about the term if they don't know, right?

07:58 I mean, we've got Warp, we've got Ghostity, I don't know how you say, Ghostity, whatever.

08:04 A bunch of other options coming along for terminals that make being in the terminal nicer.

08:10 And I think also draw people into wanting to build apps that run in that space rather than try to get away from it, right?

08:17 It's not macOS where it's just the default Z shell with no history and it's like white with weird fonts

08:26 or CMD on Windows where you're like, what is this thing, you know?

08:30 What are your thoughts on this?

08:31 I mean, first of all, like maybe just on the apps that run, do you have a favorite terminal app that you run?

08:36 I've been using iTerm for the last few years.

08:39 It's kind of my daily driver.

08:41 But there's a few other good ones.

08:42 There's Alacrity and Kitty and then Ghost TTY or Ghost T.

08:47 I'm not sure.

08:47 That's a new kid on the block.

08:50 And it is very good.

08:51 It is very good indeed.

08:52 but I like I terms feature set it's not the fastest but it's fairly reliable

08:57 and it's got lots of other features I played around with I played around with

09:03 warp.dev as well I think that's a little too unconventional for me at the moment

09:11 they've done some very cool stuff though there's lots of innovation there

09:14 it takes some good news to you, that's the one I've been using last year or so and I really

09:19 enjoyed it, it does some very cool things like For example, if I'm on my computer and then I SSH into another computer and then in that computer I Docker exec something into a Docker

09:29 system, a Docker container, Warp will say, would you like to apply the Warp shell to this Docker container instead of whatever does it?

09:39 It might not even have Bash, right?

09:40 It might just be SH.

09:42 You just hit a button and it goes, whoop.

09:43 It adapts and basically become kind of like in memory, installs itself into that container temporarily, right?

09:50 that sets all the settings and stuff.

09:52 So it's a long ways from cmd.exe.

09:55 I guess it's worth pointing out to people out there on Windows, if you're not using Windows Terminal, please,

10:02 right?

10:02 You need to use Windows Terminal.

10:05 It's such an improvement over...

10:07 Massive improvement.

10:09 I mean, you probably run into this all the time, right?

10:12 Yeah, and the solution, if someone tries something in the terminal they use is retro textual and it doesn't work properly,

10:18 I first say, we try downloading Windows Terminal.

10:21 And 99% of the time, it not only fixes it, but it makes it look beautiful.

10:26 I mean, hat's off to Microsoft.

10:29 They didn't change the terminal for decades.

10:32 But when they did, they made a very good job of it.

10:34 Yeah, it took a while.

10:36 I don't know if it's the default yet.

10:38 You know what I mean?

10:39 I'm not sure what's actually presented if you kind of pull up the start menu.

10:44 It's a very strange architecture.

10:47 You can pull up cmd.exe.

10:50 That's what I've been used to calling it because it has it in the title bar.

10:53 If you send it some magic codes, you can switch it into essentially Windows Terminal.

10:58 It'll process escape sequences like it does on Mac and Linux.

11:05 So the actual window you see is just kind of like a window onto the thing which renders the terminal.

11:13 So you can get Windows.

11:14 You might already have Windows Terminal even if you don't get the experience.

11:18 But software can switch it into this mode, which makes it work really well.

11:23 I had no idea. That's cool.

11:25 Okay.

11:25 Yeah, it's probably the same core for both of them or something, right?

11:28 Yeah, exactly.

11:29 The architecture is very component-based and you can switch it at runtime.

11:35 Yeah, that sounds very Microsoft.

11:37 Is there a code that you can put to embed?

11:39 Like if you can split the panes and then you can embed Excel on one side maybe

11:43 and then SharePoint on the other?

11:44 I haven't found it.

11:45 I'm just kidding.

11:47 I wouldn't be surprised, yeah.

11:49 I wouldn't either.

11:50 I wouldn't either.

11:52 Are you a fan of modifying your shell, I guess?

11:56 You know, like, oh, my ZSH or oh, my Posh or Starship or any of these?

12:03 Yeah, to change the prompt.

12:05 Yeah, I do have, I can't remember which one I use now.

12:08 It's not, is it Starship?

12:09 Yeah.

12:11 Yeah, it's fairly minimal.

12:12 It tells me the branch, the Git branch I'm working on, the version of Python that's running,

12:18 and the version of the application I'm working on.

12:20 So that's really nice.

12:22 Oh, it'll tell you the version of the application you're working on?

12:24 Yeah, it says v1.0.0, so it's somehow pulled out the version from Textile, I guess.

12:29 Oh, wow. That's pretty cool.

12:31 I guess if it's in a repo and knows that repo is a package, it can figure out what version it is, but that's still pretty neat.

12:36 I guess. I must go on the spec.

12:38 PyProject.toml, I'm guessing.

12:40 Yeah.

12:41 Yeah, I don't know. It's magic.

12:42 and it works.

12:43 So don't delve too deeply.

12:46 Yeah, the whole reason I'm bringing this up, it might feel like a little bit of a diversion,

12:50 but when people think about terminal apps and just being in the terminal

12:54 and somebody says, oh God, I hate the terminal.

12:57 It's so bad.

12:59 I can't stand it.

13:00 It usually means they're not adopting some of these really magical tools.

13:05 Regardless of which one, regardless of whether you adopt Starship or Oh My ZSH or Warp or whatever,

13:11 you get a dramatically different experience, right?

13:15 Yeah, it's infinitely customizable, the terminal and the shell and various other software.

13:22 So you can get the kind of experience that you want out of a terminal.

13:25 People get very comfortable with their terminal environment.

13:29 It feels like an old pair of shoes.

13:34 It's just that comfortable.

13:36 I guess it depends what tools you use as well.

13:38 For example, if you're a NeoVim or an Emacs type person, you're in the terminal even way, way more than, say, for me, for example,

13:47 use ByCharm and other tools that I only dip in and out when I'm working on code for that kind of stuff.

13:54 Yeah, for me, I'm switching between terminal and VS Code most of the day.

13:58 So I've got both open.

14:00 I write some code and I switch back to terminal and run said code.

14:04 So I'm in both.

14:06 Yeah, but funnily enough, I never got into editors that run inside the terminal.

14:12 I do like all the fancy graphics that VS Code can give you that terminal couldn't.

14:17 Yeah.

14:17 That matters less to other people, but I like it.

14:20 You know what?

14:21 When I started out doing programming, I started out doing a lot of work on silicon graphics,

14:26 mainframes, doing Emacs on Onyx workstations and stuff like that.

14:31 And I was happy as a clam to get out of there as quick as possible.

14:35 get out of. I'm like, can I get some better tools than this? And yeah, I don't know. I'm on the GUI

14:42 side as well, but I feel like kind of like the terminal. If you don't customize your terminal,

14:47 it's not a great experience. If you don't really commit to learning all the hotkeys of your GUI app,

14:53 well, it feels like, well, I got to use this mouse. It's all clunky, but like I almost never

14:56 use the mouse. You know what I mean? Even though I'm in a GUI app, right? They just customize the

15:01 keys? I think you can get very proficient in both.

15:05 You know, I've seen people using a terminal and they use Emacs or Vim and their fingers just fly

15:11 through it and they're very proficient.

15:13 But the same thing in VS Code.

15:15 If you know your editor and you know the shortcut keys and you don't reach

15:19 for the mouse very often, you can be just as proficient, really.

15:23 At Textualize, we had a policy if you use whatever you want to use to edit

15:27 code. If you take a developer who uses Vim and then force them to use code,

15:32 they will not be happy.

15:33 And they will not be as efficient at all.

15:38 Yeah, that's a really tricky balance, right?

15:40 You want to let people work where they're comfortable, for sure.

15:45 This portion of Talk Python to Me is brought to you by the folks at Posit.

15:48 Posit has made a huge investment in the Python community lately.

15:52 Known originally for RStudio, they've been building out a suite of tools

15:56 and services for Team Python.

15:58 Over the past few years, we've all learned some pretty scary terms.

16:02 Hypersquatting, supply chain attack, obfuscated code, and more.

16:06 These all orbit around the idea that when you install Python packages, you're effectively running arbitrary code off the internet on your dev machine,

16:15 and usually even on your servers.

16:17 But thought alone makes me shudder, and this doesn't even touch the reproducibility issues surrounding external packages.

16:24 But there are tools to help.

16:25 Posit Package Manager can solve both problems for you.

16:29 Think of Posit Package Manager as your personal package concierge.

16:33 You use it to build your own package repositories within your firewall that keep your project safe.

16:37 You can upload your own internal packages to share or import packages directly from PyPI.

16:43 Your team members can install from these repos in normal ways using tools like PIP, Poetry, and UV.

16:49 Posit Package Manager can help you manage updates, ensuring you're using the latest, most secure versions of your packages.

16:56 but it also takes point-in-time snapshots of your repos, which you can use to rerun your code reproducibly in the future.

17:04 Posit Package Manager reports on packages with known CVEs and other vulnerabilities so you can keep ahead of threats.

17:10 And if you need the highest level of security, you can even run Posit Package Manager in air-gapped environments.

17:16 If you work on a data science team where security matters, you owe it to you and your org to check out Posit Package Manager.

17:23 visit talkpython.fm/ppm today and get a three-month free trial to see if it's a good fit

17:29 that's talkpython.fm/ppm the link is in your podcast player show notes thank you deposit

17:35 for supporting the show all right well i think that's a good transition to tui what is what are

17:43 tuis why do people build them i know we talked about this in the last episode we won't totally

17:48 necessarily rehash it but there are some people haven't seen it there are some incredible

17:51 applications and we will see some built in textual that run in the terminal yeah

17:57 sure so 2e stands for text user interface um i don't actually like that term anymore okay um and the reason is well

18:06 the thing is um the 2e's that you build with textual are also guis um it's a very simplified

18:11 form of graphics but we've got we've got lines um and and corners and you've got like a mouse that

18:17 can move around. So it is rounded with text, but it works just like a GUI. I guess it's a little bit like

18:25 saying a web browser is a document-based UI.

18:30 It's technically correct, but no. We're having this whole video live stream recording across the internet,

18:38 across the world in a web browser, and there's only a little text on the

18:41 screen, right?

18:42 Yeah, exactly. So the term 2 is stuck, but It's a GUI that runs in your terminal.

18:49 Well, I think originally it was pretty text-based.

18:52 It was.

18:52 The advanced version might be a progress bar, right?

18:55 Or emojis or color.

18:56 Yeah, exactly.

18:57 So those are technically text, but it looks like images.

19:01 So you've got emoji.

19:02 You can have lots of, like, I'm looking at my Starship prompt, and it's got a little snake because it's in Python and that kind of stuff.

19:09 That is technically text, but it is graphical.

19:12 and progress bars.

19:14 You create a bar using characters, but essentially you're creating an image.

19:20 So I think the distinction between text user interface and graphical user interface is fuzzy now.

19:27 If you say Tui, I'll know what you're talking about, but I just don't like the term.

19:31 I don't think it's as clear.

19:33 I'm not sure we even need the distinction.

19:35 Yeah, I hear you.

19:37 Are you familiar with Vtop?

19:38 Have you seen this app?

19:39 Yeah, that's really, really pretty.

19:41 I like the fading effect.

19:43 I mean, this app has just crazy amounts of graphics and it's all written in the terminal with live updating.

19:52 I don't know how many graphs, but many, many graphs.

19:55 Interesting thing about Btop, I think most people who run it don't run it

20:00 because they really need to keep an eye on the performance of their course and file system, etc.

20:06 I think a lot of them run it because it's retro cool.

20:09 It's retro cool.

20:11 It looks cool.

20:12 It looks like how you thought computers were going to look.

20:17 I'm going to be a programmer and a hacker.

20:19 I'm going to be a super tech guy.

20:21 This, yeah, you know what?

20:22 It could be, this could be a movie UI.

20:24 You know how when you're watching a movie, they're like, they fucked for whatever reason,

20:29 won't run macOS or Windows.

20:32 They come up with some weird variation that's got weird graphics.

20:36 Yeah, and have some dialogue like, I've got to hack the Gibson.

20:39 Okay, done.

20:40 Exactly.

20:41 I need to fill up something computer.

20:43 This is Visual Basic.

20:44 I know Visual Basic.

20:45 I know this, yeah.

20:45 I'm going to get his IP address.

20:46 It's like, what?

20:47 Yeah.

20:47 I don't know.

20:49 Oh, boy.

20:49 Anyway, I think this is a really, really cool example of pushing way, way, way beyond.

20:55 This is a graphical user interface.

20:57 This is not a textual user interface, right?

20:59 Exactly.

21:00 I mean, technically it is text, but it's also graphical.

21:03 Yeah.

21:04 Yeah.

21:04 Yeah.

21:05 So we have two things that you guys make.

21:08 We've got textual and rich.

21:10 And maybe it's worth talking about Rich for a moment since that's the foundation.

21:13 Sure.

21:14 Okay, so Rich basically formats data in a very pleasing-to-the-eye way.

21:21 You can print, say, a Python dict, and it'll format with an indentation.

21:26 It'll colorize strings and things.

21:29 And you can print things like tables quite effortlessly, syntax-highlighted code,

21:35 and you can use up to 16.7 million colors.

21:40 Essentially, it was a toolkit to prettify terminal output with as little effort as possible.

21:47 And it's all kind of integrated.

21:50 So you can put syntax-halled text inside a table or a progress bar inside a cell.

21:56 And everything kind of works together.

21:58 So it's a single coherent system.

22:00 Because some of this stuff existed prior to Rich.

22:04 But they were written by different authors.

22:06 And they weren't designed to work together.

22:09 Maybe there's a table one by one person and then a progress bar by another,

22:15 and they didn't really work the same or integrate or things like that, yeah?

22:19 Exactly, exactly.

22:20 So it meant that terminal output was still quite basic.

22:26 When color was used, it would colorize a single line or something like that.

22:31 And it still, it helped, but it wasn't great for parsing, visually parsing that information.

22:38 but Rich gave you that ability that when you write output it is readable

22:44 I think prior to that we developers kind of built up this skill of staring at monochrome

22:51 text with funny characters and numbers and somehow pulling out meaningful information

22:57 out of that and it took years to develop but hopefully now we don't need to

23:02 because you can write things to the terminal that looks much like the output

23:06 you'd get in a browser.

23:08 It's formatted and readable and colorful.

23:11 Yeah.

23:11 You know, one thing that might be worth talking about is the live updates, like live tables and stuff.

23:18 Yes.

23:18 So in Ritchie can create a live object.

23:22 That's what the class is called.

23:23 And basically it clears the screen and it writes that output and then you can refresh it.

23:30 It can do that quickly, like 20 times a second or less, depending on how much data you've got.

23:36 So you can create animations, you can create dashboards, you can create like

23:41 a scrolling list of text.

23:44 But it was kind of, it was limited.

23:46 Because all it could do is update information regularly.

23:49 It wasn't interactive.

23:50 You couldn't click or scroll or even detect key presses.

23:55 It was purely for updating information.

23:58 It's kind of like a very rudimentary predecessor textual.

24:02 Another thing that's interesting is what's the frame rate?

24:06 of a terminal.

24:08 People think of it just kind of scrolling text and printing out text, and you see it kind of going by,

24:13 and you're like, well, that's kind of how the UIs must be.

24:17 But you can actually push these things pretty far, right?

24:19 You can.

24:21 The interesting thing is I don't think anyone who sat down to develop a terminal

24:26 was thinking in terms of frames per second.

24:30 They were just, they want to optimize the text output So that if you cut a large file, it didn't take forever to scroll through.

24:39 Right, or didn't flicker around or something, yeah.

24:42 Yeah, yeah.

24:43 And later on, people like myself and others were creating animations and things.

24:49 And they're thinking, well, how fast can I push this?

24:52 And it turns out pretty fast.

24:54 Because terminals use your GPU to render the text.

25:00 And if you send a text fast enough, it'll keep up, mostly.

25:04 And I found that 60 frames a second is completely workable on most terminals.

25:11 You don't really need that for most apps because if you're scrolling text, you can only move character at a time.

25:18 It's not pictal perfect because it's kind of like a grid.

25:23 So if you want to scroll at 60 frames a second, you'd have to move 60 lines in a second,

25:28 with which text would just kind of whiz by.

25:31 Yeah, it gets blurry anyway.

25:32 Yeah, so it can do 60 frames a second, which is sometimes fun for animation effects,

25:39 but generally you don't need it.

25:41 You can update 10 frames a second, and it'll look nice and smooth.

25:46 But it's nice to know that you can.

25:47 We've got a ridiculous demo, which creates a really colorful gradient and then rotates it at 60 frames a second.

25:55 Wow.

25:56 It looks very cool.

25:57 It would be terrible in an actual app because it'd just burn your retina,

26:01 but it is possible.

26:03 Yeah.

26:04 Reminds me of the early days when computers just started having VGA and stuff

26:08 and people would come out with amazing graphical, what do they call them?

26:13 I can't remember what they call them.

26:14 Demos and things?

26:15 I think I was thinking demos as well.

26:17 Yeah, I think that is what they call them.

26:18 And we would download them off of BBSs and watch them.

26:21 We're just like, wow, it's so good on my 486DX.

26:23 I can't believe it could do it.

26:25 It's so smooth.

26:27 They were amazing, right?

26:29 Similar deal with the terminals.

26:30 The manufacturer of those devices back then probably didn't have graphical animations or games in mind when they were building

26:39 those computers.

26:40 But then you got some interested parties who were willing to experiment and they actually managed to

26:45 make it do things, which the manufacturer just never intended.

26:49 The same thing with the terminal.

26:51 No one ever sat down and created a system to display 16.7 million colors at 60 frames a second.

26:57 It was other people that just tried that and got it to work.

27:01 So there's an amazing amount of creativity and freedom in working and stuff like that,

27:06 which I kind of missed from the old days because now computers are so fast and so powerful.

27:09 I know.

27:09 People don't have to even worry about it, do they?

27:14 I want to kind of work through this.

27:16 You have this live demo, the official textual demo, which you can still...

27:22 Let me go.

27:22 I think I have it in the next...

27:23 No.

27:24 If I click on it, it tells me how to get it.

27:28 props for the install step here, the UVX.

27:32 Are you a UV convert?

27:34 Are you a UV fan these days?

27:36 I am a fan.

27:37 Yeah, I mean, that I think is a killer app that you can send someone a fairly short command line

27:43 and they can run your project.

27:45 The Python, it says --Python D.12.

27:50 That was because at the time, it wouldn't install a version of Python that it can run on.

27:56 I think Astro's working on that.

27:58 So I think in the future, maybe now, you could just do UVX text-demo, and then it would just run.

28:05 But even so, I think that's a beautiful way of allowing people to test your code

28:10 with very little barriers.

28:13 Yeah, especially the script stuff now.

28:16 You can just use the script and put the dependencies at the top and just say UV run.

28:21 I can't remember the exact syntax, but basically you can embed what it needs to run

28:25 just as a comment and it'll run.

28:27 That's excellent.

28:28 It's no harder than it needs to be, which is great.

28:31 It's very clear what it does and it works very well.

28:34 It's the kind of computing that you want.

28:36 You don't want to have to send someone a page of instructions and say, go and download this software

28:41 and then write this into your configuration file.

28:44 Yeah, that's just a headache.

28:45 I don't want to deal with that ever again.

28:47 I just want to send people what UVX, blah, and then done.

28:49 I agree.

28:50 I agree.

28:52 If you have UVX, you're good to go, right?

28:55 and you don't even have to have Python and you also don't have to worry about the version of Python.

28:59 So these are all good things.

29:00 So let's run it and let's see what happens here.

29:04 There's a lot going on in this demo.

29:06 Chris May out in the audience gives you props for making it so easy to run.

29:10 And Hugo says, I can drop, I can drop the Python.

29:15 As you imagined, I can just say UVX textual demo and I'll probably get 13.1.

29:19 Let's see, something's happening.

29:21 I think it worked if you had that version of Python.

29:24 But if you didn't have the correct version of Python, it would sometimes fail.

29:28 Got it.

29:28 Maybe they fixed it.

29:29 Yeah.

29:29 Maybe.

29:30 Maybe.

29:31 So when this thing pulls up, it looks very much like a web browser.

29:38 With the main difference, I think, that there's not a lot of pictures, as in true graphics, right?

29:44 But if you imagine like a really nicely styled website that maybe used color and emoji and CSS, but just white on graphics, right?

29:54 I feel like, would you say that's kind of the extent?

29:57 It is.

29:57 It's kind of like a minimal web interface that focuses on typography.

30:02 That's not a potential because you've only got one font.

30:04 Yeah, yeah, yeah.

30:05 Exactly, you can't.

30:06 Yeah.

30:07 But as we'll see, you can actually do graphics pretty well.

30:12 Yeah, if you see the numbers up at the top, there's nothing in the terminal that's placed.

30:18 Most people are saying they won't know, but at the top it says version, then it says 100,

30:22 GitHub stars 27.2 thousand how many forks and so on but those are colored differently

30:28 they look different okay now tell us about the numbers yeah sure so they're taller

30:32 and wider than the rest of the text but there is no alternative font in the terminal

30:38 those numbers are made up of other characters box drawing characters so you've got like a vertical line

30:44 a horizontal line you've got a curved corner you've got like a T section

30:49 and you can assemble that and make it display something that looks like numbers.

30:53 It does look like numbers.

30:56 You've got the little subscript for the thousands, the kilo stars, I don't know.

31:03 If I remember correctly, it's a two by three, no, sorry, three by three grid of characters.

31:09 And I had to hand edit this in Python by cut and pasting the box characters from Wikipedia

31:16 to assemble numbers.

31:18 And I discovered the numbers work quite well.

31:20 numbers look like numbers and i've also got hex digits so a b c d e f um

31:26 i tried to keep going but i found that some characters didn't work at all um like an x there's no diagonal cross piece

31:34 in these uh unicode characters so it's an x just became impossible with a three by three grid um

31:41 but for numbers it works beautifully yeah yeah very interesting a little bit like a classical

31:46 alarm clock where the numbers like a seven digit display.

31:50 Yes, exactly.

31:51 Yeah.

31:53 Cool.

31:54 All right.

31:55 So if you run the demo, it comes up with comments that show you or sort of a

32:02 walkthrough of what's possible in these little boxes.

32:04 And it says, look, you can go and scroll it.

32:06 You scroll with your mouse.

32:08 You know, it scrolls that section.

32:10 It says you want to go to the next one.

32:13 Hit tab.

32:14 Hit enter to open it up.

32:15 It's, even though, kind of like we were talking, even though it's a user interface,

32:20 like with scroll bars and I can grab the scroll bar and go to it and I can click on the things and so on,

32:25 it's also very keyboard friendly, right?

32:28 This is probably a super important aspect of building these types of applications in the terminal

32:33 because people expect that, right?

32:34 Yeah, very much so.

32:35 So a lot of text was inspired by web development because I was a web developer for many years.

32:41 And there's a lot of work being done in interfaces in the web world.

32:45 Lots of work, you know, loads of developers working on it for years.

32:49 I wanted to bring some of that to Terminal, but I also didn't want to lose what made the Terminal interesting.

32:55 Terminal is a very keyboard-focused thing, so I wanted textual apps to also be very keyboard-focused.

33:03 The mouse is terrific for discovering things.

33:06 You know, you can move it around and there's tooltips, etc.

33:09 You can click those links or you can press the key.

33:12 But if you want to, if you use an app regularly, you'll want to learn the keys.

33:16 So it is very focused.

33:17 You can use the keyboard for everything.

33:21 In the footer, that displays all the main keys.

33:24 You can also use tab and shift tab to move around various elements.

33:28 You can use up, down to scroll.

33:31 So you could get by without ever using the term, without ever using the mouse.

33:36 And it's there if you need it, but you don't have to.

33:39 Yeah, that's incredible.

33:41 It's got code highlighting.

33:43 It's got a lot of different things.

33:46 For people who maybe want to get started with this, I think there's two, let's say three,

33:50 three things that are worth looking at.

33:52 One, you actually, you said you were a web developer, so you must have been a fan of Vue Source, right?

33:57 Oh, yeah.

33:58 So you added, you basically added Vue Source.

34:01 That's right, yeah.

34:01 You type C, it says so at the bottom, and that pulls up the source of the current page,

34:08 just like you would at my website or any website, right?

34:10 Yeah, I want people to realize that the code behind that not that complex but textual it looks like complex um what you're viewing at the moment is actually

34:19 markdown um which you might think that's quite hard to render markdown in the terminal and well

34:24 it is but we've done it for you so um you just embed a widget and somewhere in the code it'll

34:29 say yield markdown and then it'll have the contents and it renders markdown you see there's a code

34:36 block there with syntax highlight code yeah well that's the magic of of these libraries right it's

34:41 hard once for you and then it's easy for other people.

34:44 Yeah. We suffer so you don't have to.

34:46 Exactly. I feel like the most, the biggest example of that in the Python space is Jupyter Notebooks,

34:53 right? All that interactivity and all that stuff, people are writing TypeScript and JavaScript and

34:58 all these kinds of things, you know, it's for, you know, Plotly and other plotting UI things.

35:04 And that's so Python people can write Python code and not write, you know, they're not creating

35:10 websites are just typing in a notebook and they get interactive things, right? But that's

35:14 probably the biggest take one for the team sort of deal.

35:16 Yeah, there's a lot of work under the hood.

35:20 A lot of people that use Jupyter Notebooks, that's their only knowledge of Python. They think Python is

35:26 Jupyter and Jupyter is Python.

35:28 And that's fine. It's a tool and it gives them access to all this stuff which would be very hard

35:34 otherwise.

35:35 So yeah, I think Jupyter is a great example of that.

35:38 Out in the audience, And we have David points out JPTerm.

35:43 Are you familiar with JPTerm?

35:45 Yes.

35:46 Is that David Brokert?

35:47 Yeah, yeah, yeah.

35:48 The successor of MBTerm, and it's Jupyter, Jupyter Lab in the terminal.

35:55 Yeah.

35:56 It's gone full circle.

35:59 It's like the Inception movie.

36:02 It really is, right?

36:04 But you can embed the textual app inside Jupyter.

36:07 So you could have Jterm working in a textual app inside Jupyter that's also displaying another notebook.

36:13 Yeah.

36:16 Incredible.

36:17 All right.

36:18 So if we go to the terminal where I got this running, and anyone else out there listening wants to check this out,

36:25 you just run it.

36:26 And I was starting to say there's three places you should check out before I got kind of starstruck with view code.

36:34 Okay.

36:34 So I think there's probably the best place to get started is to look at the widgets.

36:39 I don't know what you think, but just like, well, what can this framework do in terms of what are the building blocks that you can look at the widgets?

36:45 And then once you decide, okay, this is pretty interesting, I think people should check out the projects because there are some pretty interesting things.

36:52 I know we've covered the mom Python bytes that are built on top of textual.

36:56 And then just to talk about the graphics, what is this game?

37:01 This is a game in inverted commas.

37:05 it's got different names sliding tile puzzle I think is the most common thing, you've probably seen it before

37:11 you've got a square the square is made up of lots of little tiles and there's one space and you can move a

37:17 square into that one space and the goal is to reassemble the image, but for here

37:23 I've used code so you've got all these tiles with code on it and you've got to reassemble it

37:29 into actual working code which I think is what most people think developers do. This is how we make code, just by

37:35 sliding these tiles around.

37:38 Yeah. I remember one of my daughters, I can't remember which one, saw me typing out code.

37:43 And she's like, what are you doing? What is that? I'm working on this project, this program,

37:47 this website. That's what websites look like. Okay.

37:51 Yeah. My wife calls it the gibberish. She says, are you working on the gibberish? Yeah, I am.

37:57 That's incredible.

37:58 That's an amazing way to put it. Yeah. You know, I was just thinking the other

38:02 the incredible power of text, right?

38:05 You think of all these systems, the internet, Linux, configuration, source code, design,

38:11 all of that is just words, right?

38:14 It's not working with GUIs and all that stuff.

38:16 It's just typing.

38:17 Text is, yeah, very important.

38:19 I quite like websites that skew graphics and have just kind of like typography-based thing.

38:26 I find you can more easily extract the information you need.

38:30 But most websites these days you go to and loads of ads, loads of image,

38:34 they pop in and out.

38:35 The text seems almost secondary.

38:39 So I'm a big fan of just text in general.

38:43 Yeah, same.

38:44 All right, so people can play the game.

38:46 That's kind of interesting, right?

38:47 Like I said, I think where maybe it's more interesting is talking about the widgets.

38:52 And when I look at this, you know, what this reminds me of is it reminds me of

38:56 if I were to go over here and type Bulma and go, which Bulma is very refreshing.

39:01 CSS framework that doesn't require build tools and shaking and webpack and all that junk.

39:10 Or like Tailwind.

39:11 Tailwind is kind of a pain to work with, but it's also super beautiful, right?

39:16 When I look at this, the widgets, it feels very much similar to that, right?

39:21 I'm sure people have checked out CSS frameworks.

39:23 Yeah, that's...

39:25 Yeah, tell us about it.

39:26 Yeah, so I mean, you know, again, coming from the web, I want to have a web-like experience.

39:33 Because everybody's familiar with the web.

39:34 It's like you can sit someone down on a website and they'll be able to use it.

39:37 I wanted the same thing for terminal apps.

39:40 You know, some terminal apps have a bad reputation and have been hard to use, and it's mostly deserved.

39:45 Like, you know, how do you quit Vim?

39:48 It's a running joke.

39:50 I don't know how to quit Vim.

39:51 You reboot your computer.

39:52 Everybody knows that.

39:53 Come on.

39:54 Exactly.

39:55 I wanted, like, an interface that people were very, very familiar with.

40:00 this is actually in alphabetical order but I think buttons was actually one of the first

40:04 thing I developed because the terminal doesn't give you anything for free, the terminal is just

40:10 text with escape sequences for colour and style like italic, bold etc but there's no escape sequence for a button

40:18 the most simplest atomic user interface thing a button with a bit of text that you click

40:23 there's nothing in the terminal spec for that, so you have to start from scratch

40:28 and we built this button that looks button-y.

40:32 It even has a 3D sort of raised effect and stuff.

40:37 Yeah, so there's a lighter color at the top and a darker color at the bottom,

40:40 and it looks like a classic Windows raised button.

40:45 Imagine if this light is coming from the top and illuminating the top edge.

40:50 Some people don't like them.

40:53 I think one of the reasons is it has to take up three characters.

40:55 In order to get that effect, you have to take up three lines.

40:59 So it creates quite a chunky button.

41:01 It looks like a button, but it's chunky.

41:04 But you can style it very trivially to be one line long, so your button can just be as tall as it needs to be.

41:11 Nice.

41:12 Yeah, these are great, and we've got other things.

41:14 I was thinking as you were talking about primitives you have to work with,

41:17 I was thinking of pretty much every 3D rendered thing that people see, video games and others as well, animations.

41:26 They're just triangles.

41:27 There's just a bunch of triangles, which is so insane that you think, that thing that looks like a real forest,

41:32 every single thing on the screen is some weird triangle with a texture and stuff applied to it.

41:37 But a lot of triangles.

41:38 Lots.

41:39 You can't think too hard about how fast things are happening on graphics cards because your mind will melt.

41:44 It's like there's so much going on.

41:47 You've got check boxes.

41:48 Is it all?

41:49 Radio buttons.

41:50 Radio buttons, which are great.

41:52 A lot of the things that people might think about, data tables, you know, this is, if you're doing data science, I think one of the things worth talking about, because when we talk about some of the performance things, there's a lot of this within that within that, right? Like, like in your data table. So this whole UI, I can scroll up and down and see all the different widgets. But then when I get to the data table section, I can scroll horizontally and vertically within the data table itself,

42:18 right?

42:18 You can also press Control-A, and that'll maximize it.

42:21 Oh, yeah.

42:22 Very cool.

42:23 I didn't even know about that.

42:24 Yeah, this is a nice feature, which I think websites should have and GUIs in general.

42:29 I remember Windows used to suffer from this.

42:30 You'd get like a dialog where you had to enter a lot of text, but the dialog was this size,

42:35 and you weren't allowed to resize it.

42:37 You can't resize it.

42:39 Yeah.

42:39 Or you tap over and hope you hit the right button that you can't see.

42:42 Yeah.

42:42 So I added a shortcut key, which can maximize anything which is focused.

42:48 can make it resize it to the size of the screen.

42:51 Nice.

42:52 Okay.

42:52 In the interest of time, let's go and talk about projects real quick.

42:56 I think one of the things that probably can inspire people a lot is some of these projects

43:02 here.

43:02 If you go to the project section, you've got a couple that are super nice here.

43:07 For example, I would say this posting I know by Darren Burns is pretty interesting.

43:13 This is like postman, right?

43:14 Like an HTTP client, but entirely in your terminal.

43:16 I'll tell people about this.

43:17 is great.

43:18 Yeah, so this is something that Darren Burns built and he's a developer of, one of the

43:23 developers, and he made a fantastic job of this.

43:25 This is not a textualized project but basically it's Postman which is a tool for making

43:32 HTTP requests. You might use it if you're developing an API and you want to experiment

43:37 and you don't want to have to build all the tooling to talk to your API. You can just go straight

43:41 into this interface and experiment and play around with it.

43:45 I did that two days ago.

43:47 Yeah, and it's super useful.

43:50 And the fact that it's in the terminal is completely appropriate for this kind of application

43:54 because that's where you're going to be a lot of the time.

43:58 It just seems to be the right place.

43:59 If you have your app in a browser, it can take you out of your flow if you're developing

44:05 and having to switch over a window.

44:07 But having it all on the terminal is super useful.

44:11 And this project has sort of grown in feature set and I think it's comparable to some much, much larger projects,

44:18 including Postman.

44:19 Yeah, nice.

44:20 And open source, right?

44:21 Open source, yeah.

44:22 Yeah, cool.

44:24 What other ones do you want to give a quick shout-out to?

44:27 Well, yeah, we'll go through them there.

44:28 We've got Memory that was written by Bloomberg.

44:32 They were an early adopter of Textual, and it's kind of a memory profiler,

44:36 so it can run your Python application and it can pinpoint exactly what's taking up too much memory

44:43 or if you've got a leak, you know, you're in a loop and you've been reserving objects, you don't know about it,

44:48 it can alert you to that, which is very hard to actually debug without any kind of tool.

44:54 So that's a fantastic project.

44:57 I had them on not too long ago to talk about it, and yeah, this is a super cool project.

45:03 I think also just graphically, it's pretty neat, right?

45:06 Yeah, it displays all the information that you need right there in the terminal in a nice way.

45:13 I mean, I can imagine a previous version of something like this writing out a big spew of JSON,

45:20 which you then got to study to figure it out.

45:25 We've saved a CSV file.

45:26 Now you can look at it.

45:27 It's like, oh, great.

45:28 Yeah, exactly.

45:29 But if you can look at it in a graphical form in a nice table, you can visually parse information much quicker

45:35 so you can pinpoint things.

45:38 We talked a little bit around this, But another huge benefit of these types of UIs with Textual

45:43 is the ability to run them remotely on servers and other, or on a Raspberry Pi or on things that are embedded types of stuff

45:53 that you just only access through the terminal, right?

45:56 Like if you wanted to do memory on your web server, you probably don't want to just open up a bunch of ports

46:03 and start serving, you know, a parallel HTTP request over it, right?

46:08 But if you SSH in and you just run this GUI that's updating, it's really nice, right?

46:12 Yeah, exactly.

46:14 You don't even need to have a desktop.

46:16 You don't need to run a server.

46:17 And like I said, you don't need to set your firewall to allow a particular port and configure applications.

46:22 All that's gone.

46:23 Because it's a text interface, you just need to have a shell SSH into the server

46:28 and then run it, and you can see this nice interface there because it is just purely text.

46:33 And there's not much text involved.

46:35 You'd be surprised about text that needs to be sent over the wire.

46:38 is surprisingly minimal.

46:40 If you compared it to a YouTube video of the lowest quality, I don't know what the lowest quality is,

46:45 like 120p or something.

46:47 Yeah, 360 by 240 or something.

46:50 Who knows?

46:51 Yeah, that would be vastly more data than a text interface like this.

46:55 Yeah, even if it's madly updating, yeah.

46:58 I mean, you could actually.

46:59 You could if you want to push it.

47:00 Like if I was to run this ridiculous demo with a spinning gradient at 60 frames a second

47:06 and setting the screen size to ridiculous.

47:09 It might turn into a video, but for most interfaces.

47:14 But is it probably applicable for like GZip transfer as well?

47:19 You know, text compresses really well.

47:21 I don't know.

47:22 True.

47:22 It depends what you're rendering.

47:24 I don't know if SSH does.

47:25 I don't know.

47:26 Maybe it does.

47:26 I would think it compresses.

47:29 Compresses and encrypts.

47:31 Yeah.

47:31 I would hope so.

47:32 Yeah, well, compression should be cheaper than the encryption.

47:35 So yeah, why not go ahead and compress it while I'm at it, right?

47:37 Exactly, yeah.

47:38 Yeah, yeah, yeah.

47:39 Okay.

47:40 So somewhere in here, we've pulled up so many things.

47:43 I pulled up your article that I wanted to talk about as well.

47:46 So let's talk about this.

47:48 This was your, you said you wanted to write an article, sort of celebrate the 1.0 milestone for Textual.

47:54 And the article was called Algorithms for High-Performance Terminal Apps.

47:58 And I thought there was just some fun things to pull out of here.

48:01 So I thought we could maybe talk through some of them.

48:03 Yeah, sure.

48:03 So there's lots of interesting technical problems to solve with Textual and lots of cool solutions.

48:10 But I still rarely have the chance to talk to anyone about it because it's one of these things that's kind of under the hood.

48:15 It just works.

48:16 Yeah.

48:17 There's nothing to discuss.

48:18 So I wrote this blog post to describe some of the cool things that go on.

48:22 The most core part of Textual is the compositor.

48:26 This takes all the widgets.

48:28 If you've got an interface, it might have data table, button, checkboxes.

48:35 And there might be loads of those, and it might be able to scroll through it.

48:38 And Textual has to take all those widgets and assemble them into a single update.

48:43 Because there's nothing in the terminal that does a kind of a tiling window manager.

48:48 You just write text to the terminal.

48:50 So the software has to create this illusion that you've got windows and things that are overlapping.

48:57 and that's essentially what the compositor does.

48:59 That's great.

49:00 And you've got transparency.

49:03 Oh my goodness.

49:05 Yeah, it's kind of magical.

49:06 It shouldn't be possible in the terminal.

49:08 It should not be possible.

49:09 It cheats.

49:11 You're not seeing true transparency because the terminal is not capable of it.

49:15 You can't put an X on top of an A and then see the X translucently with an A in the background.

49:24 But you can change background colors.

49:26 So if you put something on top of background, you can calculate the transparency on top of the widget underneath it.

49:34 So you can create the illusion of background of transparency.

49:39 And that's something that the compositor does as well.

49:43 Is textual pure Python?

49:45 Pure Python, yeah.

49:47 I had thought that at some point we would have to write something in Rust.

49:52 And I bought a couple of Rust books, which I haven't read yet.

49:56 But every time I got to a point where this is a bit too slow, I found a very good optimization in Python.

50:02 So it turns out that it's just not necessary.

50:04 I mean, if we were to write some core code in Rust, I don't think we'd get marvelous speed up.

50:10 We might get 5%, 10% if we're lucky for a lot of effort.

50:14 But since it runs on a lowly Raspberry Pi, it's not been necessary.

50:18 Python's been more than fast enough.

50:20 Yeah.

50:21 And you can just ride on the coattails of the faster CPython team.

50:25 Oh, yeah.

50:26 Yeah, I mean, yeah.

50:27 That helps too.

50:27 A new version comes along, we can say Textual just got 10% faster.

50:31 Textual has some sort of concurrency framework too, right?

50:35 Yeah, so it's based on asyncio.

50:38 And every widget from a button to a text area to a checkbox runs an asyncio task.

50:45 So all these things are running concurrently, potentially updating the state.

50:50 and Textual will update things according to the schedule of the widgets.

50:57 You might have a widget which, say, updates 10 times a second, another widget which updates when you press a key,

51:03 and another widget which updates infrequently.

51:07 And Textual will combine those updates into a single update so that it looks clean,

51:13 it doesn't flicker, and it's what does the magic of updating things.

51:20 Yeah, awesome.

51:21 That's super neat.

51:23 I think UI frameworks and AsyncIO, Async and Await, they make a lot of sense, right?

51:28 It's event-driven stuff, and so you can, instead of pulling or whatever, you just say,

51:32 I'm going to wait until this thing has some kind of event, and then I'll work on it, and if that doesn't have an event,

51:38 I'm going to go work on something else.

51:39 Yeah, it's pretty cool.

51:40 Yeah, exactly.

51:41 It's funny that AsyncIO was obviously, you can tell by the name, it wasn't intended to build user interfaces,

51:47 whether in the terminal or not, it was built for I.O., but it's very good at handling concurrency of any kind,

51:55 including text user interfaces.

51:56 So I'm a big fan of Async.

51:58 It's not perfect.

52:00 There are a bunch of gotchas which come up in our support channels.

52:04 For instance, someone will be playing around with Textual and will say, well, I'm going to need some long-running tasks,

52:10 so I'll simulate this by putting a sleep in.

52:12 We'll do like sleep 10.

52:14 The whole app just locks up.

52:16 I can't believe it.

52:17 Exactly, exactly.

52:18 So it's not perfect, but there is no perfect solution to concurrency.

52:23 But I think it's as good as it can be with the current level of technology, I think.

52:28 Yeah, yeah.

52:29 Yeah, very neat.

52:31 Free-threaded Python?

52:33 Is it going to influence that kind of thing?

52:36 I think so.

52:37 There's a whole lot of cool things that free-threaded Python can do.

52:40 We've kind of convinced ourselves that we didn't need that kind of level of concurrency,

52:43 you know, from like a web application or something, because it's really linear.

52:49 and the I.O. gives you concurrency in a sense because you're waiting for the I.O.

52:54 But I think we need a better abstraction for concurrency because having to

53:01 explicitly create a thread and run it requires that you think in terms of threads

53:08 which is quite challenging.

53:10 I think it should be the same as async.io. It should be async and await.

53:14 I think that would be a marriage between them where like you don't really know as you're consuming it you know

53:19 yeah unfortunately async io is single threaded you can use multiple threads but you've got to consciously write the

53:28 code i would like to see an async io where it just distributed work across the course if it's if it's

53:35 io based just keep it on the same thread but if it's computational based then like you're going to

53:39 just put it on another thread and give me a future sort of thing to await and we don't care

53:43 yeah Absolutely.

53:44 Yeah.

53:44 So I think in the future, we're probably going to get something like that

53:48 because I think Python is so valuable that people are going to want to use it more

53:53 and they're going to build these better abstractions.

53:56 So I imagine in a year or two, maybe less, we'll have an asyncio that's rethreaded

54:04 or something like it.

54:04 Yeah.

54:05 We say that we don't need it for web apps because we can just scale them out, right?

54:08 We'll just run four of them, four worker processes.

54:11 but there's a big memory cost to running.

54:14 Well, I've got 16 cores, so I'm going to run 16 worker processes, right?

54:19 They each take half a gig.

54:20 All of a sudden, you're like, eight gigs just for this.

54:24 Just so I can get computational parallelism, right?

54:27 Yeah.

54:27 Exactly.

54:28 I mean, yeah, the IO gets you a lot of the way there, but sometimes you accidentally create CPU-bound code,

54:36 which would run very poorly.

54:39 But if you're using all our threads efficiently, then that should be easier.

54:43 It should be easier, yeah.

54:44 You know, it's super easy.

54:46 Like, oh, I've got an ORM and I'm going to return 20,000 things that I'm parsing into Pydanic objects for FastAPI.

54:52 All of a sudden, like, most of the work is the serialization that's computational.

54:57 Anyway, all right.

54:57 So let's get back to this.

54:59 But I think that's super, super interesting.

55:01 We could do a whole other hour on that.

55:03 So one of the things in here you talked about is thinking in segments and not characters or stuff for drawing terminal lines.

55:12 Let's talk about that because I really want to talk about the switching the primitive idea you got, but I think this is a good setup.

55:17 Yeah, sure.

55:18 So people who aren't familiar with inner workings of terminals might look at terminals and think it's like a two-dimensional array of characters.

55:26 It does seem like that.

55:27 And if you're building a textual app, you might think that you could compose a screenful of text like a bitmap, as if the characters were pixels.

55:37 And as developers, that would be like stuff we're very familiar with.

55:40 We could easily build that and create stuff with it.

55:46 But the problem is that terminals aren't two-dimensional arrays of characters.

55:50 And it's for one reason is that some characters take up twice as much space as others.

55:56 Chinese, Japanese, and Korean tags typically take up two characters because their characters are more complex.

56:01 They deliver more information in a single character, so they have to be wider.

56:05 There's also emoji, which many emojis can take up twice as much space.

56:11 I'd have the snake emoji.

56:13 Exactly.

56:14 I think that might be two.

56:16 I don't know how wide it is.

56:18 Some emoji are one, like a smiley face or something, but some emoji are two.

56:24 And if you were to treat it like a two-dimensional ray, it kind of falls apart at that point

56:29 because if you subtract one from the Y coordinate, it's not directly above the original coordinate.

56:36 So in situations like that, rather than trying to fudge things into the primitive

56:42 that you first thought of, you can come up with a new primitive, a new way that represents the very lowest common denominator

56:50 of thing you want to do.

56:52 And for textual, that turned out to be segments.

56:55 So rather than a character, it's like a pixel.

56:58 We use a segment.

57:00 A segment is a string, any number of characters, plus a style.

57:04 And by assembling these segments, you can represent anything in the UI.

57:10 And all the operations which are done are kind of built on segments.

57:15 So I call this switching the primitive, because the original primitive was like a character's a pixel, right?

57:20 Which didn't work.

57:21 If you were to, you could use that with a bunch of fudges to build on top of it,

57:27 but you might find that it falls apart.

57:28 Things which should be straightforward suddenly become really complicated.

57:32 And that's because as developers, we tend to layer primitives on top of other operations.

57:37 And if the very foundational primitive doesn't represent your problem very well,

57:42 then it can fall apart.

57:44 So the switch from pixels to segments worked very well with Textual.

57:48 and Textual is essentially a segment processing engine.

57:51 Everything the compositor does takes in a bunch of segments, manipulates them, spits out new segments,

57:57 and then those get turned into escape sequences.

58:00 Yeah, very cool.

58:02 I think this idea of switching the primitive, you know, people who are new to programming or new to Python,

58:07 especially programming, I have a hard time.

58:11 I see the code, I look at someone else's code, and I see the code, and it makes sense,

58:15 but then when I sit down with a blank screen and try to solve a problem,

58:18 I'm stuck. I don't know what to do.

58:20 And this switching the primitive concept of like, we looked at it this way and it just seemed so complicated and intractable.

58:26 And then I decided to rejigger maybe the data structures and how we organize stuff and how we combine them.

58:31 And boom, it just unlocks simplicity, actually.

58:35 Kind of like going back to our triangles and graphics.

58:37 It's like, well, if you think in pixels, 3D game engines are impossible.

58:41 With ray tracing and lighting and shadows, you think in triangles, all of a sudden it unlocks it, right?

58:47 this sounds a little bit similar, and I think that's a pretty valuable lesson for people to take.

58:51 That's exactly right. So if you had a game engine, and the only function it had was like put pixel,

58:58 it would be a nightmare to build anything but the most trivial games, but the primitive of

59:03 a triangle means you can build 3D anything. So that is the bare primitive.

59:07 And they have angles, so they have reflection, and so they have all sorts of stuff, yeah.

59:11 Yeah, something similar came up when Darren was building the text area.

59:16 So the text area has lots of operations, like insert a new line, delete a character,

59:22 delete a line, delete a paragraph, cut and paste, etc.

59:26 And the primitive for that boiled down to a very, very simple primitive,

59:30 and that was replace a range of text with new text.

59:34 It's like anything you can do to a text area comes down to that primitive of a range plus new text.

59:40 Like to delete something, you replace a range with an empty string to insert something.

59:46 You replace a range with the insertion character, like a new line plus the previous

59:52 string. So everything boils down to one very simple primitive.

59:56 And once you've found the right primitive, bigger problems become much, much easier.

01:00:00 Yeah, 100%.

01:00:02 It's amazing the way it works.

01:00:03 So we'll get short on time.

01:00:05 We got, pick one more lesson from here that you want to talk about maybe. The spatial

01:00:12 map was quite interesting.

01:00:13 Yeah, I think they do.

01:00:14 A bit further down, yeah.

01:00:16 So one of the problems we had in textual, when interfaces get complex, we've got lots and lots of widgets.

01:00:22 Some of them are visible and some of them are not.

01:00:24 Some of them are scrolled off.

01:00:26 They're still there in memory.

01:00:27 We know exactly where they are.

01:00:30 And when we come to render that, the most obvious way to do that is to step through them in a for loop

01:00:34 or widgets in all widgets, do stuff to widget.

01:00:39 But that means we're iterating over all the widgets every single time, even if they aren't visible.

01:00:46 So the spatial map was an attempt to optimize that.

01:00:48 Basically, it's a very quick way of deciding which the widgets are on screen

01:00:52 and which the widgets are currently scrolled off.

01:00:55 So we literally don't need to even loop through the scrolled off widgets.

01:00:59 And the way we did that briefly is we kind of logically separated the terminal

01:01:04 into a grid.

01:01:06 There's a step where you go through and assign every widget to the grid squares

01:01:11 that it might overlap with.

01:01:13 small widgets might be right inside a grid square some may overlap one or two or more and then once

01:01:19 we've got that information it's very easy to pick out which widgets are visible and exclude which

01:01:25 widgets aren't because we can take the screen figure out which of the grid squares it covers

01:01:31 and then we can go through those grid squares and there's probably just a handful of them and each

01:01:35 of those grid squares is associated with a list of widgets which are under that

01:01:38 you can combine that with a set and then very quickly you reduce that problem domain

01:01:44 from being potentially unbounded very large to like very small some of the widgets might actually not be visible

01:01:50 they might actually be in a grid square which is off the edge of the screen

01:01:54 but we can just loop through those and discard those ones but that's a much smaller operation

01:01:59 than looping through every widget in the location. Interesting. He talked about

01:02:04 it's fast enough in Python and it's this kind of stuff that makes it fast enough

01:02:08 in Python right? If you rewrote this in other native compiled language, it

01:02:13 would be faster to go through every single one of them, but you're still potentially processing

01:02:17 a tremendous number of things, and you get the compositor and overlap and all those kinds of things,

01:02:21 and you're really worried about stuff that's way off the screen.

01:02:24 Instead of going, we're going to rewrite it, you can just think, let me think about

01:02:27 the right data structures and the right algorithms.

01:02:29 Exactly.

01:02:30 And so on, yeah.

01:02:31 I mean, I would love it if Python just got so ridiculously fast.

01:02:34 You don't have to worry about that, and I'll take any speed improvements again.

01:02:39 But the optimizations you can produce yourself just by having the right algorithm,

01:02:43 they tend to outweigh the benefits you could get from, say, running it in C.

01:02:48 First, it would be implement that in C.

01:02:49 It would obviously be faster, but it would still be much faster to have that algorithm.

01:02:54 Yeah, exactly.

01:02:56 I'll take my order of O squared versus your O factorial versus N squared.

01:03:03 You know what I mean?

01:03:04 Exactly.

01:03:05 It's going to lose on N factorial no matter what you do, right?

01:03:08 Yeah.

01:03:09 I mean, Python wins because of the speed of development.

01:03:11 We know it's not the runtime speed.

01:03:13 Yeah, absolutely.

01:03:13 The runtime speed is okay most of the time, but it's the speed of development.

01:03:17 The fact that you have got more time to pick the right algorithm means it's not as slow as people might think.

01:03:23 Yeah, absolutely.

01:03:24 It doesn't get too complicated.

01:03:26 Yeah, thinking about this, I think there's a lot of parallels back to game engines

01:03:31 that I think about a lot.

01:03:32 The game engines have this thing called oct trees, like basically break stuff into eight squares,

01:03:38 and you say, well, which I'm looking for a thing if it's going to say intersect with this, right?

01:03:43 Because it's like for hit detection and stuff.

01:03:44 Like, well, is it in this half of this world or that half?

01:03:47 Well, it's in this half.

01:03:47 Okay, well, we're not even going to compare it against half of the things in the world

01:03:50 that are in that half.

01:03:51 Now, look at the hash, right?

01:03:53 Like you can super quickly go, I have a million objects on screen, but I can determine if this thing hits it

01:03:59 with five comparisons or something like that.

01:04:01 You know what I mean?

01:04:02 With just the right data.

01:04:02 That's exactly it.

01:04:03 It's super interesting.

01:04:04 Yeah, yeah.

01:04:05 It's probably no coincidence that I used to work in video games.

01:04:10 Oh, did you? Okay, yeah.

01:04:11 Yeah.

01:04:11 So that was kind of my mindset.

01:04:13 It's like reducing the problem.

01:04:15 And Textual is quite a lot like a video game.

01:04:17 If you imagine like a 2D video game that's got lots of sprites that are over here,

01:04:21 we don't want to worry about those.

01:04:22 Yeah.

01:04:22 Replace sprites with widgets, and it's pretty much the same problem.

01:04:25 Yeah, yeah, exactly.

01:04:25 It's the same kind of mindset.

01:04:27 It certainly is.

01:04:28 All right, well, I think we're going to need to just kind of wrap it up here

01:04:31 because I think we're getting short on time.

01:04:34 Let's leave everyone with roadmap.

01:04:38 Where, you know, you've hit this 1.0 milestone with Textual.

01:04:41 Where are things going?

01:04:42 So there's a whole bunch of features which I want to implement.

01:04:46 I mean, Textual is usable as, you know, you can build fantastic apps, but there are a few things where we don't have parity with web apps.

01:04:53 Something I've implemented on a feature branch is arbitrary text selection.

01:04:58 So, you know, on a website, you can just draw your mouse over text you want to select.

01:05:03 That didn't work well with 2Es.

01:05:06 If the terminal did it, if the terminal doesn't understand the logical separation of widgets,

01:05:11 it would just copy all the text on that line, which might get parts of one widget and parts of another.

01:05:18 So I've implemented Arbority Text Selection, which seems to work quite well.

01:05:22 And it's such a fundamental feature, I think.

01:05:23 Wow.

01:05:24 It'll make Textual more useful.

01:05:26 Yeah, I guess if you've got a UI in Textual, and you're like, I need to get that and paste it into another document

01:05:32 or search it, you know, highlight the map, the address to put in a map or something, right?

01:05:37 Yeah, and things like, you know, if you're using Textual to talk to an LLM,

01:05:41 an LLM writes some code, you don't want to transcribe that code visually.

01:05:46 Yeah, and so now we can be able to like, we can select the entire code.

01:05:49 You could add a button, but it's very convenient to drag.

01:05:53 So that's something that's coming very soon.

01:05:56 And yeah, there's a whole bunch of features I've got, which I'll be working on.

01:06:00 Yeah, watch this space.

01:06:01 Excellent.

01:06:02 And PRs accepted, contributions accepted?

01:06:05 Very much so, yep.

01:06:07 There's a big pile of them there, but I tend to work on a big feature and then spend a few days going to the PRs.

01:06:13 And I love to have PRs.

01:06:14 It means that people are very interested.

01:06:17 And it saves me time, of course.

01:06:18 If someone else is fixing bugs and writing code, then it's a great thing.

01:06:22 Awesome.

01:06:23 All right, well, thanks for coming on, giving us an update on Textual and all the stuff that you guys built.

01:06:28 It's super valuable to people.

01:06:30 Always a pleasure.

01:06:30 Thank you very much.

01:06:31 Yeah, you bet.

01:06:32 See you later.

01:06:32 Bye-bye.

01:06:33 This has been another episode of Talk Python to Me.

01:06:37 Thank you to our sponsors.

01:06:38 Be sure to check out what they're offering.

01:06:40 It really helps support the show.

01:06:41 This episode is sponsored by Posit Connect from the makers of Shiny.

01:06:46 Publish, share, and deploy all of your data projects that you're creating using Python.

01:06:50 Streamlit, Dash, Shiny, Bokeh, FastAPI, Flask, Quarto, Reports, Dashboards, and APIs.

01:06:57 Posit Connect supports all of them.

01:06:59 Try Posit Connect for free by going to talkpython.fm/posit, P-O-S-I-T.

01:07:05 Want to level up your Python?

01:07:06 We have one of the largest catalogs of Python video courses over at Talk Python.

01:07:11 Our content ranges from true beginners to deeply advanced topics like memory and async.

01:07:16 And best of all, there's not a subscription in sight.

01:07:18 Check it out for yourself at training.talkpython.fm.

01:07:21 Be sure to subscribe to the show, open your favorite podcast app, and search for Python.

01:07:26 We should be right at the top.

01:07:28 You can also find the iTunes feed at /itunes, the Google Play feed at /play,

01:07:33 and the direct RSS feed at /rss on talkpython.fm.

01:07:37 We're live streaming most of our recordings these days.

01:07:40 If you want to be part of the show and have your comments featured on the air,

01:07:43 be sure to subscribe to our YouTube channel at talkpython.fm/youtube.

01:07:48 This is your host, Michael Kennedy.

01:07:50 Thanks so much for listening.

01:07:51 I really appreciate it.

01:07:52 Now get out there and write some Python code.