WEBVTT

00:00:00.020 --> 00:00:03.660
<v Michael Kennedy>Every ship in EVE Online eventually undocks and leaves the station.

00:00:04.420 --> 00:00:05.860
<v Michael Kennedy>This time, it's the whole game.

00:00:06.500 --> 00:00:09.420
<v Michael Kennedy>EVE is run on Python 2 since its launch in 2003.

00:00:10.260 --> 00:00:17.360
<v Michael Kennedy>All 2.4 million lines of it on a custom stackless interpreter that stopped at Python 3.8.

00:00:18.000 --> 00:00:20.620
<v Michael Kennedy>And stackless Python was mothballed last year.

00:00:21.080 --> 00:00:23.920
<v Michael Kennedy>Now it's destination Python 3 for EVE Online.

00:00:24.660 --> 00:00:28.660
<v Michael Kennedy>The route runs through 6,500 lines of division that decide who wins a fight

00:00:29.060 --> 00:00:33.840
<v Michael Kennedy>and 100 gigabytes of pickled Python objects that have to survive the jump intact.

00:00:34.440 --> 00:00:37.320
<v Michael Kennedy>Kristinn Sigurbergsen was on the show 10 years ago,

00:00:37.590 --> 00:00:39.740
<v Michael Kennedy>and he's back, this time with Jamie Bannister,

00:00:39.990 --> 00:00:42.500
<v Michael Kennedy>who is flying the EVE Online migration right now,

00:00:42.960 --> 00:00:46.300
<v Michael Kennedy>and Thomas Darling, who took EVE Frontier through the jump first.

00:00:47.000 --> 00:00:49.320
<v Michael Kennedy>In EVE, a destroyed ship is gone for good.

00:00:49.920 --> 00:00:54.400
<v Michael Kennedy>There are no do-overs, and the universe has to stay online the entire time.

00:00:54.860 --> 00:00:55.800
<v Michael Kennedy>That's a serious migration.

00:00:56.380 --> 00:01:02.000
<v Michael Kennedy>This is Talk Python To Me, episode 564, recorded Thursday, September 10th, 2026.

00:01:20.280 --> 00:01:24.620
<v Michael Kennedy>Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists.

00:01:24.960 --> 00:01:26.540
<v Michael Kennedy>This is your host, Michael Kennedy.

00:01:26.980 --> 00:01:30.460
<v Michael Kennedy>I'm a PSF fellow who's been coding for over 25 years.

00:01:31.100 --> 00:01:32.220
<v Michael Kennedy>Let's connect on social media.

00:01:32.540 --> 00:01:35.620
<v Michael Kennedy>You'll find me and Talk Python on Mastodon, BlueSky, and X.

00:01:35.870 --> 00:01:37.800
<v Michael Kennedy>The social links are all in your show notes.

00:01:38.580 --> 00:01:42.080
<v Michael Kennedy>You can find over 10 years of past episodes at talkpython.fm.

00:01:42.260 --> 00:01:45.460
<v Michael Kennedy>And if you want to be part of the show, you can join our recording live streams.

00:01:45.820 --> 00:01:46.300
<v Michael Kennedy>That's right.

00:01:46.600 --> 00:01:49.740
<v Michael Kennedy>We live stream the raw uncut version of each episode on YouTube.

00:01:50.320 --> 00:01:54.760
<v Michael Kennedy>Just visit talkpython.fm/youtube to see the schedule of upcoming events.

00:01:54.990 --> 00:01:58.640
<v Michael Kennedy>Be sure to subscribe there and press the bell so you'll get notified anytime we're recording.

00:01:59.700 --> 00:02:02.240
<v Michael Kennedy>This episode is sponsored by Sentry's Seer.

00:02:02.520 --> 00:02:04.980
<v Michael Kennedy>If you're tired of debugging in the dark, give Seer a try.

00:02:05.540 --> 00:02:09.220
<v Michael Kennedy>There are plenty of AI tools that help you write code, but Sentry's Seer is built to

00:02:09.220 --> 00:02:10.759
<v Michael Kennedy>help you fix it when it breaks.

00:02:11.340 --> 00:02:17.080
<v Michael Kennedy>Visit talkpython.fm/sentry and use the code talkpython26, all one word, no spaces,

00:02:17.440 --> 00:02:19.220
<v Michael Kennedy>for $100 in Sentry credits.

00:02:20.560 --> 00:02:23.440
<v Michael Kennedy>And it's also brought to you by Talk Python Courses.

00:02:24.020 --> 00:02:26.280
<v Michael Kennedy>Course completion certificates are now live.

00:02:26.460 --> 00:02:30.700
<v Michael Kennedy>If you finished a course, there's a certificate waiting for you on your account page right now.

00:02:31.180 --> 00:02:36.920
<v Michael Kennedy>Download it as a PDF or add it to your LinkedIn profile with one click under licenses and certifications.

00:02:37.480 --> 00:02:38.680
<v Michael Kennedy>Same section as your formal degrees.

00:02:39.980 --> 00:02:43.820
<v Michael Kennedy>Visit training.Talk Python.fm slash account to see what you've already earned.

00:02:45.220 --> 00:02:49.220
<v Michael Kennedy>Kristinn, Jamie, and Thomas, welcome to Talk Python To Me.

00:02:49.520 --> 00:02:50.440
<v Michael Kennedy>Kristinn, welcome back.

00:02:51.020 --> 00:02:51.800
<v Michael Kennedy>Jamie Thomas, hello.

00:02:52.310 --> 00:02:52.840
<v Michael Kennedy>Welcome to the show.

00:02:53.160 --> 00:02:53.440
<v Kristinn Þór Sigurbergsson>Hello.

00:02:53.900 --> 00:02:54.260
<v Kristinn Þór Sigurbergsson>Hello, Michael.

00:02:54.620 --> 00:02:54.760
<v Kristinn Þór Sigurbergsson>Hello.

00:02:55.220 --> 00:02:55.740
<v Kristinn Þór Sigurbergsson>Great to be back.

00:02:56.000 --> 00:02:56.420
<v Michael Kennedy>Yeah, yeah.

00:02:56.580 --> 00:02:57.380
<v Michael Kennedy>Great to have you all here.

00:02:57.820 --> 00:03:01.700
<v Michael Kennedy>So I'm really excited to be talking about EVE Online again.

00:03:02.860 --> 00:03:03.580
<v Michael Kennedy>When was it we?

00:03:03.720 --> 00:03:06.000
<v Michael Kennedy>I think it was 2016, 17.

00:03:06.070 --> 00:03:07.160
<v Michael Kennedy>It was a while ago.

00:03:07.480 --> 00:03:09.660
<v Kristinn Þór Sigurbergsson>I think it was 2016.

00:03:10.570 --> 00:03:15.920
<v Kristinn Þór Sigurbergsson>And I think we did even brought up the discussion point of the Python 3 upgrade.

00:03:16.320 --> 00:03:18.040
<v Kristinn Þór Sigurbergsson>So it's been on the table for a while.

00:03:18.120 --> 00:03:22.200
<v Michael Kennedy>Well, we're here to talk about upgrading EVE Online to Python 3.

00:03:22.740 --> 00:03:26.860
<v Michael Kennedy>Well, Virgin, are you going to start at 3.0 and work your way through all of them,

00:03:26.860 --> 00:03:28.780
<v Michael Kennedy>or are you going to do a little bit of a skip ahead?

00:03:29.520 --> 00:03:32.260
<v Thomas Dähling>So we are going to Python 3.12.

00:03:33.340 --> 00:03:34.240
<v Thomas Dähling>That's what we updated to.

00:03:34.560 --> 00:03:38.080
<v Thomas Dähling>We first did a step in between to stackless Python 3.8.

00:03:38.440 --> 00:03:43.500
<v Thomas Dähling>But when we did the work, 3.8 was already end of life, so to speak,

00:03:43.800 --> 00:03:45.400
<v Thomas Dähling>and stackless Python didn't go any further.

00:03:45.460 --> 00:03:53.400
<v Thomas Dähling>So we decided to go to 3.12 and have now worked on our infrastructure pipelines a little bit that any upcoming updates are a lot easier.

00:03:53.780 --> 00:03:56.840
<v Thomas Dähling>So we have plans to go to 3.15 later this year, for example.

00:03:57.100 --> 00:04:00.660
<v Michael Kennedy>Oh, fantastic. That's really neat. I'm going to be excited to hear about that.

00:04:00.860 --> 00:04:08.580
<v Michael Kennedy>Probably some cool performance benefits there and profiling and other types of benefits that you get from those last few versions there.

00:04:08.860 --> 00:04:12.180
<v Michael Kennedy>We're going to dive into why stackless and all that.

00:04:12.340 --> 00:04:15.680
<v Michael Kennedy>There's a lot of history here, and it's a really, really big project.

00:04:15.940 --> 00:04:20.540
<v Michael Kennedy>Before we get to the project itself and the upgrade and so on,

00:04:20.900 --> 00:04:24.040
<v Michael Kennedy>let's just start with a quick introduction for each of you.

00:04:24.260 --> 00:04:25.380
<v Michael Kennedy>I'll let you do that for yourselves.

00:04:25.660 --> 00:04:26.600
<v Michael Kennedy>Kristinn, you go first.

00:04:26.830 --> 00:04:28.940
<v Michael Kennedy>We'll go around the squares on the screen here.

00:04:30.040 --> 00:04:30.180
<v Kristinn Þór Sigurbergsson>Okay.

00:04:30.790 --> 00:04:32.060
<v Kristinn Þór Sigurbergsson>My name is Kristinn Seupersson.

00:04:32.300 --> 00:04:35.900
<v Kristinn Þór Sigurbergsson>I'm a director of gameplay engineering for eFrontier,

00:04:36.060 --> 00:04:39.940
<v Kristinn Þór Sigurbergsson>which shares the same technology stack as eOnline.

00:04:41.040 --> 00:04:51.340
<v Kristinn Þór Sigurbergsson>I've previously been everything from a game designer to software engineer to technical director of E1 Line and later technical director of platform or SEAR technology segment as well.

00:04:51.630 --> 00:05:03.380
<v Kristinn Þór Sigurbergsson>So I was part of the team that did the migration for 2.5.3 for Frontier and now I'm the primary benefit from it.

00:05:03.600 --> 00:05:10.000
<v Jamie Bannister>So I'm Jamie. I've been an engineer on EVE for about 16 years, mostly on gameplay.

00:05:11.160 --> 00:05:15.160
<v Jamie Bannister>But that has let me work across almost all the different aspects of the game,

00:05:15.760 --> 00:05:18.040
<v Jamie Bannister>from the front end of the client all the way through to the back.

00:05:18.240 --> 00:05:22.700
<v Jamie Bannister>So I've kind of seen all of the guts and the glory inside EVE.

00:05:23.440 --> 00:05:27.920
<v Jamie Bannister>So I'm at the moment working on the actual migration project for EVE Online itself,

00:05:28.540 --> 00:05:31.080
<v Jamie Bannister>building on kind of what the others have done previously.

00:05:31.880 --> 00:05:32.620
<v Michael Kennedy>Excellent. Thomas?

00:05:33.720 --> 00:05:39.520
<v Thomas Dähling>Yeah, I'm Thomas Delling. I'm a principal programmer on the platform segment for Fenris

00:05:39.700 --> 00:05:45.700
<v Thomas Dähling>Creations. And I have been leading the initial Python 3 transition for Carbon, like getting

00:05:45.840 --> 00:05:47.340
<v Thomas Dähling>us out of 2.7 onto Python 3.

00:05:48.900 --> 00:05:53.440
<v Michael Kennedy>Amazing. What a big project, y'all. What a big project. I think where we should start

00:05:53.820 --> 00:05:59.980
<v Michael Kennedy>is just what is EVE Online? And I have, of course, EVE Frontier pulled up. But tell us,

00:06:00.060 --> 00:06:02.100
<v Michael Kennedy>what is EVE Online? Whoever wants to take this one.

00:06:03.720 --> 00:06:08.500
<v Jamie Bannister>So Eve Online, it's an MMORPG in the classic sense.

00:06:09.180 --> 00:06:10.940
<v Jamie Bannister>It's been going since 2003.

00:06:12.080 --> 00:06:16.860
<v Jamie Bannister>So it's a massively multiplayer online game set in the far future

00:06:17.450 --> 00:06:20.720
<v Jamie Bannister>where humanity has gone through the wormhole to a new system

00:06:21.570 --> 00:06:24.800
<v Jamie Bannister>and society has kind of collapsed and then rebuilt in a new form.

00:06:25.960 --> 00:06:29.580
<v Jamie Bannister>It's a very, what we would call in gaming, an open world sandbox.

00:06:30.420 --> 00:06:37.560
<v Jamie Bannister>So we support all sorts of gameplay professions from mining, trading, piracy, a lot of combat.

00:06:38.480 --> 00:06:43.680
<v Jamie Bannister>I think the distinctive factor is the game is full, kind of full loss.

00:06:44.540 --> 00:06:47.000
<v Jamie Bannister>If you kill someone's ship, that's a ship removed from the game.

00:06:47.900 --> 00:06:50.300
<v Jamie Bannister>There's no kind of getting things back.

00:06:50.920 --> 00:06:51.540
<v Jamie Bannister>Nothing lasts forever.

00:06:51.780 --> 00:06:54.060
<v Jamie Bannister>So the game is built on creation and destruction.

00:06:54.380 --> 00:06:59.460
<v Michael Kennedy>A big part of it, if I understand it right, is kind of the economy and just the society.

00:06:59.640 --> 00:07:03.140
<v Michael Kennedy>Like there's a society that forms from the players.

00:07:03.580 --> 00:07:05.880
<v Jamie Bannister>Yeah, the economy I think,

00:07:06.200 --> 00:07:09.760
<v Jamie Bannister>and before I joined to work on the game,

00:07:09.850 --> 00:07:10.920
<v Jamie Bannister>I was a big player as well,

00:07:11.100 --> 00:07:12.600
<v Jamie Bannister>and the economy was a big part of it for me.

00:07:13.639 --> 00:07:16.820
<v Jamie Bannister>It's been used by, or been studied by economists,

00:07:17.170 --> 00:07:18.940
<v Jamie Bannister>because it's, if you think,

00:07:19.260 --> 00:07:20.860
<v Jamie Bannister>the data that we have on a big, real,

00:07:21.420 --> 00:07:24.720
<v Jamie Bannister>on our economy is more accurate and detailed

00:07:24.900 --> 00:07:26.840
<v Jamie Bannister>than you can kind of get from studying

00:07:26.890 --> 00:07:27.680
<v Jamie Bannister>the real world economies,

00:07:27.920 --> 00:07:29.300
<v Jamie Bannister>where you've always got imprecise data.

00:07:29.660 --> 00:07:31.620
<v Jamie Bannister>So we have every single transaction.

00:07:31.780 --> 00:07:33.060
<v Jamie Bannister>We know everything that's going on.

00:07:33.640 --> 00:07:36.060
<v Jamie Bannister>You see all the money as it moves around the game.

00:07:36.720 --> 00:07:37.420
<v Michael Kennedy>That's super interesting.

00:07:37.940 --> 00:07:39.200
<v Jamie Bannister>Yeah, very, very rich simulation.

00:07:39.600 --> 00:07:44.080
<v Michael Kennedy>What if we lived in a society where there was zero privacy and 100% data sharing?

00:07:44.520 --> 00:07:47.820
<v Michael Kennedy>It would be creepy from a human perspective, right?

00:07:47.940 --> 00:07:51.980
<v Michael Kennedy>Just like governments and alt-territarism and all that kind of stuff.

00:07:52.280 --> 00:07:56.160
<v Michael Kennedy>But from an economics perspective, you have perfect data, right?

00:07:56.500 --> 00:07:57.060
<v Jamie Bannister>Yes, yeah.

00:07:57.160 --> 00:08:03.280
<v Jamie Bannister>and it's interesting how real world things have come up in the game like there's no formal mechanic

00:08:03.310 --> 00:08:09.740
<v Jamie Bannister>for banks for example but players themselves have built banks they've built uh kind of loan schemes

00:08:10.860 --> 00:08:18.280
<v Jamie Bannister>organizations have got their own way of paying their members for turning up to operations so they

00:08:19.020 --> 00:08:24.339
<v Jamie Bannister>they have all the equivalent of like we have welfare groups we have um very centralized

00:08:24.360 --> 00:08:30.100
<v Jamie Bannister>democratic groups, communist groups, all those kind of different social phenomena have reappeared

00:08:30.120 --> 00:08:37.900
<v Michael Kennedy>in the game kind of organically. So interesting. And many online games are, you go here, you find a

00:08:38.020 --> 00:08:43.719
<v Michael Kennedy>server, you join 40 people, and you go have an experience, right? But this is a single, unified,

00:08:44.520 --> 00:08:49.420
<v Michael Kennedy>one world that everybody is in. They can all interact, all that kind of stuff, right? Yeah.

00:08:49.480 --> 00:08:52.420
<v Jamie Bannister>So we take place where we call it a single shard.

00:08:52.530 --> 00:08:54.120
<v Jamie Bannister>So yeah, like you say, it's one universe.

00:08:55.130 --> 00:08:57.680
<v Jamie Bannister>Any two people in the game, if they go to the same place,

00:08:58.420 --> 00:08:59.580
<v Jamie Bannister>they will meet and see each other.

00:09:00.700 --> 00:09:05.500
<v Jamie Bannister>And so we have, I think our largest kind of player group

00:09:06.020 --> 00:09:08.660
<v Jamie Bannister>is about 15,000 characters.

00:09:10.120 --> 00:09:12.320
<v Jamie Bannister>And so all of those potentially could get together

00:09:12.410 --> 00:09:15.620
<v Jamie Bannister>and go to one place at one time and have a huge fight.

00:09:15.800 --> 00:09:19.140
<v Jamie Bannister>I think we have two Guinness World Records for the most,

00:09:19.480 --> 00:09:26.980
<v Michael Kennedy>uh people taking part in a pvp fight wow how do you coordinate that as a as a group you know what

00:09:26.980 --> 00:09:32.480
<v Michael Kennedy>i mean like how do you coordinate just the conversations amongst the people without it

00:09:32.500 --> 00:09:38.480
<v Jamie Bannister>just being a cacophony um between in turn ourselves i mean with the players i mean with the players

00:09:38.560 --> 00:09:44.799
<v Michael Kennedy>if there's 1500 people in like uh some sort of action how can you have conversations and

00:09:44.820 --> 00:09:50.540
<v Michael Kennedy>communication without it being they built built their own solutions for a lot of this um so they've

00:09:51.000 --> 00:09:55.840
<v Jamie Bannister>replicated a lot of a chain of command that you get from from real kind of real world and some of

00:09:55.840 --> 00:10:02.240
<v Jamie Bannister>the bigger groups they have their own it departments uh that take care of of things a lot of groups use

00:10:02.440 --> 00:10:08.180
<v Jamie Bannister>uh discord for kind of coordination but they have and various other kind of voice uh systems they

00:10:08.300 --> 00:10:14.159
<v Jamie Bannister>have uh what's called an fc which is like a fleet commander so that's the person uh giving the orders

00:10:14.320 --> 00:10:15.440
<v Jamie Bannister>and kind of directing the troops.

00:10:16.340 --> 00:10:19.060
<v Jamie Bannister>And then it kind of, there's a hierarchy, if you like,

00:10:19.100 --> 00:10:20.480
<v Jamie Bannister>of communication that goes down.

00:10:21.300 --> 00:10:23.040
<v Jamie Bannister>So yeah, the bigger groups have built

00:10:24.399 --> 00:10:28.380
<v Jamie Bannister>very complex kind of command and control systems

00:10:28.720 --> 00:10:28.940
<v Jamie Bannister>with themselves.

00:10:29.500 --> 00:10:30.800
<v Michael Kennedy>I guess that makes sense, but wow.

00:10:31.080 --> 00:10:32.800
<v Michael Kennedy>And you think of a real army,

00:10:33.200 --> 00:10:34.120
<v Michael Kennedy>if there's a thousand people,

00:10:34.180 --> 00:10:36.960
<v Michael Kennedy>they don't all just get onto the same audio channel.

00:10:37.160 --> 00:10:39.280
<v Michael Kennedy>They do have certain people who talk to each other

00:10:39.380 --> 00:10:41.440
<v Michael Kennedy>and then they give commands down the line.

00:10:42.000 --> 00:10:42.380
<v Michael Kennedy>And interesting.

00:10:42.860 --> 00:10:46.520
<v Jamie Bannister>They replicate the processes like scouts filtering information up.

00:10:46.840 --> 00:10:50.200
<v Jamie Bannister>They'll have then some people gather that information and feed it into the FC,

00:10:50.580 --> 00:10:54.180
<v Jamie Bannister>who makes the calls of where they're going to go, what they kind of do next.

00:10:54.480 --> 00:10:54.620
<v Jamie Bannister>Yeah.

00:10:54.980 --> 00:11:00.160
<v Michael Kennedy>Kristinn, Thomas, you guys want to add anything to that overview before we dive in?

00:11:00.160 --> 00:11:03.500
<v Kristinn Þór Sigurbergsson>Yeah, I would say that this doesn't really happen by its own.

00:11:03.700 --> 00:11:11.120
<v Kristinn Þór Sigurbergsson>There are a lot of people that play EVE online that spend an incredible amount of time orchestrating all this.

00:11:11.360 --> 00:11:16.920
<v Kristinn Þór Sigurbergsson>not only just the fights themselves, but the reasons behind the fight, gathering everybody up.

00:11:17.690 --> 00:11:22.300
<v Kristinn Þór Sigurbergsson>So it is really fascinating to see. And to some extent, these people are playing the game,

00:11:22.310 --> 00:11:26.900
<v Kristinn Þór Sigurbergsson>but they're not actually in the game because they are managing the spreadsheets and

00:11:28.870 --> 00:11:34.779
<v Kristinn Þór Sigurbergsson>the communications and all of that. And to some extent in certain circles, like people have

00:11:34.980 --> 00:11:41.880
<v Kristinn Þór Sigurbergsson>put their experience in managing these larger corporations in EVE onto their CV and it has

00:11:42.040 --> 00:11:49.560
<v Michael Kennedy>helped them get a job. That's a level of commitment there. Yeah, I can now function as a full

00:11:50.510 --> 00:11:55.560
<v Michael Kennedy>logistics operation person because I've done it for a thousand people in EVE Online.

00:11:56.140 --> 00:12:01.720
<v Thomas Dähling>Very much so. Yeah, very much. Crazy. Thomas? Oh, I've got nothing more to add there. I think

00:12:01.740 --> 00:12:03.060
<v Thomas Dähling>they both described it really well.

00:12:03.420 --> 00:12:05.960
<v Thomas Dähling>It can sometimes be almost more like a job than a game

00:12:06.100 --> 00:12:06.940
<v Thomas Dähling>if you want it to be.

00:12:07.960 --> 00:12:09.580
<v Thomas Dähling>You learn a lot of valuable things.

00:12:10.060 --> 00:12:11.840
<v Thomas Dähling>You meet a lot of great people that way as well, right?

00:12:12.000 --> 00:12:13.980
<v Thomas Dähling>I mean, over the 20 years, so many people

00:12:14.120 --> 00:12:14.820
<v Thomas Dähling>that met each other.

00:12:15.520 --> 00:12:18.040
<v Thomas Dähling>We even had a wedding at one of our FanFests,

00:12:18.060 --> 00:12:18.460
<v Thomas Dähling>I believe.

00:12:20.120 --> 00:12:21.660
<v Michael Kennedy>We've actually had a few, I think.

00:12:22.840 --> 00:12:24.020
<v Michael Kennedy>Thomas, you gave a great talk

00:12:24.020 --> 00:12:25.240
<v Michael Kennedy>at FanFest about upgrading

00:12:25.940 --> 00:12:27.440
<v Michael Kennedy>and inspiring the folks.

00:12:28.140 --> 00:12:29.500
<v Michael Kennedy>That video is online.

00:12:30.460 --> 00:12:31.720
<v Michael Kennedy>I'll link to it in the show notes.

00:12:31.840 --> 00:12:33.440
<v Michael Kennedy>We'll maybe get to that in a little bit later.

00:12:33.640 --> 00:12:35.340
<v Michael Kennedy>So that sounds like a really cool gathering.

00:12:36.160 --> 00:12:37.920
<v Michael Kennedy>You know, I really, the last couple of years,

00:12:38.020 --> 00:12:40.360
<v Michael Kennedy>I've really enjoyed just diving into games.

00:12:41.040 --> 00:12:42.640
<v Michael Kennedy>And my daughter is getting older.

00:12:42.860 --> 00:12:44.640
<v Michael Kennedy>And one of the things she really likes to do

00:12:44.660 --> 00:12:46.220
<v Michael Kennedy>is sit down and play games together.

00:12:46.740 --> 00:12:48.800
<v Michael Kennedy>Not even online, it's too big,

00:12:48.980 --> 00:12:51.360
<v Michael Kennedy>but sort of cooperative stuff and just sort of exploring.

00:12:51.460 --> 00:12:52.880
<v Michael Kennedy>And I think that there's,

00:12:53.340 --> 00:12:55.080
<v Michael Kennedy>well, I know that studies have shown

00:12:55.140 --> 00:12:56.780
<v Michael Kennedy>that there's more loneliness.

00:12:57.740 --> 00:13:01.620
<v Michael Kennedy>There's fewer third places that people find themselves at and all that.

00:13:02.060 --> 00:13:07.100
<v Michael Kennedy>Having something like this where you can gather and be with people and immerse yourself in,

00:13:07.140 --> 00:13:08.400
<v Michael Kennedy>I think it's really neat, honestly.

00:13:08.740 --> 00:13:11.320
<v Michael Kennedy>As long as you don't take it too far, you know, still go to work and stuff.

00:13:11.660 --> 00:13:12.080
<v Jamie Bannister>Yeah, yeah.

00:13:12.220 --> 00:13:21.740
<v Jamie Bannister>I mean, before I joined and as a player, we would organize meetup events at different places around the UK where I was from.

00:13:22.340 --> 00:13:23.780
<v Jamie Bannister>And so I got to know a lot of people.

00:13:24.300 --> 00:13:28.600
<v Jamie Bannister>I travel to a lot of places just for the purpose of meeting up with other EVE players.

00:13:29.800 --> 00:13:34.640
<v Jamie Bannister>So for a lot of people, EVE or a lot of games like this,

00:13:35.020 --> 00:13:38.240
<v Jamie Bannister>they can be your primary social connection to a lot of them.

00:13:38.400 --> 00:13:45.640
<v Michael Kennedy>Let me take us back 35 years or something like that to the days of BBSs.

00:13:46.900 --> 00:13:48.740
<v Michael Kennedy>Have any of you guys done BBSs? Remember them?

00:13:49.600 --> 00:13:51.220
<v Jamie Bannister>Yeah, I used to play the MUDs back in the day.

00:13:51.780 --> 00:13:52.960
<v Michael Kennedy>Oh, I used to play MUDs as well.

00:13:53.260 --> 00:13:54.540
<v Michael Kennedy>Though that's not what I'm thinking of.

00:13:54.670 --> 00:13:57.840
<v Michael Kennedy>But actually, MUDs have a really interesting multi-user dungeons.

00:13:58.320 --> 00:14:00.580
<v Michael Kennedy>They have some really interesting analogies as well.

00:14:00.900 --> 00:14:02.300
<v Michael Kennedy>But I was thinking of Trade Wars.

00:14:03.080 --> 00:14:05.780
<v Michael Kennedy>Have any of y'all played Trade Wars from the BBS days?

00:14:06.620 --> 00:14:07.480
<v Michael Kennedy>I know of it.

00:14:07.600 --> 00:14:08.440
<v Jamie Bannister>I've never played it.

00:14:09.560 --> 00:14:10.440
<v Michael Kennedy>It's really interesting.

00:14:10.450 --> 00:14:13.080
<v Michael Kennedy>It was this multiplayer thing way back in the 90s.

00:14:13.540 --> 00:14:20.520
<v Michael Kennedy>And you would dial in and you'd have to take turns because only one phone could be connected to the BBS at the time, at a time.

00:14:21.040 --> 00:14:23.960
<v Michael Kennedy>And it was really, it has some real similar vibes.

00:14:24.160 --> 00:14:25.760
<v Michael Kennedy>It's, you know, you travel to different planets.

00:14:26.020 --> 00:14:28.300
<v Michael Kennedy>There's mining, there's an economy, there's trade.

00:14:28.820 --> 00:14:32.060
<v Michael Kennedy>Anyway, I feel like this is like, what if it could be real?

00:14:32.380 --> 00:14:33.920
<v Michael Kennedy>Instead of just a little sort of simple game,

00:14:33.980 --> 00:14:36.740
<v Michael Kennedy>what if you could make it real with all the nuances

00:14:37.160 --> 00:14:39.120
<v Michael Kennedy>and intricacies of reality and multiplayer?

00:14:39.380 --> 00:14:43.260
<v Michael Kennedy>And it's just the realization of what that thing should have become.

00:14:43.800 --> 00:14:43.860
<v Michael Kennedy>Yeah.

00:14:45.200 --> 00:14:48.780
<v Jamie Bannister>Because Eve was heavily inspired by a game called Elite,

00:14:49.060 --> 00:14:52.440
<v Jamie Bannister>which was one of the very early 3D space games.

00:14:52.510 --> 00:14:56.460
<v Jamie Bannister>And that had a big world, a big universe of systems.

00:14:57.020 --> 00:14:58.160
<v Jamie Bannister>Trading was a big part of it.

00:14:58.350 --> 00:15:00.760
<v Jamie Bannister>And so there's definitely links from there.

00:15:01.320 --> 00:15:03.720
<v Jamie Bannister>And also classic role-playing games as well.

00:15:04.200 --> 00:15:07.260
<v Jamie Bannister>Our character stats have similar kind of perception,

00:15:07.540 --> 00:15:09.800
<v Jamie Bannister>intelligence, charisma sort of stats.

00:15:09.880 --> 00:15:10.000
<v Jamie Bannister>Interesting.

00:15:10.870 --> 00:15:11.980
<v Michael Kennedy>Almost a D&D sort of.

00:15:12.020 --> 00:15:12.600
<v Michael Kennedy>Very much, yeah.

00:15:13.100 --> 00:15:14.500
<v Jamie Bannister>Yeah, you can tell there were D&D players

00:15:14.570 --> 00:15:17.000
<v Jamie Bannister>in the very early EVE designers, for sure.

00:15:18.640 --> 00:15:22.480
<v Michael Kennedy>This portion of Talk Python To Me is brought to you by Sentry and Sear AI.

00:15:23.060 --> 00:15:25.600
<v Michael Kennedy>There are plenty of AI tools that help you write code,

00:15:25.920 --> 00:15:28.840
<v Michael Kennedy>but Sentry's Seer is built to help you fix it when it breaks.

00:15:29.380 --> 00:15:30.440
<v Michael Kennedy>The difference is context.

00:15:31.040 --> 00:15:33.260
<v Michael Kennedy>Sear isn't just guessing based on syntax.

00:15:33.640 --> 00:15:35.660
<v Michael Kennedy>It's analyzing your actual Sentry data,

00:15:36.040 --> 00:15:38.000
<v Michael Kennedy>your stack traces, logs, and failure patterns.

00:15:38.540 --> 00:15:40.020
<v Michael Kennedy>Because it has the full context,

00:15:40.120 --> 00:15:43.120
<v Michael Kennedy>it can A, spot buggy code in review

00:15:43.200 --> 00:15:44.780
<v Michael Kennedy>and help prevent issues before they happen,

00:15:45.340 --> 00:15:48.240
<v Michael Kennedy>and B, identify the root cause of production errors.

00:15:49.140 --> 00:15:53.700
<v Michael Kennedy>It can even draft a fix and hand the work off to an agent-like cursor to open a PR for you.

00:15:54.280 --> 00:15:56.300
<v Michael Kennedy>Seer turns Sentry into a complete loop.

00:15:56.840 --> 00:16:01.440
<v Michael Kennedy>You have your traces, errors, logs, and replays to see the problem, and now AI to help solve it.

00:16:01.740 --> 00:16:07.720
<v Michael Kennedy>Join millions of devs at companies like Claude, Disney+, and even Talk Python, who use Sentry to move faster.

00:16:08.240 --> 00:16:16.900
<v Michael Kennedy>Check them out at talkpython.fm/sentry and use code talkpython26, all one word, for $100 in Sentry credits.

00:16:17.280 --> 00:16:19.380
<v Michael Kennedy>Thank you to Sentry for supporting Talk Python.

00:16:20.680 --> 00:16:25.680
<v Michael Kennedy>We're going to also talk about this thing called Eve Frontier, which is a little different,

00:16:26.260 --> 00:16:28.540
<v Michael Kennedy>but very much inspired, shares a lot of the code base.

00:16:28.900 --> 00:16:33.340
<v Michael Kennedy>So give us a quick comparison to Eve Frontier, which I'm sure fewer people know about.

00:16:33.920 --> 00:16:34.640
<v Kristinn Þór Sigurbergsson>Yeah, sure.

00:16:34.820 --> 00:16:41.340
<v Kristinn Þór Sigurbergsson>So what matters for the tech is that we've essentially forked Eve Online, the code base,

00:16:41.500 --> 00:16:43.960
<v Kristinn Þór Sigurbergsson>and are sharing a lot of the underlying tech.

00:16:44.200 --> 00:16:45.560
<v Kristinn Þór Sigurbergsson>We've made a lot of modifications.

00:16:45.920 --> 00:16:52.760
<v Kristinn Þór Sigurbergsson>We haven't been kind of pulling in stuff lately because it's just, it's become too different.

00:16:53.180 --> 00:17:00.160
<v Kristinn Þór Sigurbergsson>Where it differs from EVE Online is maybe if you play EVE Online, you very much come into a society.

00:17:00.540 --> 00:17:05.280
<v Kristinn Þór Sigurbergsson>Part of it is definitely player built, but part of it is also kind of NBC built.

00:17:05.280 --> 00:17:07.800
<v Kristinn Þór Sigurbergsson>The environment is there's a police, there's everything.

00:17:08.319 --> 00:17:12.959
<v Kristinn Þór Sigurbergsson>Whereas in Frontier, you're more coming into like an unbuilt, uninhabited zone

00:17:12.980 --> 00:17:15.680
<v Kristinn Þór Sigurbergsson>and you're the first kind of writers in that universe.

00:17:15.770 --> 00:17:19.939
<v Kristinn Þór Sigurbergsson>And it's up to the players to build that society up

00:17:20.020 --> 00:17:22.600
<v Kristinn Þór Sigurbergsson>and kind of build up all of that infrastructure.

00:17:23.780 --> 00:17:29.220
<v Kristinn Þór Sigurbergsson>It doesn't really differ from even a lot of like the design paradigms.

00:17:29.220 --> 00:17:34.100
<v Kristinn Þór Sigurbergsson>I think both games aim to give players as much player agency as they possibly can.

00:17:34.520 --> 00:17:41.900
<v Kristinn Þór Sigurbergsson>But this also allows us to make like a lot more dramatic changes in the gameplay itself.

00:17:42.080 --> 00:17:47.080
<v Kristinn Þór Sigurbergsson>it's very hard to change the gameplay on evil line for example very drastically because honestly

00:17:47.220 --> 00:17:51.380
<v Kristinn Þór Sigurbergsson>like after 20 years that's the game that people are playing they don't really like they didn't

00:17:51.410 --> 00:17:57.740
<v Kristinn Þór Sigurbergsson>play it for 20 years to change all of it so uh we've been doing that it's more survival

00:17:58.400 --> 00:18:05.100
<v Kristinn Þór Sigurbergsson>survival based it's more like uh like a survival game than it is uh uh not necessarily more than

00:18:05.100 --> 00:18:10.779
<v Kristinn Þór Sigurbergsson>an mmo but it definitely has a lot more survival elements to it uh yeah and the environments are

00:18:10.800 --> 00:18:13.320
<v Kristinn Þór Sigurbergsson>a little bit more kind of claustrophobic and smaller.

00:18:13.580 --> 00:18:15.140
<v Kristinn Þór Sigurbergsson>Obviously, that's very hard in a space game

00:18:15.230 --> 00:18:16.260
<v Kristinn Þór Sigurbergsson>because space is pretty vast.

00:18:17.340 --> 00:18:17.920
<v Michael Kennedy>Yeah, amazing.

00:18:18.310 --> 00:18:19.660
<v Michael Kennedy>Okay, really, really neat.

00:18:20.600 --> 00:18:23.940
<v Michael Kennedy>Now let's talk about the reason that I reached out to you,

00:18:24.060 --> 00:18:27.360
<v Michael Kennedy>which is I saw this article or post from you all

00:18:27.500 --> 00:18:29.720
<v Michael Kennedy>that said the move to Python 3 begins.

00:18:29.980 --> 00:18:31.880
<v Michael Kennedy>And, you know, credit to you all.

00:18:31.980 --> 00:18:34.200
<v Michael Kennedy>This is a really fun article that it's like,

00:18:34.280 --> 00:18:37.599
<v Michael Kennedy>it's told as sort of the next phase of a game

00:18:37.620 --> 00:18:41.880
<v Michael Kennedy>or something where it's kind of a mission in the game, I guess.

00:18:42.090 --> 00:18:43.380
<v Michael Kennedy>But a mission for you all, right?

00:18:44.190 --> 00:18:48.760
<v Jamie Bannister>Yeah, I think that we kind of got a little bit cheeky with the heading

00:18:48.850 --> 00:18:51.140
<v Jamie Bannister>because it isn't necessarily the beginning

00:18:51.420 --> 00:18:56.260
<v Jamie Bannister>because, like we were saying earlier, Frontier itself is already on Python 3.

00:18:57.220 --> 00:18:59.120
<v Jamie Bannister>But this is specifically for EVE Online itself.

00:19:00.280 --> 00:19:02.380
<v Jamie Bannister>It's now actively moving to Python 3.

00:19:02.490 --> 00:19:07.580
<v Jamie Bannister>And a lot of it is building up on the tech that we've developed

00:19:07.600 --> 00:19:12.160
<v Jamie Bannister>ready and kind of start to prove out through Frontier. But yeah, this is kind of our big

00:19:12.480 --> 00:19:19.860
<v Jamie Bannister>formal announcement that the legacy of 20 plus years of code, we're now kind of actively working

00:19:20.220 --> 00:19:26.260
<v Michael Kennedy>to migrate. I think we should start maybe with a little bit of the architecture and just a big

00:19:26.440 --> 00:19:31.900
<v Michael Kennedy>tech picture of EVE Online. When we spoke, Kristinn, way back in the day, you were doing a bunch of

00:19:32.100 --> 00:19:36.720
<v Michael Kennedy>interesting things. I think you must have been the single largest deployment of Python,

00:19:37.320 --> 00:19:38.800
<v Michael Kennedy>a Python app on Windows?

00:19:39.200 --> 00:19:39.700
<v Michael Kennedy>Is that correct?

00:19:40.000 --> 00:19:41.000
<v Michael Kennedy>I would say yes, right?

00:19:41.280 --> 00:19:42.140
<v Michael Kennedy>What else would it be?

00:19:42.360 --> 00:19:44.060
<v Michael Kennedy>That seems very plausible.

00:19:44.520 --> 00:19:48.440
<v Kristinn Þór Sigurbergsson>I think it was a very kind of like a niche environment.

00:19:48.600 --> 00:19:51.980
<v Kristinn Þór Sigurbergsson>And what I'd often run into is that I would try to take some libraries

00:19:52.260 --> 00:19:54.660
<v Kristinn Þór Sigurbergsson>and try to use them and they just wouldn't work on Windows.

00:19:54.800 --> 00:19:56.840
<v Kristinn Þór Sigurbergsson>Like somebody broke it and nobody cared

00:19:57.100 --> 00:19:59.540
<v Kristinn Þór Sigurbergsson>because they weren't really using it on Windows,

00:19:59.940 --> 00:20:00.540
<v Kristinn Þór Sigurbergsson>except we were.

00:20:01.300 --> 00:20:03.420
<v Kristinn Þór Sigurbergsson>Obviously, like at that time, like 10 years ago,

00:20:03.540 --> 00:20:07.680
<v Kristinn Þór Sigurbergsson>we couldn't really use a lot of, especially native Python

00:20:07.940 --> 00:20:08.160
<v Kristinn Þór Sigurbergsson>libraries.

00:20:08.580 --> 00:20:10.780
<v Kristinn Þór Sigurbergsson>I can't even remember if we had removed-

00:20:10.960 --> 00:20:11.540
<v Kristinn Þór Sigurbergsson>we must have removed.

00:20:11.780 --> 00:20:13.320
<v Kristinn Þór Sigurbergsson>We had our own importer as well, which

00:20:13.440 --> 00:20:16.160
<v Kristinn Þór Sigurbergsson>made it also difficult to use third party libraries.

00:20:17.260 --> 00:20:19.280
<v Kristinn Þór Sigurbergsson>But I think we fixed that.

00:20:20.000 --> 00:20:21.440
<v Thomas Dähling>Yeah, there's a lot of-

00:20:21.660 --> 00:20:22.900
<v Thomas Dähling>I mean, it's a very old project.

00:20:23.500 --> 00:20:25.780
<v Thomas Dähling>And a lot of the early beginnings in the code base

00:20:25.940 --> 00:20:27.620
<v Thomas Dähling>were before a lot of the things in Python

00:20:27.760 --> 00:20:29.220
<v Thomas Dähling>actually being standardized to a degree.

00:20:30.320 --> 00:20:35.120
<v Thomas Dähling>plus the special circumstances of having a packaged program

00:20:35.340 --> 00:20:36.560
<v Thomas Dähling>with its own Python interpreter

00:20:37.720 --> 00:20:39.120
<v Thomas Dähling>just makes certain things more interesting.

00:20:39.400 --> 00:20:40.900
<v Thomas Dähling>Like, you know, as Kristen alluded to,

00:20:40.980 --> 00:20:44.060
<v Thomas Dähling>like you can't just import a random public package

00:20:44.360 --> 00:20:46.680
<v Thomas Dähling>because we don't have pip as an example, right?

00:20:47.100 --> 00:20:48.060
<v Thomas Dähling>Or uv these days.

00:20:48.220 --> 00:20:51.020
<v Thomas Dähling>But like, there's a lot of interesting problems

00:20:51.100 --> 00:20:52.860
<v Thomas Dähling>you need to solve when you have such a large deployment,

00:20:53.160 --> 00:20:53.420
<v Thomas Dähling>for sure.

00:20:53.620 --> 00:20:54.840
<v Michael Kennedy>Yeah, there was a lot of custom stuff.

00:20:55.060 --> 00:20:56.340
<v Michael Kennedy>When did this first get created?

00:20:56.480 --> 00:20:58.000
<v Michael Kennedy>So I know we spoke in 2016,

00:20:58.320 --> 00:21:00.720
<v Michael Kennedy>but it's been around for more like 20 years, right?

00:21:00.920 --> 00:21:02.320
<v Michael Kennedy>So 2005, 2006?

00:21:04.520 --> 00:21:06.080
<v Michael Kennedy>I think it's earlier than that.

00:21:06.240 --> 00:21:09.340
<v Kristinn Þór Sigurbergsson>It's like the game is released that 2003,

00:21:10.120 --> 00:21:14.220
<v Kristinn Þór Sigurbergsson>the development started in 99, I think.

00:21:15.440 --> 00:21:17.460
<v Kristinn Þór Sigurbergsson>They didn't really choose Python.

00:21:18.290 --> 00:21:20.840
<v Kristinn Þór Sigurbergsson>I don't know the exact date of that,

00:21:21.100 --> 00:21:25.040
<v Kristinn Þór Sigurbergsson>but like there was, that is an interesting story as well.

00:21:25.110 --> 00:21:26.220
<v Kristinn Þór Sigurbergsson>Like why they chose Python,

00:21:26.800 --> 00:21:29.260
<v Kristinn Þór Sigurbergsson>which was actually rather new at the time.

00:21:29.800 --> 00:21:31.720
<v Michael Kennedy>Yeah, I was actually wondering about that as well.

00:21:32.080 --> 00:21:33.320
<v Michael Kennedy>I'm glad you chose Python.

00:21:33.460 --> 00:21:34.380
<v Michael Kennedy>I think it's really cool.

00:21:34.720 --> 00:21:36.960
<v Michael Kennedy>And I'm sure you all really enjoy working on it.

00:21:36.960 --> 00:21:39.460
<v Michael Kennedy>And this has got to be an exciting project to say.

00:21:39.560 --> 00:21:40.680
<v Michael Kennedy>We're moving to 3.15.

00:21:41.060 --> 00:21:41.760
<v Michael Kennedy>We can use uv.

00:21:42.240 --> 00:21:45.900
<v Michael Kennedy>We can import libraries, like all the exciting stuff.

00:21:46.200 --> 00:21:48.140
<v Michael Kennedy>But at the time, it was pretty new.

00:21:48.640 --> 00:21:52.040
<v Michael Kennedy>And we said, well, we could run this much faster and easier if we wrote it,

00:21:52.420 --> 00:21:56.620
<v Michael Kennedy>I don't know, with C++, Java, C#, something like that, right?

00:21:57.320 --> 00:22:25.700
<v Jamie Bannister>I think a lot of the core libraries, they are in C++, so our physics engine, which we call Destiny, that's all C++. Trinity, our graphics engine, similarly. But yeah, most of our kind of higher level, all the gameplay logic, a lot of the network stack that we have, that's all built, or clustering stack, that's all built in Python.

00:22:25.780 --> 00:22:37.240
<v Kristinn Þór Sigurbergsson>Yeah, probably add to that as well is that like the time when we started or not me, even though I've been here for a very long time, it hasn't been that long.

00:22:37.660 --> 00:22:45.700
<v Kristinn Þór Sigurbergsson>But when they started writing EVE, like that was around the.com era and kind of the older listeners probably remember that era.

00:22:45.920 --> 00:22:49.700
<v Kristinn Þór Sigurbergsson>But the programming resources were scarce and heavily sought after.

00:22:49.840 --> 00:22:51.480
<v Kristinn Þór Sigurbergsson>So very expensive.

00:22:51.760 --> 00:22:57.460
<v Kristinn Þór Sigurbergsson>So part of the reason why they chose Python was to get more junior people and even designers

00:22:57.600 --> 00:23:04.640
<v Kristinn Þór Sigurbergsson>to be able to write sequential code in Python in order to speed up development.

00:23:04.720 --> 00:23:06.320
<v Kristinn Þór Sigurbergsson>It was all about development speed.

00:23:06.580 --> 00:23:12.040
<v Jamie Bannister>Especially in Iceland, right, where you've got a much smaller pool of people to work from.

00:23:12.320 --> 00:23:15.000
<v Michael Kennedy>Honestly, that's kind of a cool success story, right?

00:23:15.260 --> 00:23:18.800
<v Michael Kennedy>Like bring more people in to help build the thing that we're building.

00:23:19.360 --> 00:23:20.140
<v Thomas Dähling>Yeah, absolutely.

00:23:21.380 --> 00:23:38.860
<v Thomas Dähling>Yeah, absolutely. And as Kristin just said, allowing people to write sequential, simple, straightforward Python code as well, which then led to us discovering stackless Python, integrating that one, and then building an abstraction on top that people don't realize that they are writing code that runs asynchronously.

00:23:40.220 --> 00:23:47.340
<v Thomas Dähling>And it enabled us to scale to the degree that we're able to scale with thousands of people being in the same solar system and so on.

00:23:47.540 --> 00:23:52.440
<v Michael Kennedy>Yeah, you needed concurrency before Python was inspired by concurrency.

00:23:52.880 --> 00:23:56.040
<v Michael Kennedy>Maybe we should talk a little bit about free-threaded Python later,

00:23:56.380 --> 00:23:57.820
<v Michael Kennedy>as well as the possibilities there.

00:23:58.020 --> 00:24:02.080
<v Michael Kennedy>But I just want to kind of touch on this historical angle a little bit more.

00:24:02.400 --> 00:24:07.400
<v Michael Kennedy>To me, there's two really significant milestones that matured Python.

00:24:07.860 --> 00:24:10.200
<v Michael Kennedy>I'm sure there are more, but these are the ones that stand out in my mind.

00:24:10.500 --> 00:24:14.420
<v Michael Kennedy>They both are around 2006 or so time frame.

00:24:14.920 --> 00:24:20.800
<v Michael Kennedy>So one is Django coming to Python, really making it a first class way to build web apps.

00:24:21.260 --> 00:24:23.040
<v Michael Kennedy>People start building on top of that.

00:24:23.240 --> 00:24:27.360
<v Michael Kennedy>That puts a lot of pressure to make Python operate a little bit differently, right?

00:24:28.340 --> 00:24:32.840
<v Michael Kennedy>10 years later, we've got Instagram running the largest Django deployment in the world

00:24:32.920 --> 00:24:38.940
<v Michael Kennedy>and doing things like trying to juggle weird memory stuff by turning off the GC, for example,

00:24:39.220 --> 00:24:41.100
<v Michael Kennedy>entirely and other things along those lines.

00:24:41.680 --> 00:24:56.220
<v Michael Kennedy>The other one around 2006 as well is the invention of NumPy and then later Jupyter and the whole data science stack, which also put really hard computational pressure on Python rather than, say, the parallelism of web apps and so on.

00:24:56.480 --> 00:24:57.920
<v Michael Kennedy>You all were before both of those, right?

00:24:58.020 --> 00:24:59.680
<v Michael Kennedy>So it was early, early days.

00:25:00.820 --> 00:25:00.900
<v Jamie Bannister>Yeah.

00:25:01.100 --> 00:25:10.880
<v Jamie Bannister>I mean, what we also found was, or what I found in some of work here is the original developers had to invent a lot of things which didn't exist.

00:25:11.240 --> 00:25:18.280
<v Jamie Bannister>So like the containerized system, a way to kind of cluster an application across multiple

00:25:19.720 --> 00:25:20.900
<v Jamie Bannister>CPUs and notes.

00:25:21.200 --> 00:25:24.180
<v Jamie Bannister>There wasn't really a lot of easy off the shelf frameworks for that kind of thing.

00:25:24.460 --> 00:25:25.600
<v Jamie Bannister>So they had to roll their own.

00:25:27.780 --> 00:25:32.640
<v Jamie Bannister>Even logging and telemetry systems, there weren't great ones of those.

00:25:32.880 --> 00:25:39.019
<v Jamie Bannister>So what we found from the early years is we have a lot of in-house tools, which we made

00:25:39.040 --> 00:25:47.580
<v Jamie Bannister>or the people at the time made, which are now using the models that you see being replicated

00:25:48.000 --> 00:25:49.680
<v Jamie Bannister>by much more popular systems.

00:25:50.400 --> 00:25:55.600
<v Jamie Bannister>So whether it be Redis for a storage system or your containers as well.

00:25:56.480 --> 00:26:00.420
<v Jamie Bannister>The way that maybe we can talk about this a bit more in a bit is the way we distribute

00:26:00.740 --> 00:26:05.920
<v Jamie Bannister>EVE as an application across about 200 nodes in our server cluster.

00:26:07.020 --> 00:26:10.260
<v Jamie Bannister>And each of those has certain jobs to do.

00:26:11.460 --> 00:26:16.260
<v Jamie Bannister>And so we have to deploy and configure all of those as one giant cluster.

00:26:16.480 --> 00:26:19.500
<v Jamie Bannister>And we have our own coordination tools that have been built over the years.

00:26:20.340 --> 00:26:21.840
<v Jamie Bannister>Nowadays, if you were starting this from scratch,

00:26:22.390 --> 00:26:26.000
<v Jamie Bannister>you would have so many off-the-shelf solutions that we just didn't have the options for.

00:26:26.520 --> 00:26:29.840
<v Michael Kennedy>Oh, you just set up Kubernetes and then do this auto-scaling thing.

00:26:30.110 --> 00:26:31.340
<v Michael Kennedy>And it's, oh, yeah, it's just easy.

00:26:33.400 --> 00:26:40.640
<v Kristinn Þór Sigurbergsson>I would also like to add, I feel like the data scientists kind of grabbing onto Python and

00:26:40.640 --> 00:26:48.140
<v Kristinn Þór Sigurbergsson>I feel like pushing it towards Python 3 because I think without that army going into Python

00:26:48.140 --> 00:26:52.960
<v Kristinn Þór Sigurbergsson>3, I feel like it mirrors the rationale of why we chose to go into stackless Python.

00:26:53.200 --> 00:26:57.240
<v Kristinn Þór Sigurbergsson>You were giving basically a tool to a bunch of people that aren't really programmers,

00:26:57.390 --> 00:26:59.620
<v Kristinn Þór Sigurbergsson>but they need to write code.

00:26:59.920 --> 00:27:02.860
<v Kristinn Þór Sigurbergsson>So I feel like that is an interesting thing.

00:27:03.260 --> 00:27:07.300
<v Michael Kennedy>Yeah, you know, I've forgotten the 2v3 wars and all that.

00:27:07.420 --> 00:27:12.140
<v Michael Kennedy>Now, yeah, it's easy to forget the details of that from, it's been a while now, actually,

00:27:12.420 --> 00:27:13.080
<v Michael Kennedy>but you're right.

00:27:13.110 --> 00:27:18.240
<v Michael Kennedy>It was the data scientists that were going into Python 3, and the rest of the folks were

00:27:18.620 --> 00:27:23.740
<v Michael Kennedy>kind of dragging their feet because they had existing code bases that needed to keep running

00:27:23.900 --> 00:27:27.860
<v Michael Kennedy>and keep being migrated, whereas the data science projects are more frequently like,

00:27:27.900 --> 00:27:31.040
<v Michael Kennedy>we'll start a new project to answer this question

00:27:31.140 --> 00:27:32.600
<v Michael Kennedy>or to study this new bit of data.

00:27:32.740 --> 00:27:34.780
<v Michael Kennedy>So they had many more opportunities

00:27:35.080 --> 00:27:36.840
<v Michael Kennedy>to choose the most modern tools.

00:27:37.240 --> 00:27:39.860
<v Michael Kennedy>And I think, honestly, we should give the data scientists

00:27:40.480 --> 00:27:43.680
<v Michael Kennedy>a little bit more credit of making the two to three change happen.

00:27:44.140 --> 00:27:45.800
<v Michael Kennedy>All right, well, let's talk stackless Python

00:27:46.420 --> 00:27:48.960
<v Michael Kennedy>because this is one of the foundations

00:27:49.600 --> 00:27:51.040
<v Michael Kennedy>that you all had to migrate from.

00:27:51.720 --> 00:27:55.400
<v Michael Kennedy>And the journey begins,

00:27:55.540 --> 00:28:02.620
<v Michael Kennedy>The move to Python 3 begins article talks about upgrading from 2.5 to 2.7 not too long ago.

00:28:02.770 --> 00:28:05.280
<v Michael Kennedy>And then finally up to 3.8.

00:28:06.270 --> 00:28:07.840
<v Michael Kennedy>What is it about Stackless Python?

00:28:07.970 --> 00:28:14.400
<v Michael Kennedy>I know it has strong origins with you all because some of the core developers worked on it and refined it there.

00:28:14.840 --> 00:28:18.580
<v Michael Kennedy>But this far down the line, I imagine there's a lot of people that don't know what this is.

00:28:19.700 --> 00:28:20.160
<v Thomas Dähling>Yeah, absolutely.

00:28:21.880 --> 00:28:25.080
<v Thomas Dähling>Stackless Python itself was a very great solution at the time, right?

00:28:25.280 --> 00:28:28.380
<v Thomas Dähling>I mean, it's basically Golang's GoRoutines, but in Python,

00:28:28.840 --> 00:28:32.620
<v Thomas Dähling>before things like this co-routine model

00:28:32.860 --> 00:28:34.800
<v Thomas Dähling>became so popular as it is in recent years.

00:28:35.940 --> 00:28:40.160
<v Thomas Dähling>A big problem that we faced was, on the one hand,

00:28:40.460 --> 00:28:42.680
<v Thomas Dähling>that because of the customizations

00:28:42.760 --> 00:28:46.480
<v Thomas Dähling>we had to do to the network stack to make all the magic of people

00:28:46.540 --> 00:28:49.320
<v Thomas Dähling>writing sequential code, but it does end up not blocking

00:28:49.480 --> 00:28:50.000
<v Thomas Dähling>the main process.

00:28:51.360 --> 00:28:53.580
<v Thomas Dähling>That was a big learning curve for a lot of people.

00:28:54.660 --> 00:28:57.620
<v Thomas Dähling>And also always something that even the experienced developers

00:28:58.280 --> 00:29:00.640
<v Thomas Dähling>tended to forget about in a moment.

00:29:00.740 --> 00:29:02.660
<v Thomas Dähling>And they're like, oh, yeah, hang on, there's a DB call.

00:29:02.800 --> 00:29:03.700
<v Thomas Dähling>Of course, that we yield.

00:29:03.760 --> 00:29:05.120
<v Thomas Dähling>I need to put a lock here or whatever.

00:29:06.840 --> 00:29:09.800
<v Thomas Dähling>So the model in itself is great, but it

00:29:09.920 --> 00:29:13.720
<v Thomas Dähling>comes with a few learning hurdles, I want to say.

00:29:15.080 --> 00:29:16.560
<v Thomas Dähling>And it served us really well.

00:29:17.240 --> 00:29:19.420
<v Thomas Dähling>One problem that we had over the years

00:29:20.640 --> 00:29:22.719
<v Thomas Dähling>that had become more and more interesting, especially

00:29:22.740 --> 00:29:28.600
<v Thomas Dähling>last 15 years, I want to say, is that we relied on a heavily customized version of it. So stackless

00:29:28.720 --> 00:29:34.640
<v Thomas Dähling>Python itself is a fork of the CPython interpreter with a lot of changes in the interpreter. And

00:29:35.530 --> 00:29:43.080
<v Thomas Dähling>we put some changes on top because doing a real-time game of sorts requires some solutions that go

00:29:43.220 --> 00:29:47.640
<v Thomas Dähling>very far down the tech stack to be performant. Like if you want to get certain metrics, if you want

00:29:47.460 --> 00:29:53.200
<v Thomas Dähling>to control how memory is allocated and so on and so forth right so all these customizations and

00:29:53.240 --> 00:29:58.000
<v Thomas Dähling>also meant that made it very difficult for us to like always go to the next newer version of python

00:29:58.220 --> 00:30:02.400
<v Thomas Dähling>as the article current calls out like you know there was some code that was still python 2.5

00:30:02.880 --> 00:30:07.740
<v Thomas Dähling>when we did the native extension porting to python 3 we discovered things from python 2.3 that we were

00:30:07.900 --> 00:30:12.239
<v Thomas Dähling>still using that shouldn't have been used but we somehow kept around on our customized version of

00:30:12.260 --> 00:30:12.900
<v Thomas Dähling>of Stackless Python.

00:30:14.380 --> 00:30:17.600
<v Michael Kennedy>This portion of Talk Python is brought to you by Talk Python courses.

00:30:18.180 --> 00:30:19.560
<v Michael Kennedy>Here's the thing that always bug me.

00:30:19.850 --> 00:30:22.300
<v Michael Kennedy>You finish one of our courses, that's hours of video,

00:30:23.000 --> 00:30:24.400
<v Michael Kennedy>a pile of code you actually wrote,

00:30:24.520 --> 00:30:26.380
<v Michael Kennedy>and real skills you didn't have a month before,

00:30:27.060 --> 00:30:27.920
<v Michael Kennedy>and then nothing happens.

00:30:28.280 --> 00:30:30.900
<v Michael Kennedy>No paper, no credential, nothing to show for it.

00:30:31.440 --> 00:30:32.100
<v Michael Kennedy>So we fixed it.

00:30:32.410 --> 00:30:36.220
<v Michael Kennedy>Every Talk Python course now generates a completion certificate automatically.

00:30:36.660 --> 00:30:38.620
<v Michael Kennedy>Go to your account page in your dashboard section,

00:30:39.100 --> 00:30:42.180
<v Michael Kennedy>scroll down to your completed courses and click certificate.

00:30:42.780 --> 00:30:43.400
<v Michael Kennedy>That's the whole process.

00:30:44.380 --> 00:30:46.980
<v Michael Kennedy>Two things you can do with these course completion certificates,

00:30:47.320 --> 00:30:48.380
<v Michael Kennedy>download the full PDF,

00:30:48.940 --> 00:30:51.480
<v Michael Kennedy>which is handy if your employer reimburses training

00:30:51.610 --> 00:30:53.160
<v Michael Kennedy>or gives you credit for finishing it.

00:30:53.720 --> 00:30:57.020
<v Michael Kennedy>Or you can make the certificate public and hit share on LinkedIn,

00:30:57.400 --> 00:31:00.740
<v Michael Kennedy>which adds it to your LinkedIn profile under licenses and certifications.

00:31:01.400 --> 00:31:02.860
<v Michael Kennedy>Not a poster that scrolls away in a day,

00:31:02.950 --> 00:31:05.180
<v Michael Kennedy>an actual credential sitting on your profile

00:31:05.360 --> 00:31:07.280
<v Michael Kennedy>where your manager and recruiters can see it.

00:31:07.780 --> 00:31:12.980
<v Michael Kennedy>Plus, if you've been taking our courses for a while, you've probably earned several of these without even knowing they existed.

00:31:13.450 --> 00:31:17.320
<v Michael Kennedy>Just visit training.Talk Python.fm slash account and collect them.

00:31:18.080 --> 00:31:20.640
<v Michael Kennedy>Thanks to all of you who have taken a Talk Python course.

00:31:21.220 --> 00:31:22.420
<v Michael Kennedy>It's a great way to support the podcast.

00:31:24.180 --> 00:31:31.540
<v Thomas Dähling>Another aspect certainly is that stack lists had become like a sort of niche project in the ecosystem overall.

00:31:32.040 --> 00:31:36.560
<v Thomas Dähling>With Python 3, we have async.io built on the twisted model.

00:31:37.620 --> 00:31:40.800
<v Thomas Dähling>And even in the Python 2 world, G event

00:31:40.950 --> 00:31:43.020
<v Thomas Dähling>had become a lot more famous than Stackless Python,

00:31:43.580 --> 00:31:45.160
<v Thomas Dähling>presumably because you could just import

00:31:45.280 --> 00:31:47.300
<v Thomas Dähling>G event in a standard CPython interpreter,

00:31:47.600 --> 00:31:51.620
<v Thomas Dähling>while Stackless Python is more complicated to get started with.

00:31:53.210 --> 00:31:53.320
<v Thomas Dähling>So-

00:31:53.320 --> 00:31:55.540
<v Michael Kennedy>So Stackless Python is basically its own runtime.

00:31:56.100 --> 00:31:57.200
<v Thomas Dähling>Yeah, it's basically-

00:31:57.220 --> 00:31:57.300
<v Thomas Dähling>Yeah.

00:31:57.640 --> 00:31:58.560
<v Thomas Dähling>Yeah, very much that.

00:31:58.660 --> 00:31:59.840
<v Thomas Dähling>It's like its own Python runtime.

00:32:00.460 --> 00:32:01.760
<v Thomas Dähling>You kind of need your own.

00:32:01.870 --> 00:32:04.300
<v Thomas Dähling>Like, it has a stackless lib, so to speak,

00:32:04.480 --> 00:32:06.320
<v Thomas Dähling>which basically monkey patches similar to what

00:32:06.320 --> 00:32:09.920
<v Thomas Dähling>Gvent does all of the threading and socket modules and so on

00:32:10.340 --> 00:32:11.840
<v Thomas Dähling>to be able to do them in non-blocking fashion.

00:32:12.919 --> 00:32:14.340
<v Thomas Dähling>And yeah, all these things.

00:32:14.580 --> 00:32:17.680
<v Michael Kennedy>So that's really, I mean, that's what made it possible.

00:32:17.980 --> 00:32:18.460
<v Michael Kennedy>That's really awesome.

00:32:18.720 --> 00:32:20.460
<v Michael Kennedy>One thing, just, I don't know if you all know this,

00:32:20.500 --> 00:32:24.840
<v Michael Kennedy>but how is it possible that it says it's forked from CPython on GitHub

00:32:25.340 --> 00:32:29.600
<v Michael Kennedy>when CPython itself didn't go to GitHub until like the 20s,

00:32:29.700 --> 00:32:31.460
<v Michael Kennedy>you know, 2020, I don't know what it was,

00:32:31.640 --> 00:32:33.980
<v Michael Kennedy>but maybe some GitHub magic, I don't know.

00:32:34.180 --> 00:32:35.020
<v Thomas Dähling>Oh, it's git magic.

00:32:35.280 --> 00:32:44.340
<v Thomas Dähling>You will find on GitHub there are repositories where there are commits from Abraham Lincoln and various other historical figures because you can just make up all the history, right?

00:32:45.040 --> 00:32:45.440
<v Thomas Dähling>Interesting.

00:32:45.780 --> 00:32:48.140
<v Michael Kennedy>Yeah, it is nice, though, actually, that it shows it.

00:32:48.500 --> 00:32:56.460
<v Michael Kennedy>If you go to the repo now, it says that it was archived in 2025, which, you know, not every project is meant to last forever.

00:32:57.040 --> 00:32:58.560
<v Michael Kennedy>But it made it possible for you.

00:32:58.920 --> 00:33:01.800
<v Michael Kennedy>Although I'm sure that you would think, you know, all right.

00:33:02.040 --> 00:33:11.040
<v Michael Kennedy>You're like, wouldn't it be cool if we could use, I don't know, libraries like Pydantic or FastAPI or AIO Redis or I don't know, whatever, right?

00:33:11.240 --> 00:33:13.100
<v Michael Kennedy>Anything that is new and fancy.

00:33:13.540 --> 00:33:16.940
<v Michael Kennedy>And then you go back and you look and you're like, well, but not here.

00:33:17.320 --> 00:33:18.200
<v Michael Kennedy>Not on Stackless, right?

00:33:18.340 --> 00:33:20.300
<v Michael Kennedy>Like that was one of the major problems, wasn't it?

00:33:21.700 --> 00:33:24.200
<v Thomas Dähling>Yes, this was a major headache for us.

00:33:24.340 --> 00:33:26.420
<v Thomas Dähling>But for every native extension-

00:33:26.420 --> 00:33:28.680
<v Thomas Dähling>or not just native extensions-- for every Python package

00:33:28.720 --> 00:33:31.000
<v Thomas Dähling>we took on, we had to take a really close look

00:33:32.200 --> 00:33:34.340
<v Thomas Dähling>about how can we integrate it in our stack.

00:33:34.980 --> 00:33:36.580
<v Thomas Dähling>Again, we didn't have a package manager.

00:33:36.780 --> 00:33:38.380
<v Thomas Dähling>We still don't have it, but we're working on it.

00:33:39.299 --> 00:33:41.780
<v Thomas Dähling>And that just meant that you would take the zip file

00:33:41.860 --> 00:33:45.200
<v Thomas Dähling>from PyPy, for example, extract it into our environment,

00:33:45.460 --> 00:33:46.580
<v Thomas Dähling>and then see what happens.

00:33:47.220 --> 00:33:48.980
<v Thomas Dähling>If that then happened to be something that

00:33:48.980 --> 00:33:51.240
<v Thomas Dähling>had a native extension, well, good luck.

00:33:51.580 --> 00:33:53.460
<v Thomas Dähling>Does this support our minimum spec environment?

00:33:53.820 --> 00:33:55.140
<v Thomas Dähling>does this run on all our environments?

00:33:55.660 --> 00:34:00.340
<v Thomas Dähling>Does this work with our version of the forked Python interpreter?

00:34:00.880 --> 00:34:04.120
<v Thomas Dähling>And we have discovered a few cases

00:34:04.420 --> 00:34:08.399
<v Thomas Dähling>where we took on modules that were built for 2.7.18,

00:34:08.960 --> 00:34:12.020
<v Thomas Dähling>and they then corrupted memory because in 2.7.6,

00:34:12.080 --> 00:34:13.960
<v Thomas Dähling>the memory layout of the PyType object changed,

00:34:14.419 --> 00:34:15.879
<v Thomas Dähling>and we didn't have that change.

00:34:16.139 --> 00:34:17.780
<v Thomas Dähling>And yeah, things like that.

00:34:17.919 --> 00:34:19.960
<v Thomas Dähling>So we had to fight the way forward.

00:34:20.480 --> 00:34:21.540
<v Jamie Bannister>Jamie, you want to jump in?

00:34:21.980 --> 00:34:24.780
<v Jamie Bannister>Yeah, no, I was just going to say when it comes to support as well,

00:34:25.020 --> 00:34:28.340
<v Jamie Bannister>because we support Mac and we have in the past,

00:34:28.399 --> 00:34:30.280
<v Jamie Bannister>but we don't now support Linux as well.

00:34:30.600 --> 00:34:33.580
<v Jamie Bannister>So sometimes with a native extension,

00:34:34.120 --> 00:34:37.220
<v Jamie Bannister>we've got to figure out does it even support all of our clients?

00:34:38.620 --> 00:34:41.000
<v Jamie Bannister>Our servers run on a Windows cluster.

00:34:42.100 --> 00:34:44.480
<v Jamie Bannister>Most of our players are running on Windows clients.

00:34:45.740 --> 00:34:48.260
<v Jamie Bannister>Mac is also kind of a smaller majority.

00:34:48.560 --> 00:34:50.379
<v Jamie Bannister>And like I say, we used to also support Linux.

00:34:51.320 --> 00:34:57.340
<v Jamie Bannister>So keeping all of those in sync as well with all of these extensions has been problematic at times.

00:34:57.680 --> 00:34:57.920
<v Michael Kennedy>Yeah.

00:34:58.680 --> 00:34:59.520
<v Michael Kennedy>Sure, that's an understatement.

00:34:59.820 --> 00:35:09.240
<v Michael Kennedy>Earlier, when you guys mentioned how you deploy as an app, or.app,.exe, whatever, or to your server clusters, how do you do that?

00:35:09.680 --> 00:35:18.580
<v Michael Kennedy>That's honestly one of the biggest challenges of Python that many attempts have been made, but I honestly don't think that there's a good answer, a really foolproof answer yet.

00:35:18.760 --> 00:35:27.760
<v Michael Kennedy>And until it's built into CPython itself with a --build flag or something, I think it's going to be 90, 95% solutions.

00:35:28.060 --> 00:35:29.300
<v Michael Kennedy>But you all have something that works.

00:35:29.740 --> 00:35:30.240
<v Michael Kennedy>What are you doing?

00:35:31.020 --> 00:35:37.860
<v Thomas Dähling>So effectively, I mean, with Python 3, this all got a lot easier because you can initialize the interpreter a lot more straightforward.

00:35:38.840 --> 00:35:47.060
<v Thomas Dähling>But what we have been doing for the last 20 years is effectively we are embedding the Python interpreter in a custom binary that we have, a custom library that we use.

00:35:48.540 --> 00:35:51.860
<v Thomas Dähling>And from that moment onwards, we basically just heavily

00:35:52.200 --> 00:35:54.860
<v Thomas Dähling>customize what paths are available to the interpreter

00:35:55.380 --> 00:35:57.580
<v Thomas Dähling>and what environment variables are considered,

00:35:57.920 --> 00:35:58.980
<v Thomas Dähling>all these little things.

00:36:00.480 --> 00:36:02.300
<v Thomas Dähling>And then we package up the remaining things

00:36:02.330 --> 00:36:05.360
<v Thomas Dähling>that we need in a way that it fits this layout that we set up

00:36:05.560 --> 00:36:06.860
<v Thomas Dähling>in the embedded interpreter.

00:36:07.110 --> 00:36:11.260
<v Thomas Dähling>But yeah, it's also more like a 75% solution than a real one

00:36:12.190 --> 00:36:13.720
<v Thomas Dähling>because the Python build process in itself

00:36:14.700 --> 00:36:16.960
<v Thomas Dähling>requires some system-specific paths.

00:36:17.580 --> 00:36:19.820
<v Thomas Dähling>We had to go to great lengths to patch that out.

00:36:20.780 --> 00:36:22.480
<v Thomas Dähling>Not necessarily in the best way possible either,

00:36:22.740 --> 00:36:23.900
<v Thomas Dähling>but it gets us there.

00:36:24.120 --> 00:36:27.140
<v Michael Kennedy>If you're shipping it, that's off you all.

00:36:27.200 --> 00:36:28.380
<v Michael Kennedy>That's pretty impressive.

00:36:28.720 --> 00:36:31.460
<v Kristinn Þór Sigurbergsson>It was kind of like that we have a solution for our players,

00:36:31.560 --> 00:36:35.220
<v Kristinn Þór Sigurbergsson>but also we built a lot of tools for game designers

00:36:35.320 --> 00:36:36.380
<v Kristinn Þór Sigurbergsson>to use and stuff like that.

00:36:36.400 --> 00:36:38.220
<v Kristinn Þór Sigurbergsson>And it's convenient to write them in Python

00:36:38.400 --> 00:36:41.080
<v Kristinn Þór Sigurbergsson>because we can share some of the code base of the actual game,

00:36:41.380 --> 00:36:44.260
<v Kristinn Þór Sigurbergsson>game logic, for example, when we're creating static data.

00:36:44.760 --> 00:36:49.800
<v Kristinn Þór Sigurbergsson>And that has been like a very painful experience of distributing Python,

00:36:50.200 --> 00:36:53.320
<v Kristinn Þór Sigurbergsson>like Python, basically a Python script to developers,

00:36:54.020 --> 00:36:58.900
<v Kristinn Þór Sigurbergsson>because there is really amazing how much people can mess with their environment

00:36:59.000 --> 00:37:02.420
<v Kristinn Þór Sigurbergsson>by just changing the Python path, running the wrong version of Python.

00:37:04.160 --> 00:37:06.660
<v Kristinn Þór Sigurbergsson>It's been a struggle for years, honestly.

00:37:07.619 --> 00:37:12.600
<v Jamie Bannister>Even if we get a new hire, the first few days are often spent setting the machines up.

00:37:13.080 --> 00:37:20.820
<v Jamie Bannister>And inevitably, as Windows slightly changes from release to release, it will or won't come with certain things.

00:37:21.690 --> 00:37:29.980
<v Jamie Bannister>Sometimes we have to disable the built-in Python that ships with Windows just so that we can use our own specific Python 2.7 versus Python 3.

00:37:31.120 --> 00:37:36.340
<v Jamie Bannister>So that's also quite a lot of fun that we have to get started out.

00:37:36.460 --> 00:37:40.100
<v Michael Kennedy>What about things like uv Run or uvx in the future?

00:37:41.140 --> 00:37:43.220
<v Michael Kennedy>There's some interesting possibilities there,

00:37:43.500 --> 00:37:43.660
<v Michael Kennedy>potentially.

00:37:44.140 --> 00:37:44.500
<v Thomas Dähling>Absolutely.

00:37:45.180 --> 00:37:47.620
<v Thomas Dähling>And as I said, we are working on it.

00:37:47.840 --> 00:37:49.520
<v Thomas Dähling>We are exploring the angle there.

00:37:50.900 --> 00:37:54.220
<v Thomas Dähling>Because it is still a bit of a hassle of getting things

00:37:54.480 --> 00:37:57.300
<v Thomas Dähling>into our special environment for the game itself.

00:37:58.460 --> 00:38:00.900
<v Thomas Dähling>We have historically had also the issue

00:38:00.930 --> 00:38:02.580
<v Thomas Dähling>that we were using a different built environment

00:38:02.710 --> 00:38:04.620
<v Thomas Dähling>than the standard Python distributions, right?

00:38:05.560 --> 00:38:07.700
<v Thomas Dähling>Partially because there's a lot of code

00:38:07.720 --> 00:38:08.600
<v Thomas Dähling>that would need to be migrated.

00:38:08.860 --> 00:38:10.740
<v Thomas Dähling>Partially because it works.

00:38:11.020 --> 00:38:13.560
<v Thomas Dähling>We don't want to spend resources on that at the moment,

00:38:14.900 --> 00:38:17.980
<v Thomas Dähling>partially because, well, especially with Stackless Python,

00:38:18.380 --> 00:38:20.840
<v Thomas Dähling>you could only go so far until there's a new compiler version

00:38:21.040 --> 00:38:22.580
<v Thomas Dähling>that then breaks some internals, and you would have

00:38:22.680 --> 00:38:23.460
<v Thomas Dähling>to do major rewrites.

00:38:24.920 --> 00:38:27.080
<v Thomas Dähling>So these issues have existed, but we

00:38:28.500 --> 00:38:31.020
<v Thomas Dähling>want to be able to be in a place where a developer can also

00:38:31.200 --> 00:38:35.820
<v Thomas Dähling>just say, "uv install this package into my game environment,"

00:38:35.820 --> 00:38:36.580
<v Thomas Dähling>then be able to use it.

00:38:37.500 --> 00:38:39.860
<v Thomas Dähling>Needs to solve a few things, like if it's a native extension,

00:38:40.360 --> 00:38:44.460
<v Thomas Dähling>needs to find the right build environment and so on, which is non-trivial, unfortunately.

00:38:45.440 --> 00:38:50.760
<v Michael Kennedy>What workflows do you think, do you see becoming possible that were much, much harder once you

00:38:50.900 --> 00:38:57.020
<v Michael Kennedy>have nice package managers and maybe more reuse, right? For example, you could ship versions of

00:38:57.180 --> 00:39:03.500
<v Michael Kennedy>things more easily than, you know, as one monolithic element. Yeah, for sure. I mean, for example,

00:39:03.540 --> 00:39:09.820
<v Thomas Dähling>we recently open-sourced basically the game engine, right? And there is, I can see a world where you

00:39:09.740 --> 00:39:15.320
<v Thomas Dähling>can just uv install or pip install these things into your custom interpreter to get going or into

00:39:15.500 --> 00:39:21.240
<v Thomas Dähling>your Python interpreter to get going without requiring our custom interpreter. For our internal

00:39:21.480 --> 00:39:26.740
<v Thomas Dähling>workflows, I don't think, I mean, sure, the acquisition of an external package will definitely

00:39:27.060 --> 00:39:32.060
<v Thomas Dähling>get a lot easier, but many of the other workflows probably won't change that much since we,

00:39:33.200 --> 00:39:39.140
<v Thomas Dähling>I was in 2011 or 12 or so, we added an interpreter mode to the game engine itself, so you can

00:39:38.940 --> 00:39:42.780
<v Thomas Dähling>can basically start the game engine with a special flag, and then it basically starts

00:39:42.780 --> 00:39:48.300
<v Thomas Dähling>the Python REPL, and you get as much of the standard Python interpreter behavior as possible.

00:39:49.220 --> 00:39:54.720
<v Michael Kennedy>That's cool. So you can kind of poke around that in the game memory and values and data

00:39:54.880 --> 00:39:56.620
<v Michael Kennedy>structures as you want, right?

00:39:56.960 --> 00:40:03.760
<v Kristinn Þór Sigurbergsson>I guess the director in me doesn't really, it doesn't see it as all negative that there's

00:40:03.760 --> 00:40:05.760
<v Kristinn Þór Sigurbergsson>a bit of a hassle to add a new dependency.

00:40:07.820 --> 00:40:19.900
<v Michael Kennedy>I was thinking about that too, Kristinn. I was. And you can see these headlines of all the supply chain issues and you're like, well, I mean, we should worry about it, but not to the degree that other people with 200 dependencies do, right?

00:40:19.960 --> 00:40:38.000
<v Kristinn Þór Sigurbergsson>Yeah, definitely. And I'm not arguing necessarily for making it hard, but you can see a reality when you have a large development team and people just kind of flippantly add the dependency that causes issue. It's very vulnerable to individual mistakes.

00:40:38.680 --> 00:40:45.560
<v Michael Kennedy>Yeah, and also just duplication of now there's three libraries that kind of handle the same thing, but not entirely.

00:40:45.660 --> 00:40:48.100
<v Kristinn Þór Sigurbergsson>On the flip side, if it's harder to upgrade,

00:40:48.240 --> 00:40:52.380
<v Kristinn Þór Sigurbergsson>you're also less likely to actually upgrade it out of the.

00:40:53.040 --> 00:40:56.480
<v Michael Kennedy>What libraries or tools or experiences

00:40:56.700 --> 00:40:58.280
<v Michael Kennedy>did you all see out there and you're like,

00:40:58.520 --> 00:40:59.920
<v Michael Kennedy>all right, we have to upgrade.

00:41:00.340 --> 00:41:02.020
<v Michael Kennedy>Were there certain packages you're like,

00:41:02.220 --> 00:41:04.340
<v Michael Kennedy>oh gosh, we really, really would benefit

00:41:04.460 --> 00:41:07.320
<v Michael Kennedy>from using this one or having this feature

00:41:08.040 --> 00:41:09.740
<v Michael Kennedy>like the JIT or something in Python

00:41:10.340 --> 00:41:11.500
<v Michael Kennedy>that makes such a difference

00:41:11.740 --> 00:41:12.740
<v Michael Kennedy>that it's worth all this effort.

00:41:13.080 --> 00:41:15.540
<v Thomas Dähling>So, I mean, Jamie, maybe you can talk

00:41:15.580 --> 00:41:16.740
<v Thomas Dähling>about the Python side a little bit,

00:41:16.740 --> 00:41:19.240
<v Thomas Dähling>but from the game engine perspective,

00:41:20.220 --> 00:41:22.880
<v Thomas Dähling>the main driver for really wanting

00:41:22.930 --> 00:41:24.700
<v Thomas Dähling>to get the upgrade project pushed through

00:41:25.060 --> 00:41:30.080
<v Thomas Dähling>was the native backline that we worked on in the early 2020s.

00:41:31.380 --> 00:41:34.360
<v Thomas Dähling>It has just like Python 2.7 was not a way of silicon

00:41:35.280 --> 00:41:35.780
<v Thomas Dähling>to begin with.

00:41:36.880 --> 00:41:38.840
<v Thomas Dähling>Then a lot of other changes that had happened,

00:41:39.470 --> 00:41:42.080
<v Thomas Dähling>like macOS behaves very different with Unicode

00:41:42.220 --> 00:41:44.740
<v Thomas Dähling>than Windows does when it comes to system calls and so on.

00:41:45.860 --> 00:41:47.500
<v Thomas Dähling>Historically, before we had the native MacWords,

00:41:47.680 --> 00:41:51.980
<v Thomas Dähling>clientists would go through a translation layer similar to Wine for Linux.

00:41:52.859 --> 00:41:56.320
<v Thomas Dähling>So we really pushed for it because we just saw that,

00:41:56.440 --> 00:41:58.500
<v Thomas Dähling>okay, if we don't go up to Python 3,

00:41:58.720 --> 00:42:04.000
<v Thomas Dähling>we are stuck with having a small team that is dedicated to nothing else

00:42:04.240 --> 00:42:05.700
<v Thomas Dähling>but maintaining the Python interpreter.

00:42:07.079 --> 00:42:10.400
<v Thomas Dähling>That seemed like it's one possible solution,

00:42:10.860 --> 00:42:14.120
<v Thomas Dähling>but not necessarily the best one if you want to just go

00:42:14.140 --> 00:42:17.100
<v Thomas Dähling>rapidly than to add new features or iterate

00:42:17.400 --> 00:42:18.500
<v Thomas Dähling>on other existing functionality.

00:42:19.400 --> 00:42:23.000
<v Jamie Bannister>I think for me as a gameplay programmer who's mostly

00:42:23.180 --> 00:42:25.420
<v Jamie Bannister>using the Python framework that we have,

00:42:26.420 --> 00:42:28.780
<v Jamie Bannister>things that we've sorely been lacking for a long time

00:42:28.940 --> 00:42:30.920
<v Jamie Bannister>is a good way of type declaration.

00:42:31.940 --> 00:42:36.360
<v Jamie Bannister>It's sort of there in Python 2.7 if you're using the right IDE,

00:42:37.040 --> 00:42:37.740
<v Jamie Bannister>but it's not consistent.

00:42:38.160 --> 00:42:40.400
<v Jamie Bannister>That gets a lot better as you move through the Python 3 versions.

00:42:41.640 --> 00:42:45.660
<v Jamie Bannister>Another thing we end up making a lot of is what's effectively a data class,

00:42:46.220 --> 00:42:49.980
<v Jamie Bannister>where we just want to kind of define a small packet of data in a particular structure

00:42:50.100 --> 00:42:51.360
<v Jamie Bannister>and pass it around and do things.

00:42:52.480 --> 00:42:56.900
<v Jamie Bannister>Having kind of the language support that at a more fundamental level

00:42:57.160 --> 00:42:59.500
<v Jamie Bannister>is kind of something I'm really looking forward to getting.

00:42:59.800 --> 00:43:00.620
<v Michael Kennedy>Yeah, that's really neat.

00:43:00.820 --> 00:43:02.940
<v Michael Kennedy>And think about such a large code base.

00:43:02.980 --> 00:43:05.360
<v Michael Kennedy>Was it 2.4 million lines of code, something like that?

00:43:06.120 --> 00:43:06.720
<v Michael Kennedy>That's non-trivial.

00:43:06.960 --> 00:43:11.080
<v Michael Kennedy>I mean, that's kind of the scenario that typed Python is perfect for

00:43:11.100 --> 00:43:14.200
<v Michael Kennedy>and could have been invented for before AI was a thing,

00:43:14.480 --> 00:43:15.380
<v Michael Kennedy>which also helps.

00:43:15.600 --> 00:43:18.740
<v Michael Kennedy>But having the ability to use tools like Pyrefly or TY

00:43:19.380 --> 00:43:21.660
<v Michael Kennedy>and ask the question in an automated way,

00:43:21.860 --> 00:43:22.880
<v Michael Kennedy>is our code still consistent?

00:43:23.200 --> 00:43:24.000
<v Michael Kennedy>That's pretty neat, right?

00:43:24.920 --> 00:43:26.960
<v Jamie Bannister>Yeah, I think given the size of our code base

00:43:27.480 --> 00:43:31.820
<v Jamie Bannister>and the fact that we have reasonable test coverage,

00:43:32.240 --> 00:43:33.880
<v Jamie Bannister>but we don't have complete by any means.

00:43:34.920 --> 00:43:37.800
<v Jamie Bannister>So the more we can put in a kind of leverage from the language

00:43:38.020 --> 00:43:40.200
<v Jamie Bannister>to help prove correctness and find issues,

00:43:41.140 --> 00:43:43.880
<v Jamie Bannister>that's going to lead to a better product and less issues.

00:43:44.220 --> 00:43:44.340
<v Michael Kennedy>Yeah.

00:43:44.660 --> 00:43:49.340
<v Michael Kennedy>Last question before we actually get into the details of the migration and how you all did it.

00:43:49.560 --> 00:43:50.820
<v Michael Kennedy>What about free-threaded Python?

00:43:51.240 --> 00:43:54.760
<v Michael Kennedy>You started with this concept of high concurrence,

00:43:55.100 --> 00:43:59.280
<v Michael Kennedy>so much so we need concurrency that we're going to get our own runtime and maintain it.

00:43:59.540 --> 00:44:01.100
<v Michael Kennedy>Are you considering free-threaded Python?

00:44:01.270 --> 00:44:02.800
<v Michael Kennedy>Is this interesting in any way?

00:44:03.420 --> 00:44:05.340
<v Michael Kennedy>Or is it all asyncio driven?

00:44:06.420 --> 00:44:07.860
<v Michael Kennedy>What are you considering here?

00:44:08.220 --> 00:44:09.200
<v Thomas Dähling>We are looking at it.

00:44:09.540 --> 00:44:14.960
<v Thomas Dähling>I've actually just been to Europe iPhone back in July and attended some of the very

00:44:15.160 --> 00:44:17.240
<v Thomas Dähling>interesting presentations on where 3.30 is heading.

00:44:19.120 --> 00:44:21.420
<v Thomas Dähling>There is a lot of work ahead of us to adopt it,

00:44:22.420 --> 00:44:26.320
<v Thomas Dähling>because there are many places which just implicitly rely on the GIL for synchronization.

00:44:27.160 --> 00:44:28.560
<v Thomas Dähling>We will need to identify those.

00:44:29.500 --> 00:44:33.740
<v Thomas Dähling>There are some interesting parts where I think we could benefit from it

00:44:34.170 --> 00:44:36.580
<v Thomas Dähling>when it comes to our resource loading system, for example.

00:44:38.000 --> 00:44:40.260
<v Thomas Dähling>But yeah, it's on our radar.

00:44:40.400 --> 00:44:40.860
<v Thomas Dähling>We are interested.

00:44:41.140 --> 00:44:45.080
<v Thomas Dähling>It's not necessarily going to be the biggest performance improvement

00:44:46.130 --> 00:44:47.740
<v Thomas Dähling>that we could possibly get, I think.

00:44:49.340 --> 00:44:52.300
<v Thomas Dähling>But once it's there, we also have more possibility

00:44:52.720 --> 00:44:54.000
<v Thomas Dähling>or more options to explore as well

00:44:54.320 --> 00:44:55.720
<v Thomas Dähling>and what we might be able to do different.

00:44:56.000 --> 00:44:56.080
<v Michael Kennedy>Yeah.

00:44:56.470 --> 00:45:00.600
<v Michael Kennedy>I imagine that AsyncIO, given how much network, Redis, etc.,

00:45:00.690 --> 00:45:03.540
<v Michael Kennedy>stuff you do, will give you a pretty mega boost

00:45:03.700 --> 00:45:06.420
<v Michael Kennedy>as well as just the performance of CPython

00:45:06.440 --> 00:45:08.000
<v Michael Kennedy>has gotten so much better, right?

00:45:08.180 --> 00:45:10.000
<v Thomas Dähling>Yeah, we've gotten, like, what was it,

00:45:10.040 --> 00:45:14.240
<v Thomas Dähling>15% to 20% improvement across the board for most things.

00:45:14.660 --> 00:45:15.900
<v Thomas Dähling>Like, there's the one-

00:45:16.170 --> 00:45:18.260
<v Thomas Dähling>like, string handling obviously changed.

00:45:18.540 --> 00:45:20.100
<v Thomas Dähling>You can't really compare that with Python 2,

00:45:20.840 --> 00:45:21.840
<v Thomas Dähling>because now it's all Unicode.

00:45:22.120 --> 00:45:23.700
<v Thomas Dähling>That always bears a little bit of overhead

00:45:24.080 --> 00:45:26.140
<v Thomas Dähling>compared to operating on bytes directly.

00:45:27.360 --> 00:45:31.520
<v Thomas Dähling>But so far, we have been very happy with the performance

00:45:31.660 --> 00:45:33.420
<v Thomas Dähling>improvements that have gotten-- that we basically

00:45:33.510 --> 00:45:34.340
<v Thomas Dähling>gotten for free, right?

00:45:35.840 --> 00:45:37.880
<v Thomas Dähling>On the topic of async I/O, though, unfortunately,

00:45:38.050 --> 00:45:40.540
<v Thomas Dähling>we can't really benefit from that because you

00:45:40.570 --> 00:45:44.220
<v Thomas Dähling>need to annotate your functions with async and so on.

00:45:45.280 --> 00:45:48.040
<v Thomas Dähling>And with the 2.4 million lines of Python code

00:45:48.210 --> 00:45:51.720
<v Thomas Dähling>written without that in mind, it had become quite quickly

00:45:51.810 --> 00:45:54.920
<v Thomas Dähling>quite clear that, unfortunately, that one

00:45:55.000 --> 00:45:56.080
<v Thomas Dähling>is out of the window for us.

00:45:56.360 --> 00:45:56.880
<v Thomas Dähling>Interesting.

00:45:58.200 --> 00:46:03.780
<v Kristinn Þór Sigurbergsson>We did discuss it at the time, whether it was just

00:46:03.800 --> 00:46:07.060
<v Kristinn Þór Sigurbergsson>completely out of scope of everything that we were doing.

00:46:07.660 --> 00:46:09.940
<v Michael Kennedy>You know, one thing in terms of colored functions,

00:46:10.120 --> 00:46:13.900
<v Michael Kennedy>that is one thing I just talked to Joe from the Gradient Project

00:46:14.360 --> 00:46:18.940
<v Michael Kennedy>about Tone.io, which is a multi-threaded async runtime in Python

00:46:19.480 --> 00:46:21.680
<v Michael Kennedy>written in Rust, but implemented in Rust.

00:46:22.120 --> 00:46:26.600
<v Michael Kennedy>But it has this way of running without necessarily decorating the thread.

00:46:26.780 --> 00:46:28.660
<v Michael Kennedy>Anyway, maybe, I don't know, maybe it's interesting.

00:46:28.740 --> 00:46:29.080
<v Michael Kennedy>Who knows?

00:46:29.700 --> 00:46:31.420
<v Thomas Dähling>I attended his talk at EuroPython.

00:46:31.980 --> 00:46:32.400
<v Thomas Dähling>Oh, yeah, yeah.

00:46:32.740 --> 00:46:35.380
<v Thomas Dähling>And it looked interesting.

00:46:36.580 --> 00:46:39.440
<v Thomas Dähling>But on the other hand, we are doing a lot of similar things

00:46:39.580 --> 00:46:40.320
<v Thomas Dähling>underneath the hood already.

00:46:40.970 --> 00:46:42.900
<v Thomas Dähling>So basically, all of the I/O already

00:46:43.150 --> 00:46:45.460
<v Thomas Dähling>happens on multiple worker threads in the background.

00:46:45.920 --> 00:46:46.340
<v Thomas Dähling>Interesting.

00:46:46.740 --> 00:46:46.800
<v Thomas Dähling>Yeah.

00:46:47.120 --> 00:46:48.500
<v Thomas Dähling>Really just that we have-

00:46:48.760 --> 00:46:50.220
<v Thomas Dähling>I mean, the main reason why-

00:46:50.670 --> 00:46:53.060
<v Thomas Dähling>the main way that we facilitated Stackless back then

00:46:53.260 --> 00:46:54.260
<v Thomas Dähling>at the new schedule and what you'll know

00:46:54.540 --> 00:46:56.640
<v Thomas Dähling>is that it acts as a synchronization point

00:46:56.920 --> 00:46:58.900
<v Thomas Dähling>to get the data from those worker threads

00:46:59.150 --> 00:47:00.560
<v Thomas Dähling>into the main thread that's running Python.

00:47:00.840 --> 00:47:06.340
<v Thomas Dähling>And I've looked at Toneio a little bit, but even that is not very straightforward to integrate

00:47:06.390 --> 00:47:09.940
<v Thomas Dähling>in our stack, unfortunately. So doing comparisons is a bit difficult.

00:47:10.260 --> 00:47:16.080
<v Michael Kennedy>Yeah, honestly, and the free-threaded stuff, honestly, it scares me. I used to do tons of

00:47:16.900 --> 00:47:22.320
<v Michael Kennedy>concurrent programming, threads, events, signal, locks, all that kind of stuff. It's not my code

00:47:22.400 --> 00:47:27.080
<v Michael Kennedy>that scares me. It's all the libraries that are written in Python that, like you all said,

00:47:27.280 --> 00:47:32.940
<v Michael Kennedy>a lot of people assume the GIL was a thing and how many of those have some little latent race

00:47:33.160 --> 00:47:36.800
<v Michael Kennedy>condition that when they figure out what it is they're gonna put locks on it then it'll become

00:47:36.820 --> 00:47:41.620
<v Michael Kennedy>a latent lock a deadlock you know what i mean like just having faith that all the foundational

00:47:41.760 --> 00:47:46.480
<v Michael Kennedy>stuff has really been through the ringer of pre-threading i don't know scares me i'm optimistic

00:47:46.620 --> 00:47:50.820
<v Thomas Dähling>for it and i want it to exist but it scares me yeah there's a lot of work for sure you know also

00:47:50.940 --> 00:47:54.780
<v Thomas Dähling>like the other side that the single threaded use case shall not degrade in performance either

00:47:54.800 --> 00:47:55.240
<v Thomas Dähling>you know

00:47:55.420 --> 00:47:55.520
<v Thomas Dähling>it's

00:47:56.600 --> 00:47:57.240
<v Thomas Dähling>there are many things

00:47:57.840 --> 00:47:58.540
<v Thomas Dähling>but that being said

00:47:58.720 --> 00:47:58.840
<v Thomas Dähling>like

00:47:59.230 --> 00:47:59.600
<v Thomas Dähling>even with the

00:47:59.820 --> 00:48:00.460
<v Thomas Dähling>asynchronous programming

00:48:00.620 --> 00:48:00.880
<v Thomas Dähling>that we

00:48:01.080 --> 00:48:01.480
<v Thomas Dähling>that we use

00:48:01.620 --> 00:48:02.000
<v Thomas Dähling>underneath the hood

00:48:02.080 --> 00:48:02.720
<v Thomas Dähling>we are already

00:48:02.750 --> 00:48:03.200
<v Thomas Dähling>in the world

00:48:03.380 --> 00:48:04.760
<v Thomas Dähling>of having to be careful

00:48:04.940 --> 00:48:05.820
<v Thomas Dähling>what we put a lock on

00:48:05.900 --> 00:48:06.140
<v Thomas Dähling>and whatnot

00:48:06.500 --> 00:48:06.600
<v Thomas Dähling>right

00:48:07.340 --> 00:48:08.100
<v Michael Kennedy>for you guys

00:48:08.400 --> 00:48:10.180
<v Michael Kennedy>you all are in this world

00:48:10.340 --> 00:48:10.740
<v Michael Kennedy>but I think

00:48:11.380 --> 00:48:13.080
<v Michael Kennedy>99% of the Python people

00:48:13.240 --> 00:48:14.360
<v Michael Kennedy>don't even think about locks

00:48:14.760 --> 00:48:14.900
<v Michael Kennedy>and

00:48:15.670 --> 00:48:16.140
<v Michael Kennedy>race conditions

00:48:16.480 --> 00:48:16.660
<v Michael Kennedy>you know

00:48:16.660 --> 00:48:17.040
<v Michael Kennedy>and that's

00:48:17.160 --> 00:48:18.260
<v Michael Kennedy>I think that's where

00:48:18.340 --> 00:48:18.700
<v Michael Kennedy>we're going to

00:48:19.160 --> 00:48:19.440
<v Michael Kennedy>have to

00:48:19.780 --> 00:48:20.520
<v Michael Kennedy>as a community

00:48:20.920 --> 00:48:21.880
<v Michael Kennedy>work through the issues

00:48:22.340 --> 00:48:23.200
<v Michael Kennedy>to make these

00:48:23.600 --> 00:48:23.920
<v Michael Kennedy>dependencies

00:48:24.540 --> 00:48:24.760
<v Michael Kennedy>stable

00:48:24.840 --> 00:48:25.820
<v Thomas Dähling>Yeah, absolutely.

00:48:26.020 --> 00:48:28.620
<v Thomas Dähling>There is actually-- I think there is a community effort going on,

00:48:29.140 --> 00:48:30.900
<v Thomas Dähling>as far as I remember, where-

00:48:32.400 --> 00:48:34.900
<v Thomas Dähling>is the URL really rb-free-threaded yet or something?

00:48:35.800 --> 00:48:37.220
<v Thomas Dähling>So there's a project where they basically

00:48:37.480 --> 00:48:40.220
<v Thomas Dähling>track all of the public libraries in PyPy

00:48:40.480 --> 00:48:43.000
<v Thomas Dähling>and how they behave under free-threaded Python already.

00:48:43.340 --> 00:48:45.880
<v Thomas Dähling>So we also-- well, I keep an eye on that as well

00:48:46.000 --> 00:48:49.160
<v Thomas Dähling>to see where that is happening and where there may be some

00:48:49.160 --> 00:48:50.260
<v Thomas Dähling>of our dependencies in there.

00:48:50.500 --> 00:48:52.960
<v Michael Kennedy>There's the compatibility, the free-threading compatibility

00:48:53.460 --> 00:48:53.920
<v Michael Kennedy>status checker.

00:48:54.320 --> 00:48:57.580
<v Michael Kennedy>It's got the tested in CI, the PyPI release,

00:48:58.020 --> 00:49:00.440
<v Michael Kennedy>when it was first supported, the free threading bit.

00:49:00.660 --> 00:49:02.700
<v Michael Kennedy>It's very data science heavy though, isn't it?

00:49:03.040 --> 00:49:04.660
<v Thomas Dähling>Yeah, I guess it's once again the data scientists

00:49:04.880 --> 00:49:05.840
<v Thomas Dähling>that are leading the way here.

00:49:06.120 --> 00:49:08.360
<v Thomas Dähling>I guess they're also mostly benefiting from a free thread in Python,

00:49:08.980 --> 00:49:10.500
<v Thomas Dähling>given that they are doing a lot of number crunching.

00:49:11.120 --> 00:49:11.500
<v Michael Kennedy>Right, exactly.

00:49:12.000 --> 00:49:14.760
<v Michael Kennedy>Because I think async.io, even though it may not apply to you all,

00:49:15.140 --> 00:49:17.800
<v Michael Kennedy>does solve the problem of I got to talk to the database

00:49:17.900 --> 00:49:22.900
<v Michael Kennedy>a whole bunch of times concurrently or to microservices concurrently

00:49:23.440 --> 00:49:24.880
<v Michael Kennedy>because it's the IO bit,

00:49:25.080 --> 00:49:26.020
<v Michael Kennedy>but when you got to do computation,

00:49:26.980 --> 00:49:27.940
<v Michael Kennedy>it doesn't help you a bit.

00:49:28.120 --> 00:49:28.960
<v Michael Kennedy>So yeah, you're right.

00:49:29.040 --> 00:49:30.440
<v Michael Kennedy>I think that's fundamentally why.

00:49:30.740 --> 00:49:33.560
<v Michael Kennedy>All right, let's talk the actual migration.

00:49:34.020 --> 00:49:36.680
<v Michael Kennedy>So you all started by just making sure,

00:49:36.780 --> 00:49:39.000
<v Michael Kennedy>well, first converting everything to run

00:49:39.240 --> 00:49:40.920
<v Michael Kennedy>in two and three, right?

00:49:41.210 --> 00:49:41.840
<v Michael Kennedy>Tell us about that.

00:49:43.560 --> 00:49:43.700
<v Jamie Bannister>Yeah.

00:49:45.240 --> 00:49:46.080
<v Jamie Bannister>All right, so I'm going to take this.

00:49:47.959 --> 00:49:52.020
<v Jamie Bannister>So a lot of the work that kind of came our way

00:49:52.040 --> 00:49:56.500
<v Jamie Bannister>was to take our 2.4 million lines of code that we have

00:49:57.380 --> 00:50:00.000
<v Jamie Bannister>and bring it forward so that we can eventually end up

00:50:00.380 --> 00:50:01.100
<v Jamie Bannister>everything on Python 3.

00:50:01.920 --> 00:50:04.860
<v Jamie Bannister>What we found is, like we mentioned earlier on,

00:50:04.960 --> 00:50:09.100
<v Jamie Bannister>we still have a lot of code that was technically 2.4, 2.5,

00:50:10.060 --> 00:50:12.060
<v Jamie Bannister>or using 2.4 and 2.5 kind of idioms,

00:50:13.220 --> 00:50:15.280
<v Jamie Bannister>which are no longer suitable in Python 3.

00:50:15.780 --> 00:50:17.840
<v Jamie Bannister>So the very first step that we've taken

00:50:18.160 --> 00:50:20.200
<v Jamie Bannister>is what we're calling legacy to 2.7,

00:50:20.420 --> 00:50:24.560
<v Jamie Bannister>which is get all of those legacy bits of code up to 2.7 standard,

00:50:25.760 --> 00:50:29.460
<v Jamie Bannister>because a lot of those changes then become more forwards compatible into 3.

00:50:30.560 --> 00:50:35.900
<v Jamie Bannister>We've actually already released and deployed the first part of that a few weeks ago.

00:50:36.100 --> 00:50:38.040
<v Jamie Bannister>That was around the time that we wrote that blog.

00:50:38.500 --> 00:50:41.800
<v Jamie Bannister>And for a lot of that, we use these Futurize scripts,

00:50:43.160 --> 00:50:44.660
<v Jamie Bannister>all the fixes from Futurize.

00:50:44.980 --> 00:50:48.480
<v Jamie Bannister>And what they do is they take one particular idiom of Python

00:50:49.380 --> 00:50:51.280
<v Jamie Bannister>and update it to Python 3.

00:50:51.660 --> 00:50:54.200
<v Jamie Bannister>There's a subset of those which you can run

00:50:54.590 --> 00:50:57.400
<v Jamie Bannister>whilst still being 2.7 compatible.

00:50:58.120 --> 00:50:59.880
<v Jamie Bannister>And so those are the ones that we've done so far.

00:51:00.120 --> 00:51:04.200
<v Michael Kennedy>Yeah, and you just basically got them all moved

00:51:04.800 --> 00:51:07.180
<v Michael Kennedy>to a way that theoretically would run on Python 3

00:51:07.400 --> 00:51:08.400
<v Michael Kennedy>because for people who don't know,

00:51:08.740 --> 00:51:10.720
<v Michael Kennedy>some of the older ways of working in Python 2

00:51:10.880 --> 00:51:12.460
<v Michael Kennedy>were literally incompatible,

00:51:12.780 --> 00:51:14.820
<v Michael Kennedy>like ways of declaring exceptions,

00:51:15.900 --> 00:51:17.660
<v Michael Kennedy>handling some of the string stuff.

00:51:19.059 --> 00:51:23.840
<v Jamie Bannister>Yeah, the big one we found a lot of was testing membership in a dictionary.

00:51:24.380 --> 00:51:26.580
<v Jamie Bannister>So now you do if X in Y.

00:51:27.230 --> 00:51:31.460
<v Jamie Bannister>In the older Pythons, you would do if there was a has key function.

00:51:32.600 --> 00:51:35.320
<v Jamie Bannister>That is no longer supported in Pyth 3.

00:51:35.620 --> 00:51:38.760
<v Jamie Bannister>But we still have a lot of code, or we did have a lot of code that was using has key.

00:51:39.050 --> 00:51:43.360
<v Jamie Bannister>So we had to change all of those to using the in state in operator.

00:51:43.940 --> 00:51:47.140
<v Jamie Bannister>So that's the kind of thing that we've kind of done in this first phase.

00:51:48.320 --> 00:51:57.920
<v Jamie Bannister>What we found was we were actually 94% of our code was technically Python 3 syntactically valid.

00:51:58.620 --> 00:52:01.360
<v Jamie Bannister>We've now got up to 99.8%.

00:52:02.340 --> 00:52:03.860
<v Jamie Bannister>So that's syntactically valid.

00:52:03.880 --> 00:52:07.080
<v Jamie Bannister>It doesn't mean it's going to do the right thing when you put it in Python.

00:52:07.440 --> 00:52:08.880
<v Jamie Bannister>The classic example is division.

00:52:09.980 --> 00:52:12.280
<v Jamie Bannister>Being a game that does a lot of things with a lot of numbers,

00:52:13.040 --> 00:52:16.960
<v Jamie Bannister>we have about 6,500 lines that feature a division.

00:52:17.520 --> 00:52:22.080
<v Jamie Bannister>some of those are going to break if we just switch to Python 3 without doing anything with them

00:52:23.680 --> 00:52:27.460
<v Jamie Bannister>because the way it changes from integer to float division.

00:52:28.240 --> 00:52:31.020
<v Jamie Bannister>So that's going to be part of the next phase that we're going to get into.

00:52:31.280 --> 00:52:33.360
<v Michael Kennedy>Yeah, that was one of the main deals.

00:52:33.530 --> 00:52:40.020
<v Michael Kennedy>If you have x slash y in Python 2, that would do basically integer division, truncate it.

00:52:40.360 --> 00:52:42.040
<v Michael Kennedy>They take the floor of it more or less, right?

00:52:42.270 --> 00:52:46.660
<v Michael Kennedy>And then those just become floating point numbers, which that's terrible for like this thing equal,

00:52:46.820 --> 00:52:48.100
<v Michael Kennedy>equal that thing and all, it's-

00:52:48.360 --> 00:52:50.880
<v Michael Kennedy>And if you're doing a lot of damage calculations,

00:52:51.240 --> 00:52:52.940
<v Jamie Bannister>you've got one ship shooting another ship,

00:52:54.000 --> 00:52:54.820
<v Jamie Bannister>that can make a lot of difference

00:52:55.100 --> 00:52:56.760
<v Jamie Bannister>if suddenly your damage changes

00:52:57.100 --> 00:52:58.720
<v Jamie Bannister>and now I should have won this fight

00:52:58.780 --> 00:53:00.160
<v Jamie Bannister>and now I've lost it because the maths

00:53:00.240 --> 00:53:01.080
<v Jamie Bannister>is working out differently.

00:53:01.380 --> 00:53:04.760
<v Jamie Bannister>So that's gonna be important for us to get right.

00:53:04.980 --> 00:53:05.540
<v Michael Kennedy>Yeah, absolutely.

00:53:05.980 --> 00:53:08.660
<v Michael Kennedy>Okay, so what next, what happened after that?

00:53:08.760 --> 00:53:12.180
<v Michael Kennedy>You got it working on Python 2, but syntactically Python 3?

00:53:12.560 --> 00:53:13.780
<v Jamie Bannister>Yeah, okay, so the next phase,

00:53:13.980 --> 00:53:15.240
<v Jamie Bannister>which we're now in at the moment,

00:53:17.360 --> 00:53:23.460
<v Jamie Bannister>is we've forked the Python 2 branch into a Python 3 branch.

00:53:24.160 --> 00:53:26.160
<v Jamie Bannister>Immediately, many, many things failed.

00:53:26.440 --> 00:53:29.840
<v Jamie Bannister>All of our tests started showing up red and so on.

00:53:30.700 --> 00:53:33.120
<v Jamie Bannister>And so what we're calling that is our Python 3 unstable.

00:53:33.640 --> 00:53:35.480
<v Jamie Bannister>And so from there, we need to work through,

00:53:36.720 --> 00:53:39.860
<v Jamie Bannister>get all of our tests passing, start getting that more stable.

00:53:40.940 --> 00:53:45.120
<v Jamie Bannister>At the same time, we've got what we're calling a 2.7 unstable branch,

00:53:45.560 --> 00:53:48.980
<v Jamie Bannister>where we're kind of introducing a lot of the shims and the adapters.

00:53:49.400 --> 00:53:54.680
<v Jamie Bannister>So this is where we can have code that's going to maybe fork if 2 do this, if 3 do this.

00:53:55.300 --> 00:53:57.200
<v Jamie Bannister>The downside of that is it's going to be a lot slower.

00:53:57.620 --> 00:54:02.040
<v Jamie Bannister>So that's more of a compatibility proof versus a releasable version.

00:54:02.780 --> 00:54:06.320
<v Jamie Bannister>But that is going to be our middle ground for code that we can then say,

00:54:06.980 --> 00:54:09.660
<v Jamie Bannister>we can put this into 2.7 and make it 4.0 compatible,

00:54:10.280 --> 00:54:11.680
<v Jamie Bannister>or this is going to be breaking compatible,

00:54:11.850 --> 00:54:13.640
<v Jamie Bannister>so we need to put it on the three side.

00:54:14.500 --> 00:54:17.260
<v Jamie Bannister>And yeah, that's our next however many months of work.

00:54:18.280 --> 00:54:20.600
<v Jamie Bannister>But the other big angle of it,

00:54:20.780 --> 00:54:22.460
<v Jamie Bannister>which is really worth calling out,

00:54:22.680 --> 00:54:24.720
<v Jamie Bannister>is it's not just the runtime code.

00:54:24.790 --> 00:54:29.080
<v Jamie Bannister>We have a lot of data that is either data at rest

00:54:29.260 --> 00:54:32.140
<v Jamie Bannister>or data going across a network that is serialized Python.

00:54:33.140 --> 00:54:34.560
<v Jamie Bannister>We need to get all of that compatible.

00:54:34.810 --> 00:54:39.540
<v Jamie Bannister>So if we have, for example, Python 2 Alt-Star classes

00:54:39.560 --> 00:54:43.740
<v Jamie Bannister>and we want to serialize that to send it to your Python 3 node.

00:54:43.970 --> 00:54:46.120
<v Jamie Bannister>Well, that's not going to work because Python 3 is going to,

00:54:46.440 --> 00:54:46.700
<v Jamie Bannister>what's this?

00:54:46.770 --> 00:54:47.640
<v Jamie Bannister>I don't know what to do with it.

00:54:47.840 --> 00:54:49.640
<v Jamie Bannister>So as well as changing runtime code,

00:54:49.740 --> 00:54:53.360
<v Jamie Bannister>there's a lot of data at rest and data on the network that we also need to change.

00:54:53.580 --> 00:54:53.760
<v Michael Kennedy>Right.

00:54:53.930 --> 00:54:55.620
<v Michael Kennedy>You did mention that you're using Redis,

00:54:55.880 --> 00:54:59.360
<v Michael Kennedy>and one of the real common things to do is just to pickle the objects

00:54:59.720 --> 00:55:00.940
<v Michael Kennedy>and put them into Redis.

00:55:01.340 --> 00:55:01.400
<v Michael Kennedy>Yeah.

00:55:01.460 --> 00:55:02.260
<v Michael Kennedy>But you know what?

00:55:02.400 --> 00:55:05.700
<v Michael Kennedy>Memory shape changes from version to version.

00:55:05.940 --> 00:55:08.480
<v Jamie Bannister>I've literally been writing a blog that's going to come out

00:55:08.500 --> 00:55:10.300
<v Jamie Bannister>in the next week or two, a public blog,

00:55:11.280 --> 00:55:13.400
<v Jamie Bannister>talking about exactly that problem that we have here.

00:55:14.280 --> 00:55:16.020
<v Kristinn Þór Sigurbergsson>Is that the agent blog?

00:55:17.060 --> 00:55:17.360
<v Jamie Bannister>Yes.

00:55:17.630 --> 00:55:21.460
<v Jamie Bannister>So in EVE, we have a bunch of NPCs

00:55:23.110 --> 00:55:25.640
<v Jamie Bannister>that we call agents, nothing to do with LLM agents.

00:55:25.940 --> 00:55:28.440
<v Jamie Bannister>These are just characters and they have memory.

00:55:28.650 --> 00:55:29.760
<v Jamie Bannister>So when you talk to them,

00:55:30.640 --> 00:55:32.040
<v Jamie Bannister>it's going to remember that you talked to them.

00:55:32.360 --> 00:55:33.600
<v Jamie Bannister>And then later on, you're going to come back

00:55:33.930 --> 00:55:35.520
<v Jamie Bannister>and they'll give you a mission or to go,

00:55:35.780 --> 00:55:37.320
<v Jamie Bannister>go and fetch me this or go and kill this.

00:55:38.400 --> 00:55:40.380
<v Jamie Bannister>it saves that bit of memory in the database.

00:55:40.980 --> 00:55:43.800
<v Jamie Bannister>The way it saves it is as a serialized Python block,

00:55:44.100 --> 00:55:48.080
<v Jamie Bannister>so that when it comes back, we can kind of reload the memory

00:55:48.340 --> 00:55:50.280
<v Jamie Bannister>or reload that bit of memory from the disk.

00:55:51.420 --> 00:55:53.480
<v Jamie Bannister>We have those going back now for 20 years.

00:55:54.020 --> 00:55:57.060
<v Jamie Bannister>I think there's about 100 gigabytes of Python memory in the DB,

00:55:57.460 --> 00:56:01.240
<v Jamie Bannister>of agent memory in the DB in the form of pickled Python objects.

00:56:02.180 --> 00:56:05.720
<v Jamie Bannister>So that's a big part we need to go through.

00:56:07.180 --> 00:56:08.800
<v Jamie Bannister>Because we can't just write a database script

00:56:09.000 --> 00:56:09.940
<v Jamie Bannister>that's going to do those, right?

00:56:10.020 --> 00:56:12.180
<v Jamie Bannister>Because a SQL script isn't going to be able to

00:56:13.340 --> 00:56:14.740
<v Jamie Bannister>serialize and deserialize Python objects.

00:56:15.040 --> 00:56:16.140
<v Michael Kennedy>It has no idea, yeah.

00:56:16.460 --> 00:56:16.520
<v Michael Kennedy>Yeah.

00:56:16.900 --> 00:56:19.460
<v Michael Kennedy>Are you considering moving to something

00:56:19.800 --> 00:56:22.280
<v Michael Kennedy>that's version independent, like msgspec,

00:56:23.180 --> 00:56:24.900
<v Michael Kennedy>or Pydantic, or JSON?

00:56:26.340 --> 00:56:27.540
<v Jamie Bannister>Some of these things, what we want to do

00:56:27.660 --> 00:56:31.320
<v Jamie Bannister>is change the format to probably protobuf,

00:56:31.500 --> 00:56:33.420
<v Jamie Bannister>because we use that for a lot of our

00:56:34.940 --> 00:56:36.980
<v Jamie Bannister>kind of communications and storage as well,

00:56:37.860 --> 00:56:39.020
<v Jamie Bannister>either protobuf or flat buffer,

00:56:39.260 --> 00:56:40.120
<v Jamie Bannister>depending on the size.

00:56:41.120 --> 00:56:44.800
<v Jamie Bannister>But yeah, a big part of our internal and external APIs

00:56:45.800 --> 00:56:47.120
<v Jamie Bannister>are all built on top of protobuf.

00:56:47.230 --> 00:56:48.140
<v Jamie Bannister>They would probably use that.

00:56:48.420 --> 00:56:49.400
<v Michael Kennedy>Yeah, so that'd be really natural.

00:56:49.560 --> 00:56:51.520
<v Michael Kennedy>Maybe you could even just not even deserialize it,

00:56:51.620 --> 00:56:54.140
<v Michael Kennedy>just give me the stuff and shoot it back over the network

00:56:54.310 --> 00:56:55.820
<v Michael Kennedy>if it wants in that format anyway,

00:56:56.240 --> 00:56:56.840
<v Michael Kennedy>digitally, I guess.

00:56:57.180 --> 00:56:58.460
<v Michael Kennedy>All right, we'll get really short on time.

00:56:58.600 --> 00:57:01.460
<v Michael Kennedy>Maybe just take us through the remaining arc

00:57:01.720 --> 00:57:03.200
<v Michael Kennedy>of what you all planned out,

00:57:04.100 --> 00:57:05.920
<v Michael Kennedy>What's still in the story to be told?

00:57:07.560 --> 00:57:11.940
<v Jamie Bannister>So, yeah, so we've got the data formats that we need to change.

00:57:12.160 --> 00:57:13.440
<v Jamie Bannister>We've got the runtime code.

00:57:14.360 --> 00:57:15.460
<v Jamie Bannister>Once we've got those worked out,

00:57:16.240 --> 00:57:19.060
<v Jamie Bannister>some of the things we want to start doing are what we call a mix mode,

00:57:19.320 --> 00:57:23.980
<v Jamie Bannister>which is where we take our cluster of 200 nodes plus all of our clients,

00:57:24.560 --> 00:57:26.660
<v Jamie Bannister>all of which are 2.7 at the moment,

00:57:27.040 --> 00:57:31.220
<v Jamie Bannister>and start to be able to selectively replace one piece of those

00:57:31.660 --> 00:57:32.720
<v Jamie Bannister>with the Python 3 equivalent.

00:57:33.080 --> 00:57:36.560
<v Jamie Bannister>So rather than doing a big deployment day where we say,

00:57:36.780 --> 00:57:39.480
<v Jamie Bannister>okay, this is the moment, we want to incrementally do it,

00:57:40.000 --> 00:57:41.940
<v Jamie Bannister>kind of spread that risk and that load at the time.

00:57:42.580 --> 00:57:44.780
<v Michael Kennedy>Can you imagine the stress of just

00:57:45.120 --> 00:57:45.580
<v Jamie Bannister>Oh yeah.

00:57:45.860 --> 00:57:46.740
<v Jamie Bannister>Pushing the whole thing.

00:57:46.920 --> 00:57:48.520
<v Jamie Bannister>I've been there for those big deployments,

00:57:50.120 --> 00:57:50.840
<v Jamie Bannister>very stressful.

00:57:51.720 --> 00:57:53.940
<v Jamie Bannister>So by doing this, we can spread that risk

00:57:53.980 --> 00:57:55.100
<v Jamie Bannister>and that load over a lot of time,

00:57:55.400 --> 00:57:57.920
<v Jamie Bannister>but it also means we're developing a lot of tools

00:57:58.280 --> 00:58:01.840
<v Jamie Bannister>to support this project that aren't necessarily

00:58:01.860 --> 00:58:05.820
<v Jamie Bannister>Absolutely required, but they're going to give us a lot better abilities.

00:58:06.100 --> 00:58:14.720
<v Jamie Bannister>So like this ability to deploy a kind of a heterogeneous mix of types of builds and even mixed.

00:58:15.080 --> 00:58:18.620
<v Jamie Bannister>Do we want to do some nodes on Linux and some on Windows, for example?

00:58:19.500 --> 00:58:21.540
<v Jamie Bannister>By doing these tools, that's going to give us a lot of these options.

00:58:21.960 --> 00:58:24.660
<v Michael Kennedy>Yeah, give you more deployment options and so on, right?

00:58:24.860 --> 00:58:25.520
<v Michael Kennedy>That's pretty interesting.

00:58:25.940 --> 00:58:26.740
<v Michael Kennedy>It's an interesting consequence.

00:58:26.920 --> 00:58:29.160
<v Michael Kennedy>I didn't really consider coming along, but yeah, it makes sense.

00:58:29.480 --> 00:58:29.680
<v Michael Kennedy>All right.

00:58:29.900 --> 00:58:31.460
<v Michael Kennedy>Kristinn, what else do you want to say about this?

00:58:31.540 --> 00:58:38.520
<v Kristinn Þór Sigurbergsson>migration from maybe from the director high level yeah it's i mean i think it helps a lot that we

00:58:38.520 --> 00:58:42.880
<v Kristinn Þór Sigurbergsson>did this for frontier first because one of the problems we had when we started this is that like

00:58:42.880 --> 00:58:47.220
<v Kristinn Þór Sigurbergsson>we could start migrating the python code but none of the python code would really work without the

00:58:47.300 --> 00:58:55.100
<v Kristinn Þór Sigurbergsson>C++ modules being um importable to python as well and we could start in the python C++ modules but

00:58:55.320 --> 00:58:59.780
<v Kristinn Þór Sigurbergsson>like they wouldn't really make sense without the python glue kind of tying it together so

00:59:00.340 --> 00:59:02.940
<v Kristinn Þór Sigurbergsson>it was really convenient to do this on a project

00:59:03.420 --> 00:59:04.840
<v Kristinn Þór Sigurbergsson>like Frontier where the

00:59:05.050 --> 00:59:06.780
<v Kristinn Þór Sigurbergsson>risk of breaking

00:59:07.040 --> 00:59:09.000
<v Kristinn Þór Sigurbergsson>things is a

00:59:09.140 --> 00:59:10.620
<v Kristinn Þór Sigurbergsson>lot less and like the

00:59:10.820 --> 00:59:12.760
<v Kristinn Þór Sigurbergsson>first release we did we had hourly

00:59:13.060 --> 00:59:14.400
<v Kristinn Þór Sigurbergsson>downtime because after like

00:59:14.880 --> 00:59:16.020
<v Kristinn Þór Sigurbergsson>one and a half, two hours

00:59:16.600 --> 00:59:18.400
<v Kristinn Þór Sigurbergsson>the process would just

00:59:18.660 --> 00:59:20.420
<v Kristinn Þór Sigurbergsson>some of the process, some process somewhere

00:59:20.620 --> 00:59:22.740
<v Kristinn Þór Sigurbergsson>would just stop responding at all and we were

00:59:22.880 --> 00:59:24.560
<v Kristinn Þór Sigurbergsson>like a bit in the dark there

00:59:24.590 --> 00:59:25.260
<v Kristinn Þór Sigurbergsson>it was like a

00:59:27.100 --> 00:59:28.480
<v Kristinn Þór Sigurbergsson>very stressful time

00:59:28.500 --> 00:59:35.580
<v Kristinn Þór Sigurbergsson>and it turned out to be something really really stupid so as it obviously would have but so i think

00:59:36.540 --> 00:59:42.740
<v Kristinn Þór Sigurbergsson>i think the team city or the build system helped us a lot obviously being a game with you can

00:59:42.880 --> 00:59:48.580
<v Kristinn Þór Sigurbergsson>imagine we don't have a robust robust suite of unit tests for all of our code base like

00:59:49.370 --> 00:59:54.900
<v Kristinn Þór Sigurbergsson>i mean even automated tests wasn't really like a big when we started doing this and a lot of the

00:59:54.880 --> 00:59:56.760
<v Kristinn Þór Sigurbergsson>code was written in the first place.

00:59:56.760 --> 01:00:00.700
<v Kristinn Þór Sigurbergsson>So doing like an incremental thing was kind of hard.

01:00:00.740 --> 01:00:03.080
<v Kristinn Þór Sigurbergsson>But what we did do when we started,

01:00:03.380 --> 01:00:05.380
<v Kristinn Þór Sigurbergsson>we started just running down the build system

01:00:05.580 --> 01:00:07.140
<v Kristinn Þór Sigurbergsson>and just getting further and further

01:00:07.400 --> 01:00:08.760
<v Kristinn Þór Sigurbergsson>into like an actual build.

01:00:09.680 --> 01:00:10.100
<v Michael Kennedy>So, yeah.

01:00:10.320 --> 01:00:13.800
<v Michael Kennedy>Yeah, it's hard to test graphics and physics and stuff

01:00:14.660 --> 01:00:16.760
<v Michael Kennedy>through automated tests to some degree, right?

01:00:17.020 --> 01:00:19.500
<v Kristinn Þór Sigurbergsson>And when we started, when I, at least when I started,

01:00:19.660 --> 01:00:22.780
<v Kristinn Þór Sigurbergsson>I started talking to Thomas about this

01:00:22.800 --> 01:00:24.660
<v Kristinn Þór Sigurbergsson>when I was a technical director in Yves

01:00:24.780 --> 01:00:26.600
<v Kristinn Þór Sigurbergsson>and then we were thinking about it for Yves

01:00:26.700 --> 01:00:29.420
<v Kristinn Þór Sigurbergsson>and it just, the entire concept of doing it

01:00:29.520 --> 01:00:33.520
<v Kristinn Þór Sigurbergsson>just felt so, just so daunting and like big

01:00:33.720 --> 01:00:36.260
<v Kristinn Þór Sigurbergsson>that I like couldn't imagine how we would do it.

01:00:36.260 --> 01:00:37.880
<v Kristinn Þór Sigurbergsson>We did have a good plan, I think,

01:00:38.100 --> 01:00:39.080
<v Kristinn Þór Sigurbergsson>and that plan kind of,

01:00:41.120 --> 01:00:43.740
<v Kristinn Þór Sigurbergsson>we more or less did that for Frontier,

01:00:44.140 --> 01:00:47.120
<v Kristinn Þór Sigurbergsson>but obviously like the stakes are higher for Yves Online.

01:00:47.460 --> 01:00:49.340
<v Kristinn Þór Sigurbergsson>So like it's a difficult thing.

01:00:49.800 --> 01:00:51.220
<v Jamie Bannister>And I can do a big difference

01:00:51.240 --> 01:00:54.200
<v Jamie Bannister>So a critical thing for EVE is it's an online game,

01:00:54.620 --> 01:00:57.160
<v Jamie Bannister>so the servers need to be running every single day.

01:00:58.880 --> 01:01:04.200
<v Jamie Bannister>We can't afford to be taking the server down for a few hours every day.

01:01:05.000 --> 01:01:06.220
<v Jamie Bannister>We need to kind of keep the product up.

01:01:06.420 --> 01:01:09.460
<v Jamie Bannister>So that's where Frontier kind of running ahead of us was really, really helpful

01:01:09.860 --> 01:01:11.360
<v Jamie Bannister>because they could prove a lot of this out

01:01:12.280 --> 01:01:15.320
<v Jamie Bannister>before they got to the point where they needed to be online 24-7.

01:01:15.580 --> 01:01:19.500
<v Michael Kennedy>Yeah, it gives you a lot of flexibility to do all that testing, doesn't it?

01:01:20.360 --> 01:01:20.940
<v Thomas Dähling>Yeah, absolutely.

01:01:21.120 --> 01:01:25.860
<v Thomas Dähling>especially because the transition to Python 3 is like an all or nothing step. You can't just put

01:01:25.900 --> 01:01:30.180
<v Thomas Dähling>like one of your-- you can't just put the base interpreter on Python 3 and then keep all your

01:01:30.300 --> 01:01:35.480
<v Thomas Dähling>native extensions to Python 2, right? You need to go all the way. Otherwise, it just doesn't work.

01:01:36.100 --> 01:01:40.620
<v Thomas Dähling>That was actually the thing that gave-- I think that was the initial very biggest concern that we

01:01:40.700 --> 01:01:45.760
<v Thomas Dähling>had, like, can we actually pull this off by doing this push out all at once? And I think it was also

01:01:45.760 --> 01:01:50.540
<v Thomas Dähling>the main reason why we did this intermediate step of going to Python 3.8, stackless Python 3.8 first,

01:01:50.640 --> 01:01:55.060
<v Thomas Dähling>because then we knew, okay, we are only changing the Python major version.

01:01:55.520 --> 01:01:58.360
<v Thomas Dähling>We're not also changing the async library underneath the hood,

01:01:58.560 --> 01:02:02.360
<v Thomas Dähling>which a lot of the code still has some timing dependency somewhere, right?

01:02:02.540 --> 01:02:04.240
<v Thomas Dähling>And all these things.

01:02:04.580 --> 01:02:05.520
<v Thomas Dähling>Yeah, 100%.

01:02:05.820 --> 01:02:09.660
<v Michael Kennedy>Well, congratulations for Frontier and good luck on EVE Online.

01:02:10.090 --> 01:02:12.740
<v Michael Kennedy>It sounds, honestly, it sounds like a really exciting project.

01:02:13.220 --> 01:02:14.860
<v Michael Kennedy>Just a lot of fun to work on,

01:02:15.270 --> 01:02:17.780
<v Michael Kennedy>even if it means you can't sleep quite as well at night for a little while.

01:02:18.420 --> 01:02:19.080
<v Michael Kennedy>It's a little stressful.

01:02:19.840 --> 01:02:22.800
<v Kristinn Þór Sigurbergsson>It is exciting.

01:02:23.240 --> 01:02:27.300
<v Kristinn Þór Sigurbergsson>I am proud of the team for delivering it to Frontieras.

01:02:27.400 --> 01:02:30.440
<v Kristinn Þór Sigurbergsson>I'm sure I'll be proud of Eve when they deliver this.

01:02:30.880 --> 01:02:36.380
<v Kristinn Þór Sigurbergsson>What I was fascinated by was that when I started looking into it, I wanted kind of like the war

01:02:36.580 --> 01:02:40.200
<v Kristinn Þór Sigurbergsson>stories and I didn't really find a lot of people that had done it.

01:02:40.200 --> 01:02:44.220
<v Kristinn Þór Sigurbergsson>And I'm sure a lot of people have done a very painful Python 3 migration.

01:02:44.780 --> 01:02:48.700
<v Kristinn Þór Sigurbergsson>So my running theory was that nobody really wanted to talk about it after they had done.

01:02:49.600 --> 01:02:53.140
<v Kristinn Þór Sigurbergsson>some traumatic experience and just let it go.

01:02:54.380 --> 01:02:55.480
<v Michael Kennedy>Exactly, exactly.

01:02:55.840 --> 01:02:57.540
<v Michael Kennedy>All right, well, let's close out the show.

01:02:58.360 --> 01:03:01.640
<v Michael Kennedy>People who are maybe trying to accomplish the same thing,

01:03:01.640 --> 01:03:02.740
<v Michael Kennedy>they have some older code base,

01:03:02.740 --> 01:03:05.020
<v Michael Kennedy>they're trying to migrate even from maybe like old three

01:03:05.320 --> 01:03:07.140
<v Michael Kennedy>to new three, something like that.

01:03:07.740 --> 01:03:10.300
<v Michael Kennedy>Give them really quickly just some parting advice

01:03:11.000 --> 01:03:13.140
<v Michael Kennedy>and also people interested in EVE online.

01:03:13.640 --> 01:03:14.260
<v Michael Kennedy>Kristi, you wanna go first?

01:03:15.500 --> 01:03:17.880
<v Michael Kennedy>So parting advice for people that wanna do this.

01:03:18.180 --> 01:03:23.360
<v Kristinn Þór Sigurbergsson>One of the things that I didn't mention is that we actually used a contractor for a lot

01:03:23.370 --> 01:03:25.500
<v Kristinn Þór Sigurbergsson>of the stuff called Recon Digital.

01:03:26.300 --> 01:03:27.500
<v Kristinn Þór Sigurbergsson>They deserve a shout as well.

01:03:27.500 --> 01:03:30.640
<v Kristinn Þór Sigurbergsson>They did a really good job kind of migrating most of the Python code.

01:03:30.750 --> 01:03:36.220
<v Kristinn Þór Sigurbergsson>And one of the reasons for it was also that developing a live game, we kind of knew that

01:03:36.500 --> 01:03:41.800
<v Kristinn Þór Sigurbergsson>Frontier, the development team, would get immediately derailed when they had to do some

01:03:42.040 --> 01:03:44.040
<v Kristinn Þór Sigurbergsson>feature work and not supporting Python 3.

01:03:44.300 --> 01:03:45.900
<v Kristinn Þór Sigurbergsson>So that was one of the things.

01:03:46.760 --> 01:03:52.000
<v Kristinn Þór Sigurbergsson>So what I would probably argue there is like secure the resources, like make them untouchable

01:03:52.540 --> 01:03:53.500
<v Kristinn Þór Sigurbergsson>while you're doing this migration.

01:03:53.730 --> 01:03:55.660
<v Kristinn Þór Sigurbergsson>Because once you start, you have to finish it.

01:03:55.910 --> 01:03:58.200
<v Kristinn Þór Sigurbergsson>If you pause it, it's almost wasted effort.

01:03:58.420 --> 01:03:59.180
<v Jamie Bannister>Yeah, that's a good point.

01:04:00.140 --> 01:04:02.560
<v Jamie Bannister>Yeah, I was going to actually say something similar to that.

01:04:03.600 --> 01:04:08.220
<v Jamie Bannister>Because we've got one group working on pushing ahead with the migration, we've still got a

01:04:08.220 --> 01:04:11.160
<v Jamie Bannister>whole bunch of developers adding features to the game every day.

01:04:11.470 --> 01:04:14.000
<v Jamie Bannister>So they're adding those features against our 2.7.

01:04:14.920 --> 01:04:20.700
<v Jamie Bannister>And so we're putting a lot of work into trying to make sure that they have the new code they're

01:04:20.900 --> 01:04:23.040
<v Jamie Bannister>adding is as forward compatible as possible.

01:04:24.000 --> 01:04:27.660
<v Jamie Bannister>So a big thing we've rolled out this week is a new linter.

01:04:28.700 --> 01:04:33.060
<v Jamie Bannister>So we're getting and we're going to progressively ratchet up the strictness of it.

01:04:35.340 --> 01:04:38.920
<v Jamie Bannister>So those kind of tools to kind of do code quality inspection, I think, are going to be

01:04:39.060 --> 01:04:40.220
<v Jamie Bannister>really important for us going forward.

01:04:40.800 --> 01:04:41.880
<v Thomas Dähling>Yeah, 100% agree.

01:04:43.040 --> 01:04:44.300
<v Thomas Dähling>Yeah, very much that.

01:04:44.800 --> 01:04:48.160
<v Thomas Dähling>I would add the age-old advice of have really good test coverage.

01:04:48.570 --> 01:04:49.500
<v Thomas Dähling>It helps a lot.

01:04:50.440 --> 01:04:52.860
<v Thomas Dähling>But also, touching upon what Jamie just said,

01:04:53.100 --> 01:04:55.940
<v Thomas Dähling>if you have a product that's live and you need to migrate it,

01:04:56.520 --> 01:04:58.880
<v Thomas Dähling>make sure that you do this in a separate branch

01:04:59.080 --> 01:05:02.280
<v Thomas Dähling>but have constantly ongoing migrations from the Python 2

01:05:02.520 --> 01:05:04.840
<v Thomas Dähling>side to the Python 3 side so that you discover

01:05:04.990 --> 01:05:08.240
<v Thomas Dähling>all of these new pain points that may pop up as soon as possible.

01:05:08.820 --> 01:05:10.740
<v Thomas Dähling>This helped us on Frontier as well,

01:05:11.040 --> 01:05:14.220
<v Thomas Dähling>where sometimes things would come in that were clearly not

01:05:14.240 --> 01:05:17.120
<v Thomas Dähling>Python 3 compatible, and then we could fix it up right away.

01:05:17.640 --> 01:05:19.700
<v Thomas Dähling>And the more frequent you do these integrations,

01:05:19.950 --> 01:05:20.900
<v Thomas Dähling>like the less painful they are.

01:05:21.480 --> 01:05:23.020
<v Thomas Dähling>If you do it once a month, oh my god.

01:05:23.320 --> 01:05:25.140
<v Thomas Dähling>Like there was a period where we only did it

01:05:25.180 --> 01:05:27.140
<v Thomas Dähling>like every other week, and we would basically

01:05:27.380 --> 01:05:29.580
<v Thomas Dähling>spend a whole day just resolving all of the issues.

01:05:29.920 --> 01:05:32.160
<v Thomas Dähling>So do it frequent, do it often, keep it small.

01:05:32.460 --> 01:05:33.040
<v Thomas Dähling>Keep it small.

01:05:33.210 --> 01:05:33.620
<v Michael Kennedy>Good advice.

01:05:34.180 --> 01:05:35.820
<v Michael Kennedy>Do it in small, small bite-sized bits.

01:05:36.020 --> 01:05:36.480
<v Michael Kennedy>Yeah, absolutely.

01:05:36.920 --> 01:05:38.420
<v Michael Kennedy>All right, you guys, thank you for being here.

01:05:38.620 --> 01:05:41.340
<v Michael Kennedy>Everyone listening, check out Eve Online, eveonline.com.

01:05:41.660 --> 01:05:45.000
<v Michael Kennedy>Very cool game, very cool ecosystem.

01:05:45.420 --> 01:05:47.700
<v Michael Kennedy>Check out the video I linked that says This Is Eve.

01:05:47.950 --> 01:05:49.960
<v Michael Kennedy>It's got some great graphics to inspire you as well.

01:05:50.240 --> 01:05:50.520
<v Michael Kennedy>Bye, everyone.

01:05:50.880 --> 01:05:51.080
<v Michael Kennedy>Thank you.

01:05:51.280 --> 01:05:51.600
<v Michael Kennedy>Thank you.

01:05:52.300 --> 01:05:52.580
<v Michael Kennedy>Thank you.

01:05:53.460 --> 01:05:55.600
<v Michael Kennedy>This has been another episode of Talk Python To Me.

01:05:55.980 --> 01:05:56.700
<v Michael Kennedy>Thank you to our sponsors.

01:05:56.950 --> 01:05:58.240
<v Michael Kennedy>Be sure to check out what they're offering.

01:05:58.410 --> 01:05:59.760
<v Michael Kennedy>It really helps support the show.

01:06:00.380 --> 01:06:02.920
<v Michael Kennedy>This episode is sponsored by Sentry's Seer.

01:06:03.220 --> 01:06:05.560
<v Michael Kennedy>If you're tired of debugging in the dark, give Seer a try.

01:06:06.220 --> 01:06:08.300
<v Michael Kennedy>There are plenty of AI tools that help you write code,

01:06:08.520 --> 01:06:11.420
<v Michael Kennedy>but Sentry's Seer is built to help you fix it when it breaks.

01:06:12.020 --> 01:06:17.740
<v Michael Kennedy>Visit talkpython.fm/century and use the code talkpython26, all one word, no spaces,

01:06:18.160 --> 01:06:19.880
<v Michael Kennedy>for $100 in Century credits.

01:06:20.840 --> 01:06:23.720
<v Michael Kennedy>And it's also brought to you by Talk Python Courses.

01:06:24.260 --> 01:06:26.540
<v Michael Kennedy>Course completion certificates are now live.

01:06:26.740 --> 01:06:30.960
<v Michael Kennedy>If you finished a course, there's a certificate waiting for you on your account page right now.

01:06:31.480 --> 01:06:36.660
<v Michael Kennedy>Download it as a PDF or add it to your LinkedIn profile with one click under licenses and

01:06:36.860 --> 01:06:37.200
<v Michael Kennedy>certifications.

01:06:37.900 --> 01:06:39.080
<v Michael Kennedy>Same section as your formal degrees.

01:06:40.120 --> 01:06:44.080
<v Michael Kennedy>Visit training.Talk Python.fm slash account to see what you've already earned.

01:06:44.680 --> 01:06:49.760
<v Michael Kennedy>If you or your team needs to learn Python, we have over 270 hours of beginner and advanced

01:06:50.040 --> 01:06:56.140
<v Michael Kennedy>courses on topics ranging from complete beginners to async code, Flask, Django, HTMX, and even

01:06:56.460 --> 01:06:56.800
<v Michael Kennedy>LLMs.

01:06:57.100 --> 01:06:59.480
<v Michael Kennedy>Best of all, there's no subscription in sight.

01:06:59.860 --> 01:07:01.620
<v Michael Kennedy>Browse the catalog at talkpython.fm.

01:07:02.360 --> 01:07:06.420
<v Michael Kennedy>And if you're not already subscribed to the show on your favorite podcast player, what

01:07:06.420 --> 01:07:06.980
<v Michael Kennedy>are you waiting for?

01:07:07.600 --> 01:07:09.400
<v Michael Kennedy>Just search for Python in your podcast player.

01:07:09.500 --> 01:07:10.380
<v Michael Kennedy>we should be right at the top.

01:07:10.810 --> 01:07:12.360
<v Michael Kennedy>If you enjoyed that geeky rap song,

01:07:12.450 --> 01:07:13.580
<v Michael Kennedy>you can download the full track.

01:07:13.790 --> 01:07:15.700
<v Michael Kennedy>The link is actually in your podcast blur show notes.

01:07:16.540 --> 01:07:17.860
<v Michael Kennedy>This is your host, Michael Kennedy.

01:07:18.260 --> 01:07:19.320
<v Michael Kennedy>Thank you so much for listening.

01:07:19.530 --> 01:07:20.320
<v Michael Kennedy>I really appreciate it.

01:07:20.760 --> 01:07:21.460
<v Michael Kennedy>I'll see you next time.

01:07:31.680 --> 01:07:32.900
I thought of me.

01:07:34.260 --> 01:07:35.360
Can we break the roll?

01:07:37.060 --> 01:07:38.180
Upgrade the code.

01:07:39.060 --> 01:07:40.680
No fear of getting old.

01:07:42.180 --> 01:07:45.560
We tapped into that modern vibe overcame each storm.

01:07:46.400 --> 01:07:49.060
Talk Python and me, async is the norm.