WEBVTT

00:00:00.001 --> 00:00:03.420
By now, surely you've heard how awesome Pydantic version 2 is.

00:00:03.420 --> 00:00:07.780
The team led by Samuel Colvin spent almost a year refactoring and reworking the core

00:00:07.780 --> 00:00:13.320
into a high-performance Rust version while keeping the public API in Python and largely unchanged.

00:00:13.320 --> 00:00:18.780
The main benefit of this has been massive speedups for the frameworks and devs using Pydantic.

00:00:18.780 --> 00:00:24.940
But just how much work is it to take a framework deeply built on Pydantic and make that migration?

00:00:24.940 --> 00:00:26.500
And what are some of the pitfalls?

00:00:26.500 --> 00:00:31.020
On this episode, we welcome back Roman Wright to talk about his experience converting Beanie,

00:00:31.020 --> 00:00:36.460
the popular MongoDB async framework based on Pydantic, from Pydantic 1 to 2.

00:00:36.460 --> 00:00:39.660
And we'll have some fun talking about MongoDB while we're at it.

00:00:39.660 --> 00:00:45.000
This is Talk Python To Me, episode 432, recorded August 16th, 2023.

00:00:45.000 --> 00:01:01.320
Welcome to Talk Python To Me, a weekly podcast on Python.

00:01:01.320 --> 00:01:03.060
This is your host, Michael Kennedy.

00:01:03.060 --> 00:01:08.080
Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython,

00:01:08.080 --> 00:01:10.540
both on fosstodon.org.

00:01:10.540 --> 00:01:13.160
Be careful with impersonating accounts on other instances.

00:01:13.160 --> 00:01:14.120
There are many.

00:01:14.120 --> 00:01:19.180
Keep up with the show and listen to over seven years of past episodes at talkpython.fm.

00:01:19.180 --> 00:01:23.220
We've started streaming most of our episodes live on YouTube.

00:01:23.220 --> 00:01:29.280
Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows

00:01:29.280 --> 00:01:30.740
and be part of that episode.

00:01:31.480 --> 00:01:34.680
This episode is brought to you by Studio 3T.

00:01:34.680 --> 00:01:39.740
Studio 3T is the IDE that gives you full visual control of your MongoDB data.

00:01:39.740 --> 00:01:44.660
With a drag and drop visual query builder and multi-language query code generator,

00:01:44.660 --> 00:01:47.240
new users will find they're up to speed in no time.

00:01:47.240 --> 00:01:52.880
Try Studio 3T for free at talkpython.fm/Studio 2023.

00:01:52.880 --> 00:01:56.940
And it's brought to you by us over at Talk Python Training.

00:01:56.940 --> 00:02:01.080
Did you know we have over 250 hours of Python courses?

00:02:01.080 --> 00:02:03.560
And we have special offers for teams as well.

00:02:03.560 --> 00:02:07.260
Check us out over at talkpython.fm/courses.

00:02:07.260 --> 00:02:11.360
Roman, welcome back to Talk Python To Me.

00:02:11.360 --> 00:02:12.120
Hi.

00:02:12.120 --> 00:02:13.800
Hey, so good to have you back on the show.

00:02:13.800 --> 00:02:14.420
See you again.

00:02:14.420 --> 00:02:15.420
Yeah, you as well.

00:02:15.420 --> 00:02:15.760
Thank you.

00:02:15.760 --> 00:02:16.180
How have you been?

00:02:16.180 --> 00:02:18.860
So it was a nice adventure for me.

00:02:18.860 --> 00:02:25.540
Like, these two years of building Python projects, of moving to other countries.

00:02:25.540 --> 00:02:26.080
So, yeah.

00:02:26.080 --> 00:02:27.300
Are you up for sharing that with people?

00:02:27.300 --> 00:02:28.300
What are you up to?

00:02:28.300 --> 00:02:28.620
Sorry?

00:02:28.620 --> 00:02:30.460
Are you up for sharing where you've moved to?

00:02:30.460 --> 00:02:33.040
Last time we spoke, you were in Berlin, I believe.

00:02:33.040 --> 00:02:34.000
I was in Germany.

00:02:34.000 --> 00:02:35.920
And now I'm in America.

00:02:35.920 --> 00:02:38.880
So, completely different country.

00:02:38.880 --> 00:02:40.560
Completely different continent.

00:02:40.560 --> 00:02:41.180
Yeah.

00:02:41.180 --> 00:02:42.240
Are you enjoying your time there?

00:02:42.240 --> 00:02:42.800
Yeah.

00:02:42.800 --> 00:02:44.660
So, I like it so much.

00:02:44.660 --> 00:02:46.560
Honestly, I like Germany too.

00:02:46.560 --> 00:02:48.500
So, Germany is a great country.

00:02:48.500 --> 00:02:50.720
But I like different things.

00:02:50.720 --> 00:02:53.900
Yeah, but you're probably looking forward to not freezing cold winters.

00:02:53.900 --> 00:02:54.840
I want to try.

00:02:54.840 --> 00:02:57.200
Honestly, I don't know if I will like it or not.

00:02:57.200 --> 00:03:00.280
But I want to try until just a few years, for example.

00:03:00.280 --> 00:03:04.220
I moved to San Diego, California a long time ago.

00:03:04.220 --> 00:03:08.740
I didn't really enjoy the fact that there was no winters and there was no fall.

00:03:08.740 --> 00:03:10.180
It was just always nice.

00:03:10.180 --> 00:03:10.760
Always.

00:03:10.760 --> 00:03:12.940
Until one day I realized, you know what?

00:03:12.940 --> 00:03:15.580
We just headed out mountain biking in the mountains.

00:03:15.840 --> 00:03:16.900
And the weather was perfect.

00:03:16.900 --> 00:03:17.820
And it was February.

00:03:17.820 --> 00:03:21.000
And we didn't even check if it was going to rain or be nice.

00:03:21.000 --> 00:03:21.860
Because it's always nice.

00:03:21.860 --> 00:03:22.380
You know what?

00:03:22.380 --> 00:03:23.500
That's a good trade-off.

00:03:23.500 --> 00:03:26.340
You could do a lot of cool stuff when you live in places like that.

00:03:26.340 --> 00:03:27.800
So, I'm glad to hear.

00:03:27.800 --> 00:03:28.620
I'm glad to hear.

00:03:28.620 --> 00:03:30.600
You'll have to let us know how it goes.

00:03:30.600 --> 00:03:35.020
And Beanie has been going really well as well.

00:03:35.020 --> 00:03:35.500
Right?

00:03:35.660 --> 00:03:39.640
So, when we first spoke, Beanie was just kind of a new project.

00:03:39.640 --> 00:03:44.780
And the thing that caught my eye about it was two really cool aspects.

00:03:44.780 --> 00:03:46.800
Asynchronous and Pydantic.

00:03:46.800 --> 00:03:49.880
I'm like, oh, those things together plus MongoDB sound pretty awesome.

00:03:49.880 --> 00:03:53.360
And so, it's been really fun to watch it grow over the last two years.

00:03:53.360 --> 00:03:55.160
And it's gaining quite a bit of popularity.

00:03:55.400 --> 00:03:55.760
I don't know.

00:03:55.760 --> 00:03:58.100
It's kind of popular, but not that much.

00:03:58.100 --> 00:04:00.300
As Pydantic itself, as FastAPI.

00:04:00.300 --> 00:04:02.960
But, yeah, still popular in context of MongoDB.

00:04:02.960 --> 00:04:05.860
I think it's a little bit different than a web framework, potentially.

00:04:05.860 --> 00:04:06.380
Right?

00:04:06.380 --> 00:04:11.800
Like, it's hard to say, you know, is it as popular as FastAPI or Flask or something like that.

00:04:11.800 --> 00:04:12.020
Right?

00:04:12.080 --> 00:04:16.120
Because those things, you know, those are the contexts in which this is used.

00:04:16.120 --> 00:04:17.220
But not everyone uses Mongo.

00:04:17.220 --> 00:04:19.200
Not everybody cares about async Mongo.

00:04:19.200 --> 00:04:19.820
All that, right?

00:04:19.820 --> 00:04:21.180
There's a lot of filters down.

00:04:21.180 --> 00:04:24.960
But it's, I think you've done a really great job shepherding this project.

00:04:24.960 --> 00:04:27.340
I think you've been really responsive to people.

00:04:27.340 --> 00:04:29.720
I know I've seen the issues coming back and forth.

00:04:29.720 --> 00:04:31.880
I've seen lots of releases on it.

00:04:31.880 --> 00:04:35.240
And I guess the biggest news is around, yeah, you're welcome.

00:04:35.240 --> 00:04:39.920
I think the biggest news is around, you know, when Pydantic 2 came out.

00:04:39.920 --> 00:04:41.780
That kind of changed so many things.

00:04:41.900 --> 00:04:46.840
You know, I spoke to Samuel Colvin about his plan there.

00:04:46.840 --> 00:04:53.560
I spoke to Sebastian Ramirez from FastAPI about, like, what he was thinking and where that was going.

00:04:53.560 --> 00:05:01.440
And it sounded like it was not too much work for people using Pydantic, but quite a bit of work for people like you that was, like, deep inside of Pydantic.

00:05:01.440 --> 00:05:01.800
Yeah?

00:05:01.800 --> 00:05:02.300
Yeah.

00:05:02.300 --> 00:05:06.820
So, honestly, I have a Discord channel for support Bini users.

00:05:06.820 --> 00:05:11.720
And there I was talking with other guys, like, probably, I'm sorry,

00:05:11.720 --> 00:05:14.760
but probably I will not support both versions at the same time.

00:05:14.760 --> 00:05:19.520
Maybe I will have two different branches of Bini with V1 and V2 supporting.

00:05:19.520 --> 00:05:23.880
But finally, I am ended, you know, in a single branch.

00:05:24.260 --> 00:05:27.360
And this was very challenging, honestly, because...

00:05:27.360 --> 00:05:28.100
Okay, interesting.

00:05:28.100 --> 00:05:35.420
I didn't realize you were going backwards in that maintainability there for the people who didn't want to move to Pydantic 2.

00:05:35.560 --> 00:05:37.720
Yeah, there are legacy stuff.

00:05:37.720 --> 00:05:44.360
It must support new features for Pydantic 2.0 also, definitely, because it may be...

00:05:44.360 --> 00:05:50.660
Maybe people want to move to Pydantic 2.0, but they could be stuck on other libraries that support only V1, for example.

00:05:50.660 --> 00:05:53.580
And it can go for a while, like, months.

00:05:53.840 --> 00:05:59.260
And even for me, it took, like, three weeks to make it work.

00:05:59.260 --> 00:06:07.100
Just so people know, pockpython.fm and pythonbytes.fm, both of those are based on MongoDB and Bini.

00:06:07.880 --> 00:06:15.000
And when you came out with the new one, I saw when the release for V2 came out, like, all right, how long until Bini supports this?

00:06:15.000 --> 00:06:18.440
You know, and I saw that you were, like, right on top of it and working on that.

00:06:18.440 --> 00:06:18.860
That was great.

00:06:18.860 --> 00:06:23.140
And then when I went to upgrade it, it was really easy, right?

00:06:23.140 --> 00:06:29.520
Just I use pip-tools and I use pip compile to just get all the latest versions of the dependencies and update the requirements.

00:06:29.520 --> 00:06:35.500
So then I installed a new one and it wouldn't run because there's not because of anything that Bini did,

00:06:35.500 --> 00:06:37.960
but just some changes to Pydantic 2.

00:06:37.960 --> 00:06:44.640
For example, if I had a, let's say there's a database field that was URL and it was an optional string.

00:06:44.640 --> 00:06:50.120
In Pydantic 1, you could say URL colon optional bracket str.

00:06:50.120 --> 00:06:50.840
That's it.

00:06:50.840 --> 00:06:53.680
But there's no default value explicitly set.

00:06:53.680 --> 00:06:56.720
So in Pydantic 2, that's not accepted, right?

00:06:56.720 --> 00:06:57.520
It says, no, no, no.

00:06:57.520 --> 00:07:01.500
If you want it to be none by default, you have to set it to be none explicitly.

00:07:01.500 --> 00:07:04.520
So I had to go through and, like, find all my database documents.

00:07:04.980 --> 00:07:07.920
Basically, anytime there's an optional something, set it equal to none.

00:07:07.920 --> 00:07:08.780
And then that was it.

00:07:08.780 --> 00:07:10.080
That was the upgrade process.

00:07:10.080 --> 00:07:11.600
And now the website runs faster.

00:07:11.600 --> 00:07:12.300
Thank you.

00:07:12.300 --> 00:07:12.900
Welcome.

00:07:12.900 --> 00:07:20.980
Yeah, I invented a middleware, like a few classes and functions that just check if you use Pydantic

00:07:20.980 --> 00:07:22.220
1 or Pydantic 2.

00:07:22.220 --> 00:07:25.680
And based on this, you use a different kind of backends.

00:07:25.680 --> 00:07:28.440
But interface is the same for both.

00:07:28.440 --> 00:07:31.520
So I'm a unified interface inside of Mini.

00:07:31.520 --> 00:07:37.000
So before we get too far down this conversation, give us two quick bits of background information

00:07:37.000 --> 00:07:37.200
here.

00:07:37.200 --> 00:07:39.100
First of all, why MongoDB?

00:07:39.100 --> 00:07:46.840
There's a lot of excitement around things like MySQL, but especially Postgres, relational

00:07:46.840 --> 00:07:49.680
databases, MongoDBs, document database.

00:07:49.680 --> 00:07:50.960
Give us the elevator pitch.

00:07:50.960 --> 00:07:52.780
Why do you like to work with Mongo?

00:07:52.780 --> 00:07:54.760
Honestly, I like to work with all the databases.

00:07:54.760 --> 00:07:57.280
I become databases fun and nerd.

00:07:57.280 --> 00:07:59.660
So I like them all.

00:07:59.660 --> 00:08:02.280
But yeah, MongoDB is a document database.

00:08:02.280 --> 00:08:08.000
And it means the schema, the data scheme is much more flexible than in SQL databases.

00:08:08.000 --> 00:08:14.260
Because in SQL, you use tables, plain tables, while in MongoDB, you use extra documents,

00:08:14.260 --> 00:08:16.140
which could be nested.

00:08:16.140 --> 00:08:20.820
And the level of this nestedness could be really, really high.

00:08:20.820 --> 00:08:23.960
There are some trade-offs based on this.

00:08:23.960 --> 00:08:30.520
The relation system for plain tables could be implemented much simpler than for documents

00:08:30.520 --> 00:08:35.240
because this flexy structure is hard to make nice relations.

00:08:35.240 --> 00:08:44.060
But I'd say it's much more useful for if you use nested data structures in your applications,

00:08:44.060 --> 00:08:49.020
it's much simpler to keep this same data structure in your database.

00:08:49.020 --> 00:08:56.260
And this makes all the processes of development much more easy and more simple to understand

00:08:56.260 --> 00:08:57.180
them, I'd say.

00:08:57.180 --> 00:09:02.660
You don't have the so-called object relational impedance mismatch, where it's like, well,

00:09:02.660 --> 00:09:06.400
you break it all apart like this in the database, and you reassemble it into an object hierarchy

00:09:06.400 --> 00:09:07.000
over here.

00:09:07.000 --> 00:09:09.000
And then you do it again in the other way.

00:09:09.000 --> 00:09:12.180
And all that stuff, it's kind of just mirrored the same, right?

00:09:12.180 --> 00:09:12.480
True.

00:09:12.480 --> 00:09:16.580
And I really like MongoDB to make small projects.

00:09:16.580 --> 00:09:19.080
I mean, when I just want to play with something.

00:09:19.080 --> 00:09:21.560
And I play with data structures a lot.

00:09:21.560 --> 00:09:25.940
And using Postgres or MySQL, I have to do a lot of migrations.

00:09:26.320 --> 00:09:31.500
Because when I change the type of field or just the number of fields, I have to do this

00:09:31.500 --> 00:09:31.820
stuff.

00:09:31.820 --> 00:09:35.380
And this is kind of annoying, because I just want to make fun and to play.

00:09:36.040 --> 00:09:36.780
Yes, exactly.

00:09:36.780 --> 00:09:37.740
Exactly.

00:09:37.740 --> 00:09:41.160
For me, it's easy to make MongoDB fast.

00:09:41.160 --> 00:09:45.060
And it's operationally almost trivial, right?

00:09:45.060 --> 00:09:50.420
If I want to add a field to some collection, I just add it to the class and start using it.

00:09:50.420 --> 00:09:52.680
And it just, it appears, you know, it just shows up.

00:09:52.680 --> 00:09:54.140
And you want adding an asset object?

00:09:54.140 --> 00:09:54.700
You just add it.

00:09:54.700 --> 00:09:58.820
And it just, you don't have to keep running migrations and having server downtime and all

00:09:58.820 --> 00:09:59.020
that.

00:09:59.020 --> 00:09:59.880
It's just, it's glorious.

00:10:00.320 --> 00:10:03.680
Okay, so that's the background where people maybe haven't done anything with Mongo.

00:10:03.680 --> 00:10:04.740
What about Beanie?

00:10:04.740 --> 00:10:05.700
What is Beanie?

00:10:05.700 --> 00:10:06.580
Really quick for people.

00:10:06.580 --> 00:10:07.560
We're talking about it a bit.

00:10:07.560 --> 00:10:10.340
But give us the quick rundown on Beanie and why you built it.

00:10:10.340 --> 00:10:13.260
Like there were other things that talked to MongoDB and Python before.

00:10:13.260 --> 00:10:13.680
Yeah.

00:10:13.680 --> 00:10:15.440
So there's a lot of tools.

00:10:15.440 --> 00:10:18.080
There is MongoEngine, which is nice and which is official.

00:10:18.080 --> 00:10:18.920
Yeah, I like MongoEngine too.

00:10:18.920 --> 00:10:19.380
Yeah.

00:10:19.480 --> 00:10:19.760
Yeah.

00:10:19.760 --> 00:10:26.280
But one day I was playing again with new technologies and FastAPI was super new at

00:10:26.280 --> 00:10:26.620
time.

00:10:26.620 --> 00:10:30.500
It was like, it wasn't that famous at time, like three years ago.

00:10:30.500 --> 00:10:32.300
And already was super nice.

00:10:32.300 --> 00:10:37.680
And I wanted to play with it before I can use it in my production projects.

00:10:37.680 --> 00:10:45.060
And I found that there is no nice, let's say, connector to MongoDB from FastAPI because

00:10:45.060 --> 00:10:50.940
there is nothing that could support and Pydantic and asynchronous MongoDB driver motor.

00:10:50.940 --> 00:10:52.660
And I decided like, but why?

00:10:52.660 --> 00:10:54.520
I think I can implement it myself.

00:10:54.520 --> 00:10:55.180
Why not?

00:10:55.180 --> 00:10:58.240
And I made a very small, tiny ODM.

00:10:58.240 --> 00:11:01.280
I even thought it would be tiny all the time.

00:11:01.280 --> 00:11:05.240
It was like, you know, it could support only models of the documents.

00:11:05.240 --> 00:11:06.540
It could insert them.

00:11:06.540 --> 00:11:13.100
And all the operations of MongoDB, it wasn't hidden inside of Beanie.

00:11:13.400 --> 00:11:17.180
You had to use MQL, Mongo query language there.

00:11:17.180 --> 00:11:18.940
So I just released this.

00:11:18.940 --> 00:11:26.640
And somehow in one month it got not that popular, but people just came to me and like, I like what

00:11:26.640 --> 00:11:27.080
you did.

00:11:27.080 --> 00:11:29.520
Could you please add this feature and that feature?

00:11:29.520 --> 00:11:31.960
And this part works wrongly.

00:11:31.960 --> 00:11:33.080
So please fix this.

00:11:33.080 --> 00:11:37.060
And I was like, whoa, I didn't know, but I made an open source product.

00:11:37.060 --> 00:11:38.080
Wow.

00:11:38.080 --> 00:11:38.720
Yeah, that's cool.

00:11:38.720 --> 00:11:42.440
Some weird podcaster guy goes, this is great, except for where the index is.

00:11:42.660 --> 00:11:46.460
Yeah, this was like first or maybe second week after I published it.

00:11:46.460 --> 00:11:49.420
And yeah, you came to my, to GitHub issues.

00:11:49.420 --> 00:11:50.520
Could you add indexes?

00:11:50.520 --> 00:11:52.480
And I was like, I forgot about indexes.

00:11:52.480 --> 00:11:54.580
Yeah, I have to add them.

00:11:54.580 --> 00:11:56.700
Yeah, indexes are like database magic.

00:11:56.700 --> 00:11:57.780
They're just awesome.

00:11:57.780 --> 00:12:00.300
So yeah, and this was kind of playground project.

00:12:00.440 --> 00:12:01.740
And now this is nice.

00:12:01.740 --> 00:12:03.060
So damn, I'm going to be.

00:12:03.060 --> 00:12:06.160
It's been really, really reliable for all the work that we've been doing.

00:12:06.160 --> 00:12:07.660
So good work on that.

00:12:07.660 --> 00:12:08.480
Let's see.

00:12:08.480 --> 00:12:11.020
I guess there's two angles to go here.

00:12:11.020 --> 00:12:20.340
One, if we go over the releases, the big release is this 1.21.0, which says Pydantic V2 support.

00:12:20.680 --> 00:12:25.480
So I want to spend a lot of time talking to you about like, what was your experience going from Pydantic 1 to 2?

00:12:25.480 --> 00:12:30.140
Because as you said, there's the really famous ones like FastAPI and others.

00:12:30.140 --> 00:12:34.520
But there's many, many projects out there that use Pydantic.

00:12:34.520 --> 00:12:36.620
I wonder if we could get it to show it.

00:12:36.620 --> 00:12:42.920
So, you know, GitHub has that feature where it shows used by 229,000 projects.

00:12:43.420 --> 00:12:46.040
228,826 projects.

00:12:46.040 --> 00:12:48.060
Yeah, they're all the projects with Pydantic now, right?

00:12:48.060 --> 00:12:49.140
Exactly.

00:12:49.140 --> 00:12:51.460
Just on GitHub, use Pydantic.

00:12:51.460 --> 00:12:57.060
So, you know, many of them still haven't necessarily done this work to move to 2.

00:12:57.060 --> 00:12:59.580
And so I want to make that the focus of our conversation.

00:12:59.580 --> 00:13:04.900
However, since we had a nice episode on Beanie before, before we get into that aspect,

00:13:04.900 --> 00:13:08.900
let's just do a catch up on like what's happened with Beanie in the last two years.

00:13:08.900 --> 00:13:12.500
What are some of the cool new features and things that you want to highlight for folks?

00:13:12.780 --> 00:13:14.440
I added a lot of features, honestly.

00:13:14.440 --> 00:13:17.480
But there were a few really big.

00:13:17.480 --> 00:13:18.800
I really like one.

00:13:18.800 --> 00:13:21.220
I didn't know that it could be needed for anybody.

00:13:21.220 --> 00:13:25.440
But I was continuously asked about, please add, add this.

00:13:25.440 --> 00:13:26.920
And I didn't want to add.

00:13:26.920 --> 00:13:28.420
But finally, I added.

00:13:28.420 --> 00:13:30.960
And now I love this so much.

00:13:30.960 --> 00:13:32.900
This is called inheritance.

00:13:32.900 --> 00:13:35.820
You can inherit documents.

00:13:35.820 --> 00:13:42.140
So you can make a big inherited structure like car, then vehicle, then

00:13:42.140 --> 00:13:47.780
from here you can inherit bicycle, by car, and from car inherit something else.

00:13:47.780 --> 00:13:54.140
And the thing is, everything will be stored in the same collection, in the same MongoDB collection.

00:13:54.140 --> 00:13:59.700
And if you want to make statistics over all the types, you can do it.

00:13:59.700 --> 00:14:05.780
And when you need to operate only with a type or subtype, you can do it as well.

00:14:05.780 --> 00:14:07.560
You can choose what you want to do.

00:14:07.560 --> 00:14:12.020
And I know this feature is used in productions now in many projects.

00:14:12.020 --> 00:14:13.200
And this is nice.

00:14:13.200 --> 00:14:14.840
Yeah, this is really cool.

00:14:14.840 --> 00:14:27.640
So when I first heard about it, my first impression was, okay, so instead of deriving from beanie.document, you create some class that has some common features, maybe properties and validation and stuff.

00:14:27.640 --> 00:14:30.020
And then other documents can derive from it.

00:14:30.060 --> 00:14:33.020
So, like you said, bicycle versus car.

00:14:33.020 --> 00:14:37.060
But in my mind, those would still go into different collections, right?

00:14:37.060 --> 00:14:38.060
They would go in different collections.

00:14:38.060 --> 00:14:42.640
And that would just be a simpler way to have the code that would have a significant bit of reuse.

00:14:42.700 --> 00:14:49.940
But the fact that they all go into the same collection and the documents are kind of supersets of each other, I think that's pretty interesting.

00:14:49.940 --> 00:14:51.780
I hadn't really thought about how I'd use that.

00:14:51.780 --> 00:14:57.240
This portion of Talk Python To Me is brought to you by Studio 3T.

00:14:57.240 --> 00:14:59.720
Do you use MongoDB for your apps?

00:14:59.720 --> 00:15:04.420
As you may know, I'm a big fan of Mongo and it powers all the Talk Python web apps and APIs.

00:15:04.420 --> 00:15:08.540
I recently created a brand new course called MongoDB with Async Python.

00:15:08.940 --> 00:15:13.880
This course is an end-to-end journey on getting fully up to speed with Mongo and Python.

00:15:13.880 --> 00:15:18.720
When writing this course, I had to choose a GUI query and management tool to use and recommend.

00:15:18.720 --> 00:15:20.540
I chose Studio 3T.

00:15:20.540 --> 00:15:28.500
It strikes a great balance between being easy to use, very functional and remaining true to the native MongoDB shell CLI experience.

00:15:28.500 --> 00:15:32.520
That's why I'm really happy that Studio 3T has joined the show as a sponsor.

00:15:32.520 --> 00:15:36.080
Their IDE gives you full visual control of your data.

00:15:36.260 --> 00:15:43.180
Plus, with a drag-and-drop visual query builder and a multi-language code generator, new users will find they're up to speed in no time.

00:15:43.180 --> 00:15:47.800
For your team members who don't know MongoDB query syntax but are familiar with SQL,

00:15:47.800 --> 00:15:57.300
they can even query MongoDB directly with Studio 3T using SQL and migrate tabular data from relational databases into MongoDB documents.

00:15:57.300 --> 00:16:01.200
Recently, Studio 3T has made it even easier to collaborate, too.

00:16:01.260 --> 00:16:08.240
Their brand-new team-sharing feature allows you to drag-and-drop queries, scripts, and connections into permission-based shared folders.

00:16:08.240 --> 00:16:12.480
Save days of onboarding team members and tune queries faster than ever.

00:16:12.480 --> 00:16:18.180
Try Studio 3T today by visiting talkpython.fm/studio2023.

00:16:18.180 --> 00:16:20.900
The links in your podcast player show notes.

00:16:20.900 --> 00:16:23.040
And download the 30-day trial for free.

00:16:24.040 --> 00:16:24.600
Studio 3T.

00:16:24.600 --> 00:16:27.480
It's the MongoDB management tool I use for talkpython.

00:16:27.480 --> 00:16:31.520
You can do even two different...

00:16:31.520 --> 00:16:37.980
If you want to count all the vehicles, you can do it without making requests to each of the collections,

00:16:37.980 --> 00:16:39.720
because everything is in the single collection.

00:16:39.720 --> 00:16:43.400
And you can do this with different fields there as well.

00:16:43.400 --> 00:16:45.700
And you can make aggregations over all of them.

00:16:45.980 --> 00:16:48.220
And even over cars separately.

00:16:48.220 --> 00:16:49.200
This is nice.

00:16:49.200 --> 00:16:50.160
And yeah, I like it.

00:16:50.160 --> 00:16:53.900
Does the record have something, some kind of indicator of what...

00:16:53.900 --> 00:16:55.300
Yeah, inside there is...

00:16:55.300 --> 00:16:56.180
What class it is.

00:16:56.180 --> 00:16:57.380
It's like, I'm a car class.

00:16:57.380 --> 00:16:58.440
I'm a bike class.

00:16:58.440 --> 00:16:58.940
Yeah, yeah.

00:16:58.940 --> 00:16:59.980
You can specify...

00:16:59.980 --> 00:17:04.400
Originally, it is called class name or something like this with underscore.

00:17:04.660 --> 00:17:07.640
But you can specify which fields would work for this.

00:17:07.640 --> 00:17:09.400
So you can specify the name of this field.

00:17:09.400 --> 00:17:14.300
And in this field, it stores not only the name of the class, but the structure itself.

00:17:14.300 --> 00:17:19.020
Like for bus, it will keep vehicle, car, bus.

00:17:19.020 --> 00:17:19.540
Yeah.

00:17:19.540 --> 00:17:20.680
So in this field.

00:17:20.680 --> 00:17:21.040
Okay.

00:17:21.040 --> 00:17:22.600
That's why it will be able to...

00:17:22.600 --> 00:17:27.320
Even on the database level, it will understand the hierarchy of this object.

00:17:27.320 --> 00:17:27.560
Right.

00:17:27.560 --> 00:17:34.540
And so if you want to do data science-y things, you could use the aggregation framework to run a bunch of those types of queries on it.

00:17:34.540 --> 00:17:34.680
Right.

00:17:34.680 --> 00:17:34.940
Yeah.

00:17:34.940 --> 00:17:41.040
And it's better to do all this stuff on the database layer because Python is not that fast with iterations.

00:17:41.040 --> 00:17:43.200
While MongoDB is super fast.

00:17:43.200 --> 00:17:43.700
Yeah.

00:17:43.700 --> 00:17:49.800
And plus, you know, you don't need necessarily to pull all the data back just to read some field or whatever.

00:17:49.800 --> 00:17:50.120
Right.

00:17:50.120 --> 00:17:51.340
So, yeah, that's really cool.

00:17:51.340 --> 00:17:55.400
This is not what I expected when I first heard about it, but this is quite cool.

00:17:55.400 --> 00:18:00.460
First time I heard this about this feature, I was like, nobody wants this.

00:18:00.460 --> 00:18:02.340
Why do you try to...

00:18:02.340 --> 00:18:05.900
But then I found how flexible this is getting to be.

00:18:05.900 --> 00:18:07.780
And so, yeah, this is nice.

00:18:07.900 --> 00:18:19.460
The reason I guess it's a surprise to me is it leverages an aspect of MongoDB that's in document databases in general that are interesting, but that I don't find myself using very much is in that you don't have to have...

00:18:19.460 --> 00:18:21.860
There's not a real structured schema.

00:18:21.860 --> 00:18:24.900
And a lot of people say that and kind of get a sense for it.

00:18:24.940 --> 00:18:28.740
For me, that's always meant like, well, the database doesn't control the schema, but my code does.

00:18:28.740 --> 00:18:30.260
And that's probably going to be the same.

00:18:30.260 --> 00:18:30.620
Right.

00:18:30.700 --> 00:18:36.460
So there's kind of an implicit static schema at any given time that matches the code.

00:18:36.460 --> 00:18:41.440
But you can do things like put different records into the same collection.

00:18:41.440 --> 00:18:46.720
You wouldn't do it just like, well, here's a user and here's a blog post and just put them in the same collection.

00:18:46.720 --> 00:18:47.540
That would be insane.

00:18:48.180 --> 00:18:52.900
But there's, you know, if you have this commonality of this base class, I can see why you might do this.

00:18:52.900 --> 00:18:53.480
It's interesting.

00:18:53.480 --> 00:18:53.840
Yeah.

00:18:53.840 --> 00:18:59.420
And in this context, blog post or video post could be different by structure, but could be stored in the single collection.

00:18:59.420 --> 00:19:02.640
One other thing on the page here that we could maybe talk about is link.

00:19:02.640 --> 00:19:04.380
I want to tell people about what link.

00:19:04.380 --> 00:19:10.180
MongoDB is not a non-relational database, but you can force it to work with relations.

00:19:10.460 --> 00:19:21.080
There are even, there is a data type in MongoDB called db-ref, db-reference, which is used to work with this link type in Bini.

00:19:21.080 --> 00:19:26.980
So in Bini, with this generic type link, you can put inside of the link any document type.

00:19:26.980 --> 00:19:29.140
It can make relations based on this link.

00:19:29.140 --> 00:19:36.000
So you can fetch linked documents from another collections using just standard find operations in Bini.

00:19:36.000 --> 00:19:39.100
This is, there is kind of magic under the hood.

00:19:39.340 --> 00:19:48.220
I use, instead of using find operations, MongoDB find operations, I use aggregation framework of MongoDB, but it is hidden under the hood of Bini.

00:19:48.220 --> 00:19:51.080
And so, yeah, and you can use relations then.

00:19:51.080 --> 00:19:57.780
And the nice thing about new features, because link already was implemented, I think, two years ago.

00:19:57.780 --> 00:20:02.120
But again, I don't come up with my own features, I think.

00:20:02.120 --> 00:20:04.260
Every feature somebody asked me for.

00:20:04.520 --> 00:20:11.660
And I was asked for another feature to make backtracking, back references for these links.

00:20:11.660 --> 00:20:19.260
Like, if you have a link from one document to another, another document should be able to have to fetch this relation backwards.

00:20:19.260 --> 00:20:23.220
So in this case, you've got an owner which has a list of vehicles.

00:20:23.220 --> 00:20:26.740
But given a vehicle, you would like to ask who is its owner, right?

00:20:26.820 --> 00:20:27.100
True.

00:20:27.100 --> 00:20:27.440
Okay.

00:20:27.440 --> 00:20:28.840
And I implemented this.

00:20:28.840 --> 00:20:34.740
I named backlinks, backlink, and it can just fetch it in reverse direction.

00:20:34.740 --> 00:20:35.320
That's cool.

00:20:35.320 --> 00:20:45.260
And the nice thing about this is it only uses the magic of aggregations, and it doesn't store anything for backlinks fields in the collection itself.

00:20:45.260 --> 00:20:45.620
Okay.

00:20:45.620 --> 00:20:52.080
In the MongoDB document, you never will find this field for backlink, because everything you need is on the link.

00:20:52.480 --> 00:20:53.260
And this is nice.

00:20:53.260 --> 00:20:54.340
Yeah, that is really cool.

00:20:54.340 --> 00:20:59.520
In the queries, in the find statement, you add fetch underscore link equals true.

00:20:59.520 --> 00:21:00.820
And that's kind of like a join.

00:21:00.820 --> 00:21:01.720
Is that how that works?

00:21:01.720 --> 00:21:02.400
There are options.

00:21:02.400 --> 00:21:04.680
Eager versus lazy loading type of thing.

00:21:04.680 --> 00:21:07.440
When you find without this option, default it is false.

00:21:07.440 --> 00:21:11.880
You will see in the field, in the link at field, you will see only link itself.

00:21:11.880 --> 00:21:15.660
It will be linked with ID inside of the object.

00:21:15.840 --> 00:21:22.000
But if you put fetch, and you can fetch it manually, like with method dot fetch, it will work.

00:21:22.000 --> 00:21:28.120
But when you put fetch links true, it will fetch everything automatically on the database layer.

00:21:28.120 --> 00:21:30.540
And yeah, it will return all the link documents.

00:21:30.540 --> 00:21:31.320
That's really cool.

00:21:31.320 --> 00:21:31.900
Other one?

00:21:31.900 --> 00:21:33.080
Lazy parsing.

00:21:33.080 --> 00:21:34.440
I mean, we all want to be lazy.

00:21:34.440 --> 00:21:35.560
But what are we doing here?

00:21:35.560 --> 00:21:36.240
What is this one?

00:21:36.380 --> 00:21:42.100
Yeah, so this is, in some cases, Bini could be used for really high load projects.

00:21:42.100 --> 00:21:47.720
And sometimes you need to fetch like thousands of documents in a moment.

00:21:47.720 --> 00:21:55.900
And the nature of Pydantic is synchronous, not synchronous, because it uses CPU bound operations there.

00:21:55.900 --> 00:22:02.240
And when you fetch hundreds of documents, even thousands of documents, you completely block your system.

00:22:02.520 --> 00:22:06.660
Because a lot of loops to parse data, to validate data, and etc.

00:22:06.660 --> 00:22:12.420
And if you use it in asynchronous framework, this is not behavior that you like to have.

00:22:12.420 --> 00:22:14.640
And to fix this problem.

00:22:14.640 --> 00:22:18.820
Maybe even in a synchronous framework, it might be the behavior you don't want to have as well.

00:22:18.820 --> 00:22:19.160
Yeah.

00:22:19.160 --> 00:22:19.400
Right?

00:22:19.400 --> 00:22:19.900
Even then.

00:22:19.900 --> 00:22:20.220
Yeah, yeah.

00:22:20.220 --> 00:22:20.920
This is true.

00:22:20.920 --> 00:22:23.480
But even in asynchronous, you don't accept.

00:22:23.480 --> 00:22:24.160
Yeah, exactly.

00:22:24.320 --> 00:22:28.600
But it's totally reasonable to think, well, I'm going to do a query against this document.

00:22:28.600 --> 00:22:30.760
And it's got some nested stuff.

00:22:30.760 --> 00:22:32.700
Maybe it's a big sort of complex one.

00:22:32.700 --> 00:22:35.620
But you really just want three fields in this case.

00:22:35.620 --> 00:22:38.160
Now, you can use projections, right?

00:22:38.160 --> 00:22:39.840
Like that is the purpose of projections.

00:22:39.840 --> 00:22:42.060
But it limits the flexibility.

00:22:42.060 --> 00:22:44.520
Because you only have those fields that were projected.

00:22:44.520 --> 00:22:50.640
And in different situations, maybe you don't really know what parts you're going to use, right?

00:22:50.640 --> 00:22:51.980
You can have a complicated approach.

00:22:51.980 --> 00:22:52.200
Yeah.

00:22:52.200 --> 00:22:56.000
So this kind of lets the consumer of the query use only what they need, right?

00:22:56.000 --> 00:22:56.240
Yeah.

00:22:56.240 --> 00:23:02.080
And so when you use this lazy parsing, Pydantic doesn't parse anything on the initial call.

00:23:02.080 --> 00:23:07.820
Like you receive everything and store everything in a raw format in dictionaries, in Python dictionaries there.

00:23:07.820 --> 00:23:16.440
And when you call any field of the document, just parse it using Pydantic tools to parse as Pydantic do it internally.

00:23:16.440 --> 00:23:20.280
So is this lazy parse primarily implemented by Pydantic?

00:23:20.280 --> 00:23:22.220
Or is this something you've done on top of Pydantic?

00:23:22.220 --> 00:23:24.440
I implemented my own library for this.

00:23:24.440 --> 00:23:26.400
Like it's on top of Pydantic for sure.

00:23:26.400 --> 00:23:31.080
But it uses, in Pydantic, there are tools and different in V1 and V2.

00:23:31.080 --> 00:23:38.440
So the name of this tool is different, but you can parse something into type, not into base model, but just into type.

00:23:38.440 --> 00:23:44.360
You can provide a type and the value to be parsed into this type and it can parse it.

00:23:44.360 --> 00:23:45.180
So I use this.

00:23:45.180 --> 00:23:54.600
And additionally, I had to handle with all the validator stuff because this is a very important part of Pydantic and you have to be able to validate things.

00:23:54.600 --> 00:24:01.880
And with lazy parsing, if it sees that there are root validators, then it will validate it against any field.

00:24:01.880 --> 00:24:07.720
Or if there is a field-specific validator, it will validate it against the field if this field was called.

00:24:07.720 --> 00:24:11.260
So yeah, I had to do some magic with Pydantic there.

00:24:11.960 --> 00:24:28.500
But it, especially with Pydantic V1, which was slower than V2, significantly slower, it was very helpful for people who have to fetch really big amounts of documents and to not block their pipeline in this step.

00:24:28.500 --> 00:24:30.280
This is a nice feature also.

00:24:30.440 --> 00:24:32.040
Yeah, I think this is a really nice feature.

00:24:32.040 --> 00:24:33.200
What's the harm?

00:24:33.200 --> 00:24:41.000
And does it make certain things slower if you're going to use every field or, you know, why not just turn this on all the time, right?

00:24:41.000 --> 00:24:42.000
That's my question.

00:24:42.000 --> 00:24:42.940
Yes, for sure.

00:24:42.940 --> 00:24:44.120
There are trade-offs.

00:24:44.120 --> 00:24:54.480
If you will use this all the time, it would be like around twice slower than just Pydantic validation if you will use all the fields.

00:24:54.480 --> 00:24:57.380
If you will use just a few fields, it would be faster.

00:24:57.640 --> 00:25:06.960
But I didn't turn it on by default because in general case, when people just want to fetch 10, maybe 20 documents and use all the fields of them, it would be slower.

00:25:06.960 --> 00:25:08.100
That's kind of what I expected.

00:25:08.100 --> 00:25:13.700
But if you've got a really complicated document and you only use a few fields here and there, then it seems like a real win.

00:25:13.700 --> 00:25:15.260
But you're going to use everything anyway.

00:25:15.260 --> 00:25:21.000
And especially for Pydantic V2, when all the validation happens on the Rust layer.

00:25:21.000 --> 00:25:27.040
But here I cannot do this because I cannot, you know, I cannot put the logic into the Rust layer because there is no Rust layer for Beanie.

00:25:27.440 --> 00:25:35.120
And if you will fetch all the fields of the documents using this lazy parsing, everything will happen on the Python layer instead of the Rust layer.

00:25:35.120 --> 00:25:37.460
And it would be as slow as V1.

00:25:37.460 --> 00:25:39.180
So we will not see benefits.

00:25:39.620 --> 00:25:49.860
It's interesting, even in the V1 version of Pydantic, but now with Pydantic 2 being roughly 22 times faster, that all of a sudden you want to let Pydantic do its thing if it can.

00:25:49.860 --> 00:25:50.800
Yeah, true.

00:25:50.800 --> 00:26:00.300
Speaking of Pydantic getting some speed up from Rust, is any part of Beanie some other runtime compilation story than pure Python?

00:26:00.300 --> 00:26:04.740
Is there like a Cython thing or a Numba or any of those?

00:26:04.740 --> 00:26:15.560
The thing about the speed of Beanie is, Beanie is not about, so as Pydantic is very CPU bound, all the stuff happens on the CPU layer.

00:26:15.560 --> 00:26:21.400
While Beanie uses mostly input-output operations because it interacts with the database.

00:26:21.400 --> 00:26:28.120
And for this, just default, I think, await pattern of Python uses, works the best.

00:26:28.120 --> 00:26:37.320
And all the time, if there are any delays, it's most likely about this interaction process between application and MongoDB.

00:26:37.320 --> 00:26:45.080
It could be network, it could be just delay from the query and et cetera, but not Beanie because Beanie doesn't compute anything.

00:26:45.080 --> 00:26:47.560
Beanie doesn't do very much, I guess, right?

00:26:47.560 --> 00:26:54.520
It coordinates motor, the asynchronous engine from MongoDB, and it coordinates Pydantic.

00:26:54.520 --> 00:27:00.760
It kind of clicks those together using async and await, a nice query API that you put together.

00:27:00.760 --> 00:27:07.400
And so, right, it's more about letting motor be fast and letting Pydantic be fast and getting out of the way, I suppose.

00:27:07.400 --> 00:27:08.360
This is true, so yeah.

00:27:08.360 --> 00:27:14.200
Beanie is mostly about making some magic and convert Python syntax into MongoDB syntax.

00:27:14.200 --> 00:27:15.560
And thank you for that.

00:27:15.560 --> 00:27:16.360
That's really nice.

00:27:16.360 --> 00:27:19.240
It's super nice the way that the syntax works, right?

00:27:19.240 --> 00:27:23.160
The fact that you're able to use native operators, for example, right?

00:27:23.160 --> 00:27:24.360
You do the queries.

00:27:24.360 --> 00:27:25.000
I really like that.

00:27:25.000 --> 00:27:25.640
Yeah, it was.

00:27:25.640 --> 00:27:26.120
It is.

00:27:26.120 --> 00:27:29.720
I don't like when somebody uses this in production applications.

00:27:29.720 --> 00:27:32.840
I mean, when, because it's hard to find problems.

00:27:32.840 --> 00:27:37.720
But when we are talking about libraries, this is really nice when it supports Python syntax.

00:27:37.720 --> 00:27:39.080
So that's why I decided to implement it.

00:27:39.080 --> 00:27:42.040
People shouldn't get too crazy with overloading their own operators.

00:27:43.080 --> 00:27:44.360
But as an API, it's really good.

00:27:44.360 --> 00:27:47.720
So for example, in this case, you have a sample document and it has a number.

00:27:47.720 --> 00:27:49.720
And so the query is sample.find.

00:27:49.720 --> 00:27:54.440
And then the argument is sample.number =10, right?

00:27:54.440 --> 00:27:57.000
Which is exactly the way you would do it in an if statement.

00:27:57.000 --> 00:28:04.360
You know, contrast that with other languages or other frameworks such as MongoEngine, which I used previously and is nice.

00:28:04.360 --> 00:28:10.840
But you would say sample.find and then just number_eq =10, right?

00:28:10.840 --> 00:28:13.160
And you're like, I know what that means.

00:28:13.160 --> 00:28:20.920
But it's not speaking to me the same way as if I was just doing a raw database query or writing pure Python, right?

00:28:20.920 --> 00:28:23.400
Yeah, it sounds like you have to learn another one of English, right?

00:28:23.400 --> 00:28:23.960
Exactly.

00:28:23.960 --> 00:28:32.920
You've got to like, if you want to do a nested thing, it's the double underscore, you know, it'd be like number double underscore item underscore eq =10.

00:28:32.920 --> 00:28:34.040
You're like, oh, my goodness.

00:28:34.040 --> 00:28:35.320
So yeah, that's kind of tricky.

00:28:35.320 --> 00:28:36.040
Definitely true.

00:28:36.040 --> 00:28:36.600
Cool.

00:28:36.600 --> 00:28:44.840
Okay, well, let's talk about this, this upgrading story for the 22, 229,000 other folks out there who maybe haven't done this.

00:28:44.840 --> 00:28:56.040
So a while ago, back in 2022, almost exactly to the day, a year ago, I had Samuel Colvin on to talk about the plan to move to Pydantic v2, why he did it.

00:28:56.040 --> 00:28:57.240
It was really interesting.

00:28:57.240 --> 00:28:58.440
So that's worth listening to.

00:28:58.440 --> 00:29:04.760
If people want to learn more, as well as I had Sebastian Ramirez and Samuel Colvin on to talk about it live at PyCon.

00:29:04.760 --> 00:29:05.480
And that was fun too.

00:29:05.480 --> 00:29:10.200
So if people want background on what is the story of Pydantic 2, they can check that out.

00:29:10.200 --> 00:29:19.480
But the big announcement was on June 30th, a couple months ago, I guess that's a month and a half ago, Pydantic v2 is here after just one year of hard work.

00:29:19.480 --> 00:29:25.880
That was a huge project for the Pydantic folks, which, you know, they've done a great job on it.

00:29:25.880 --> 00:29:32.040
And I guess the big takeaway really is that Pydantic v2 is quite a bit faster.

00:29:32.040 --> 00:29:33.400
Maybe you could speak to that.

00:29:33.400 --> 00:29:40.440
And it's mostly but not exactly the same because, as you already pointed out, the core of it is rewritten in Rust for performance reasons.

00:29:40.440 --> 00:29:41.960
I've made a lot of tests.

00:29:41.960 --> 00:29:52.120
It is much faster when you talk about validation of the models itself, especially when you validate parts really nested and complicated models.

00:29:52.120 --> 00:29:56.840
Then it's much, much faster than Python, then v1 implementation.

00:29:56.840 --> 00:30:06.120
While with Bini, you still can see significant upgrade, performance upgrade, but not that much because with Bini works with MongoDB.

00:30:06.120 --> 00:30:14.920
And there is this input output operation, which is slow and which could not be upgraded by just by decreasing processing time.

00:30:14.920 --> 00:30:15.320
Sure.

00:30:15.320 --> 00:30:23.400
When we are talking about simple documents, it's not that visible, like 10, sometimes 20% faster.

00:30:23.400 --> 00:30:32.200
But when we talk about nested documents, when there are nested dictionaries or nested lists of dictionaries, then it's much, much faster.

00:30:32.200 --> 00:30:37.160
In my Allo test, it is twice faster, v2 against v1.

00:30:37.160 --> 00:30:49.400
And I was super impressed by this because I was expecting this, expecting this, that it would not be that, that faster than as a pedantic v2 itself because of this output operation.

00:30:49.400 --> 00:30:51.720
But yeah, this is so this crazy.

00:30:51.720 --> 00:30:52.040
Right.

00:30:52.040 --> 00:30:55.960
Because it's not just the parsing that Bini does, right?

00:30:55.960 --> 00:30:58.040
Bini sends the message over to Mongo.

00:30:58.040 --> 00:30:59.320
The network does some stuff.

00:30:59.320 --> 00:31:03.240
Mongo does its thing, sends it back, serialized as Bison.

00:31:03.240 --> 00:31:07.560
And then it's got to deserilize into objects somehow.

00:31:07.560 --> 00:31:09.240
And then the pedantic part kicks in.

00:31:09.240 --> 00:31:09.800
Right.

00:31:09.800 --> 00:31:11.800
And plus all the extra bits you've already talked about.

00:31:11.800 --> 00:31:12.040
Right.

00:31:12.040 --> 00:31:14.360
So it, it can only affect that part.

00:31:14.360 --> 00:31:19.320
But I think it, your example here shows, you know, standard computer science answer.

00:31:19.320 --> 00:31:20.520
It depends, right?

00:31:20.520 --> 00:31:22.520
As a, as a faster, it depends.

00:31:22.520 --> 00:31:28.840
I would guess the more complicated your document is, the bigger bonus you get, which you've already said.

00:31:28.840 --> 00:31:31.960
And the more documents you return.

00:31:31.960 --> 00:31:35.960
So if I return one record from the database, it's got five fields.

00:31:35.960 --> 00:31:39.080
The amount of that processing that is pydantic is small.

00:31:39.080 --> 00:31:43.880
But if I return a thousand records and there's all that serialization, like, you know, the database

00:31:43.880 --> 00:31:46.360
has kind of done more or less the same amount of work.

00:31:46.360 --> 00:31:47.400
It's streamed the stuff back.

00:31:47.400 --> 00:31:52.040
But at this, when it gets to Python, it's like, whoa, I've got a lot of stuff to validate and parse.

00:31:52.040 --> 00:31:54.680
And I suspect that also matters how many records are coming back.

00:31:54.680 --> 00:31:59.800
Yeah, this is true. That's why this affected this case for lazy parsing that was implemented for

00:31:59.800 --> 00:32:06.440
v1. And now, now it's not necessary for many cases, only for very extreme, extreme high load.

00:32:06.440 --> 00:32:07.640
That's really cool.

00:32:07.640 --> 00:32:15.000
What makes me smile from this is the more pydantic you use, the more awesome this upgrade to 2 becomes.

00:32:15.000 --> 00:32:17.320
And it's like I said, it's almost no work.

00:32:17.320 --> 00:32:20.120
Technically, I had to set all the optionals to be none.

00:32:20.120 --> 00:32:21.160
That's not a beanie thing.

00:32:21.160 --> 00:32:23.560
That's a pydantic thing, but it's not a big deal.

00:32:23.560 --> 00:32:28.680
So upgrading basically means all the parts of your frame, all the frameworks you use that are

00:32:28.680 --> 00:32:30.440
built on pydantic get faster.

00:32:30.440 --> 00:32:35.240
So for me, when I upgraded the website, it went about, I don't know, 40% faster or something like

00:32:35.240 --> 00:32:39.320
that, which is a huge speed up for very little work.

00:32:39.320 --> 00:32:40.520
It's already really fast, right?

00:32:40.520 --> 00:32:43.400
If you go to, you know, the podcast, you pull up an episode page.

00:32:43.400 --> 00:32:44.680
It's like 30 milliseconds.

00:32:44.680 --> 00:32:48.120
You go to the courses and you pull up a video to play.

00:32:48.120 --> 00:32:49.480
It's got many queries.

00:32:49.480 --> 00:32:51.800
It does, but it's probably 20 milliseconds.

00:32:51.800 --> 00:32:54.840
So now it's 15 milliseconds or 14 milliseconds.

00:32:54.840 --> 00:32:59.880
But still to get that much of a speed up and do basically no work on my part.

00:32:59.880 --> 00:33:00.520
That's awesome.

00:33:00.520 --> 00:33:06.440
And I'm not using a framework like FastAPI where the other side of that story is also pydantic.

00:33:06.440 --> 00:33:11.080
So if you're using FastAPI and beanie, which I think is probably a common combination,

00:33:11.080 --> 00:33:14.840
both the database side that gets you the pydantic things is faster.

00:33:14.840 --> 00:33:20.520
And then the outbound and inbound processing that the API itself does is a lot faster because

00:33:20.520 --> 00:33:21.400
of pydantic.

00:33:21.400 --> 00:33:26.680
And so you get this kind of multiplicative doubling of the speed on both ends, right?

00:33:26.680 --> 00:33:33.880
The numbers you just told about your website is like this time is only about this communication.

00:33:33.880 --> 00:33:38.120
I mean, how bits are going from one computer to another.

00:33:38.120 --> 00:33:39.800
There is no computation there probably.

00:33:40.680 --> 00:33:42.440
It's super, super impressive.

00:33:42.440 --> 00:33:45.640
You know, people say, well, Python is not fast enough.

00:33:45.640 --> 00:33:52.280
That may be true for a few very rare, you know, extremely high load situations, but I would bet

00:33:52.280 --> 00:33:54.280
it's fast enough for most people.

00:33:54.280 --> 00:34:00.360
You know, if your website end to end processing is responding in 15 milliseconds, like, you know,

00:34:00.360 --> 00:34:05.560
you've got other parts of your system like CDNs and an amount of JavaScript you send to worry about,

00:34:05.560 --> 00:34:07.480
not your, not your, pretty awesome.

00:34:07.960 --> 00:34:10.920
Because of indexes also not, don't forget your indexes.

00:34:10.920 --> 00:34:11.480
Yeah.

00:34:11.480 --> 00:34:17.160
Each time, each time when I have any requests about my things are slow and probably I have

00:34:17.160 --> 00:34:19.000
to switch Python to anything else.

00:34:19.000 --> 00:34:24.680
Usually the problem is about the data model, not about the processing because, you know, people could

00:34:24.680 --> 00:34:30.040
store things a bit, a bit wrongly, a bit too nested or less nested than it should be.

00:34:30.040 --> 00:34:31.640
And yeah, indexes.

00:34:31.640 --> 00:34:32.120
Okay.

00:34:32.120 --> 00:34:32.680
Yeah.

00:34:32.680 --> 00:34:33.160
Indexes.

00:34:33.160 --> 00:34:38.280
Or you've done a query where you return a hundred things of huge documents and you only want,

00:34:38.280 --> 00:34:42.040
like say the title and last updated, but you don't do a project projection.

00:34:42.040 --> 00:34:43.880
And so you're sending way too much data.

00:34:43.880 --> 00:34:47.960
It's like select star instead of select title, comma, date, you know, something like that.

00:34:47.960 --> 00:34:53.000
Sometimes, sometimes, sometimes also works to, to make protection on the database layer,

00:34:53.000 --> 00:34:58.920
like to, to make some, like to find maximum elements or minimum elements and to do this

00:34:58.920 --> 00:35:02.920
stuff on the database is better than in Python, because in Python, you have to iterate towards

00:35:02.920 --> 00:35:06.120
the objects to, to find things which is not that efficient.

00:35:06.120 --> 00:35:06.920
So we just have to,

00:35:06.920 --> 00:35:11.240
you got to pull them all back, deserialize them and validate them and then iterate over

00:35:11.240 --> 00:35:14.120
it rather than just let that happen on just the fields in the database.

00:35:14.120 --> 00:35:15.080
That's, that's a good point.

00:35:15.080 --> 00:35:16.680
And maybe the query is just bad as well.

00:35:16.680 --> 00:35:17.160
Okay.

00:35:17.160 --> 00:35:19.000
So what was your experience?

00:35:19.000 --> 00:35:23.880
You know, they, they shipped the migration guide about the things that you've got to do.

00:35:23.880 --> 00:35:28.040
And if you look at the scroll bar here, the migration guide is large.

00:35:28.040 --> 00:35:29.560
I don't know how many pages.

00:35:29.560 --> 00:35:33.000
Let's see if I press print, if it'll tell me how many pages.

00:35:33.000 --> 00:35:35.720
No, sadly, it doesn't want to tell me how many pages it would print,

00:35:35.720 --> 00:35:39.800
but there are many pages here, I guess, if I were to print this out,

00:35:39.800 --> 00:35:41.400
it was this a daunting experience.

00:35:41.400 --> 00:35:42.440
How did it go for you?

00:35:42.440 --> 00:35:43.320
It was very.

00:35:43.320 --> 00:35:47.320
So when I just, we should, we can install a potential tool.

00:35:47.320 --> 00:35:48.840
Or you're like, let's see if it just works.

00:35:48.840 --> 00:35:49.240
Come on.

00:35:49.240 --> 00:35:52.840
And just run tests and everything was.

00:35:52.840 --> 00:35:56.280
Firstly, it was like only one error.

00:35:56.280 --> 00:35:57.000
Nothing can work.

00:35:57.000 --> 00:35:59.640
I don't run any tests.

00:35:59.640 --> 00:36:00.360
So one here.

00:36:00.360 --> 00:36:01.800
Yeah.

00:36:01.800 --> 00:36:05.480
I handled this and each test was read and everything.

00:36:05.480 --> 00:36:07.880
It was really interesting and challenging.

00:36:07.880 --> 00:36:12.520
Interesting because it's kind of really computer science problem sometimes there.

00:36:12.520 --> 00:36:21.560
So Pydantic moved a lot of logic to the Rust layer and it got hidden for me as a user of Pydantic.

00:36:21.560 --> 00:36:26.200
For example, there is in Python, there is a thing called forward ref.

00:36:26.200 --> 00:36:27.560
What is it?

00:36:27.560 --> 00:36:33.960
And even you have two classes in a single module, for example, and in one class you have field of the second class.

00:36:33.960 --> 00:36:37.320
And in the second class, you have field of type of the first class.

00:36:37.320 --> 00:36:40.360
You cannot just put these names without any magic.

00:36:40.360 --> 00:36:46.840
You can make input from annotation or you can just write it as a string instead of the class itself.

00:36:46.840 --> 00:36:47.240
I see.

00:36:47.240 --> 00:36:52.600
But then Python will understand it as a forward ref, forward reference for another class.

00:36:52.600 --> 00:36:59.640
And Pydantic can resolve it and make it an actual class finally in the base model.

00:36:59.640 --> 00:37:06.200
And in V1, this mechanism was implemented in Python and I could use it and I could use the results of this.

00:37:06.680 --> 00:37:09.960
While in Pydantic V2, everything is in the Rust layer.

00:37:09.960 --> 00:37:16.120
And when it updates this reference, it updates inside of the Rust and I cannot see the actual class finally.

00:37:16.120 --> 00:37:18.360
And I had to implement my own resolver there.

00:37:18.360 --> 00:37:20.760
And there are a few stuff like this.

00:37:20.760 --> 00:37:23.800
Another one big change was about...

00:37:23.800 --> 00:37:25.000
That does not sound easy.

00:37:25.000 --> 00:37:26.040
Yeah.

00:37:26.040 --> 00:37:28.120
I was like, so will I?

00:37:28.120 --> 00:37:28.760
Oh no.

00:37:28.760 --> 00:37:29.960
Am I going to do this?

00:37:29.960 --> 00:37:31.640
Yes, exactly.

00:37:31.640 --> 00:37:32.600
Exactly.

00:37:32.600 --> 00:37:36.600
And, you know, for people who maybe haven't played with this, it's really important.

00:37:37.080 --> 00:37:42.440
Because Pydantic and hence Beanie depends on the types, right?

00:37:42.440 --> 00:37:48.600
The parsing and the validation and all of those things, you have to know exactly what type a thing is, right?

00:37:48.600 --> 00:37:53.800
And so if you lose access to the forward reference resolution, that's going to be bad.

00:37:53.800 --> 00:37:54.120
Yeah.

00:37:54.120 --> 00:37:54.280
Yeah.

00:37:54.280 --> 00:38:01.640
For example, in links or in backward links, if you don't clearly understand which document is linked to this document,

00:38:01.640 --> 00:38:03.640
you cannot build a query for this.

00:38:03.640 --> 00:38:04.040
So yeah.

00:38:04.040 --> 00:38:05.560
And I have made this resolver.

00:38:05.560 --> 00:38:09.720
Honestly, I just was reading Pydantic with one code and was...

00:38:09.720 --> 00:38:10.120
Hold on.

00:38:10.120 --> 00:38:14.360
I didn't copy paste, but I was using nearly the same algorithm.

00:38:14.360 --> 00:38:20.680
And another big feature was about how validation of the custom types happens.

00:38:20.680 --> 00:38:21.640
Yeah.

00:38:21.640 --> 00:38:23.400
And get Pydantic core schema.

00:38:23.400 --> 00:38:26.760
So everything is in the schema inside of the Rust layer now.

00:38:26.760 --> 00:38:35.960
And you can write instructions in the Python, which would be important to this Rust layer and would be called from there.

00:38:35.960 --> 00:38:40.120
That's why the whole syntax of this completely changed.

00:38:40.120 --> 00:38:40.680
Completely.

00:38:40.680 --> 00:38:49.480
And that's why I was thinking like, but so how can I have two completely different syntaxes for the same thing inside of the same class?

00:38:49.480 --> 00:39:02.280
And I was thinking like, maybe I have to switch branches now and have Beanie V2 after Pydantic V2 and Beanie V1 and support all the new features in both versions.

00:39:02.280 --> 00:39:04.520
And it was, oh no, it will be a nightmare.

00:39:04.520 --> 00:39:06.120
You're already busy with one project.

00:39:06.120 --> 00:39:06.600
Do you need...

00:39:06.600 --> 00:39:07.960
Because I already have Bonnet.

00:39:07.960 --> 00:39:08.200
Yeah.

00:39:08.200 --> 00:39:09.240
Do you want two projects?

00:39:09.240 --> 00:39:09.560
Yeah.

00:39:09.560 --> 00:39:15.080
And I already have Bonnet, which is a synchronous version of Beanie, which also would be split into two then.

00:39:15.080 --> 00:39:19.480
And I will have four projects, which is too crazy.

00:39:19.480 --> 00:39:20.600
Yeah.

00:39:20.600 --> 00:39:21.320
No kidding.

00:39:21.320 --> 00:39:22.520
I can split myself.

00:39:22.520 --> 00:39:23.400
This is a problem.

00:39:23.400 --> 00:39:26.520
And yeah, finally, finally, finally, I found...

00:39:26.520 --> 00:39:31.880
I honestly, I just went to FastAPI code and I was reading how they deal with this.

00:39:31.880 --> 00:39:33.960
And like, nice idea.

00:39:33.960 --> 00:39:34.840
I will do the same.

00:39:34.840 --> 00:39:35.640
Thank you guys.

00:39:35.640 --> 00:39:38.120
And that's the power of open source.

00:39:38.120 --> 00:39:40.680
Yeah, it is.

00:39:40.680 --> 00:39:45.960
I feel like Sebastian and FastAPI are kind of examples for a lot of different projects and different people.

00:39:45.960 --> 00:39:49.080
You know, people look to them for kind of an example.

00:39:49.080 --> 00:39:49.880
Yeah, this is true.

00:39:49.880 --> 00:39:53.320
And so, and finally, finally, I solved all the problems.

00:39:53.320 --> 00:39:55.480
It took me to solve all the problems.

00:39:55.480 --> 00:39:58.120
It took me like one, maybe one and a half weeks.

00:39:58.120 --> 00:40:03.800
But then I published a better version and there were performance problems with better version.

00:40:03.800 --> 00:40:09.240
There were some corner cases that I didn't catch and community found them.

00:40:09.240 --> 00:40:14.680
And this is great also about open sourcing, because honestly, I would not be able to find

00:40:14.680 --> 00:40:17.720
all of these problems myself for this short period of time.

00:40:17.720 --> 00:40:18.200
Definitely.

00:40:18.200 --> 00:40:23.320
I guess how much of this is at runtime and how much of it, I guess it's all at runtime,

00:40:23.320 --> 00:40:27.240
but how much of this is startup versus kind of once the app is up and running.

00:40:27.240 --> 00:40:34.200
So, you know, you've got your beanie init code where you pass the motor async client over and you pass the models.

00:40:34.200 --> 00:40:37.640
And it's got to like verify that they all click together correctly.

00:40:37.640 --> 00:40:45.080
You know, they all have the settings that say, you know, what database they go to and the indexes are set up correctly and whatever else is in there.

00:40:45.080 --> 00:40:50.200
And then once I imagine once that gets processed, it kind of just knows it and runs.

00:40:50.200 --> 00:40:59.240
So how much of this stuff that you're talking about was kind of the setup, get things working and how much of it is happening on every query, every insert?

00:40:59.240 --> 00:41:04.360
Most of this was about runtime of about every query and every, not even this way.

00:41:04.360 --> 00:41:05.480
Not even this way.

00:41:05.480 --> 00:41:08.920
Like how do I set up the document itself?

00:41:08.920 --> 00:41:10.920
How do I set up validators?

00:41:10.920 --> 00:41:17.720
Because I also have validators in the documents to make things simpler and this index changed and et cetera.

00:41:17.720 --> 00:41:25.640
So yeah, there are, for example, the config, there was config nested class called config before in Pedantic V1.

00:41:25.640 --> 00:41:32.680
And now this is a field called model config, which is, you see, it's completely different interface again, because this is not a class.

00:41:32.680 --> 00:41:33.720
This is just a field now.

00:41:33.720 --> 00:41:37.640
And I had to, and I was using and still use this config stuff.

00:41:37.640 --> 00:41:43.560
And I had to not only switch to the new syntax, but support config class if you use Pedantic V1.

00:41:43.560 --> 00:41:51.720
And that's why inside of my classes, I have conditions like which field I want to define based on the version of Pedantic.

00:41:51.720 --> 00:42:03.160
And yeah, most of the, most of the changes were about, about this, the tabs of the documents and the tabs of the types, but you use it not on the initialization layer, but on the runtime.

00:42:03.160 --> 00:42:03.640
Yeah.

00:42:03.640 --> 00:42:04.840
All these things.

00:42:04.840 --> 00:42:07.880
You also talked about the performance story.

00:42:07.880 --> 00:42:12.600
Do you do any profiling or have any tools like that?

00:42:12.600 --> 00:42:13.640
Honestly, I didn't.

00:42:13.640 --> 00:42:19.080
So there are P profile and, and other tools to do this stuff.

00:42:19.080 --> 00:42:29.400
I was measuring using just time, start, time, end, but I was doing this for, for different parts of the code, just to see what's happening there and here.

00:42:29.400 --> 00:42:33.640
And honestly, it was when I faced a performance problem.

00:42:33.640 --> 00:42:41.240
And this was because there are some methods in Pedantic V2 that they keep, but marked as deprecated.

00:42:41.240 --> 00:42:42.760
They keep from, from V1.

00:42:42.760 --> 00:42:46.120
In some cases, I just didn't switch it to the, to the new versions.

00:42:46.120 --> 00:42:48.920
And this was the performance problem.

00:42:48.920 --> 00:42:53.880
And when I found all the places where I use deprecated methods, then everything, first of all, then.

00:42:53.880 --> 00:42:56.840
You switch it to the new, more intended one.

00:42:56.840 --> 00:43:00.360
And that's got the, you know, fully optimized version or something like that.

00:43:00.360 --> 00:43:00.760
Yeah.

00:43:00.760 --> 00:43:01.000
Yeah.

00:43:01.000 --> 00:43:01.240
Yeah.

00:43:01.240 --> 00:43:01.400
True.

00:43:01.400 --> 00:43:07.960
Because, because Bini is, is working with a lot of internal things of Pedantic and it uses this very heavily.

00:43:08.360 --> 00:43:18.120
And, sometimes I just don't know that in this, I mean, don't remember that in this, specific part, there is something internal for Pedantic and, I use this also.

00:43:18.120 --> 00:43:22.440
And, so I had to check everything and Pedantic code basis is big already.

00:43:22.440 --> 00:43:24.280
So it's hard to keep everything in mind.

00:43:24.280 --> 00:43:24.520
Yeah.

00:43:24.520 --> 00:43:29.160
I feel like that's the challenge for people like you and FastAPI and others.

00:43:29.160 --> 00:43:37.960
You have your, your code way deeper in the internals of Pedantic than people who just consume FastAPI or consume Beanie like I do.

00:43:37.960 --> 00:43:39.160
This is very interesting.

00:43:39.160 --> 00:43:41.480
Like, I mean, this is true computer science problem.

00:43:41.480 --> 00:43:49.080
Like when you have to swap interfaces and, and you don't even know where all these interfaces are used and you have to detect them.

00:43:49.080 --> 00:43:50.360
So yeah, this is nice.

00:43:50.360 --> 00:43:51.400
It's super interesting.

00:43:51.400 --> 00:43:51.640
Yeah.

00:43:51.640 --> 00:44:01.320
And again, on your, your side, you've got to do that to adapt to the new Pedantic, but you've also got to present some kind of consistent forward looking view to people consuming Beanie.

00:44:01.320 --> 00:44:03.640
So they don't have to rewrite all their code too.

00:44:03.640 --> 00:44:03.800
Right?

00:44:03.800 --> 00:44:04.040
Yeah.

00:44:04.040 --> 00:44:04.360
True.

00:44:04.360 --> 00:44:07.880
So, and, and all the interfaces, all the,

00:44:07.880 --> 00:44:13.160
I didn't change any Beanie interface when I was working with this, like all the interfaces,

00:44:13.160 --> 00:44:18.840
still the same for Beanie and, nobody should change their code, which uses Beanie.

00:44:18.840 --> 00:44:25.320
I implemented kind of middleware between Pedantic and Beanie and this middleware, have static interfaces.

00:44:25.320 --> 00:44:34.600
And inside of this, there are these conditions like if Pedantic V2, then, and, and yeah, this kind of logic.

00:44:34.600 --> 00:44:35.240
Nice.

00:44:35.240 --> 00:44:37.800
It's pretty good question on the audience from Marwan here.

00:44:37.800 --> 00:44:46.600
It says, other than getting the code to execute correctly, were there any gnarly parts you had to figure out to appease type checkers and linters and that kind of stuff?

00:44:46.600 --> 00:44:53.480
I don't, remember any changes about this time, about these things like, mypy and, rough.

00:44:53.480 --> 00:44:56.840
I use rough for instead of flake eight currently.

00:44:56.840 --> 00:44:57.160
Mm-hmm.

00:44:57.160 --> 00:44:58.280
I didn't fix this stuff.

00:44:58.280 --> 00:45:00.120
everything was okay.

00:45:00.120 --> 00:45:01.320
It was, it was new Pedantic.

00:45:01.320 --> 00:45:12.040
I think, I think, guys made a really great job about this to, to make everything work because yeah, I have, I have other checks about my pipe and about rough and everything went smoothly about this.

00:45:12.040 --> 00:45:23.240
So yeah, this is, I can't imagine being Samuel and team to have to rewrite Pedantic with such a major refactoring realizing quarter million other projects depend on this.

00:45:23.240 --> 00:45:29.960
And then people depend on those projects, you know, and like, how are we going to do this without the world just completely breaking, you know?

00:45:29.960 --> 00:45:33.480
Currently they have a discord channel also for Pedantic.

00:45:33.480 --> 00:45:51.080
And I see, I see many people asking for, for help for, and, some questions about new, new syntax, but I see that it could be much more because so current, current syntax is, is very similar to, to the previous syntax, but the change it's completely rewritten.

00:45:51.080 --> 00:45:51.480
Right.

00:45:51.480 --> 00:45:51.960
Oh yeah.

00:45:51.960 --> 00:45:53.560
This is, this is very impressive work.

00:45:53.560 --> 00:45:53.720
Yeah.

00:45:53.720 --> 00:45:56.520
Completely rewritten a good part of it in another language.

00:45:56.520 --> 00:45:57.080
Yeah.

00:45:57.080 --> 00:45:57.520
Yeah.

00:45:57.520 --> 00:46:01.080
Broken into multiple modules and still it seems to go pretty well.

00:46:01.080 --> 00:46:01.480
Yeah.

00:46:01.480 --> 00:46:06.680
And the assumptions about the default values for, for optionals, that was the only thing that seemed to have caught me out.

00:46:06.680 --> 00:46:17.800
You can put in, in, in the configs that, you, you can have options, like the same, use the same logic as for V1, but you just have to mention it in the, in the config now.

00:46:17.800 --> 00:46:18.760
But I don't remember.

00:46:18.760 --> 00:46:23.960
Honestly, there was a few other things that maybe I ran across, like, and honestly, I don't know why some of these changed.

00:46:23.960 --> 00:46:26.920
So it used to be able to call dot JSON on an object.

00:46:26.920 --> 00:46:33.400
And now it's model underscore dump underscore JSON or there was dict model dump, right?

00:46:33.400 --> 00:46:36.600
Everything is, has prefix model currently in Pydantic.

00:46:36.600 --> 00:46:46.680
And I believe this is to not make conflicts with the user logic, because when you, when you have your own, suffix or prefix, then yeah.

00:46:46.680 --> 00:46:50.440
Then it's simpler for, for users to, to have their own methods.

00:46:50.440 --> 00:46:51.800
And I like this solution.

00:46:51.800 --> 00:46:52.920
This, this is really nice.

00:46:52.920 --> 00:46:57.320
I guess this is the other thing that I ran into is I have a few places I was calling dot dict, I think.

00:46:57.320 --> 00:46:58.120
For me as well.

00:46:58.120 --> 00:46:59.640
And I have like conditions.

00:46:59.640 --> 00:47:00.280
I see.

00:47:00.280 --> 00:47:07.320
Anything else you want to talk about, about the migration, like what's, go okay for you or any other thing you want to highlight about how it went?

00:47:07.320 --> 00:47:14.520
Honestly, I think I mentioned everything and yeah, it, it, it, it, it's when, it's when very, it's mostly thanks to, to, to Podentic team.

00:47:14.520 --> 00:47:17.800
And, yeah, I really, I really like, Podentic V2.

00:47:17.800 --> 00:47:27.080
I still really like Podentic V1 actually, like both, but this completely, this completely different libraries inside and very similar,

00:47:27.080 --> 00:47:28.840
libraries, I regard the interfaces.

00:47:28.840 --> 00:47:30.840
So I really like how, how it's.

00:47:30.840 --> 00:47:31.000
Yeah.

00:47:31.000 --> 00:47:36.520
It's not too often that you get that massive of a speed up of your code and you didn't have to do anything.

00:47:36.520 --> 00:47:36.760
Right.

00:47:36.760 --> 00:47:41.960
You just, I was using this library and it was really right in the core of all the processing and now it's a lot faster.

00:47:41.960 --> 00:47:43.160
So, so it was my code.

00:47:43.160 --> 00:47:48.360
When you read the code, the Rust part of the, of Podentic, I started to read it just to learn.

00:47:48.360 --> 00:47:53.320
And, this is really nice how it can interact with Python parts.

00:47:53.320 --> 00:47:56.840
I mean, I can, I can write Rust itself, just logic.

00:47:56.840 --> 00:48:03.960
I even implemented database in the Rust, but how to do all this Python stuff inside of another language.

00:48:03.960 --> 00:48:05.320
So this is super impressive.

00:48:05.320 --> 00:48:06.520
It is super impressive.

00:48:06.520 --> 00:48:07.800
We'll get a little short on time here.

00:48:07.800 --> 00:48:10.760
Let's see what, I guess I'll give a shout out to this course.

00:48:10.760 --> 00:48:17.880
I recently released MongoDB with async Python, which of course uses Beanie as well.

00:48:17.880 --> 00:48:27.560
And I created this course on Beanie one dot, something prior to the one dot 21 that switches to Pydantic two and it's all Pydantic one.

00:48:27.560 --> 00:48:31.480
And so I was always wondering like, well, how, you know, how tricky is it going to be to upgrade?

00:48:31.480 --> 00:48:34.520
And there was really either zero or very little changes.

00:48:34.520 --> 00:48:39.240
I think maybe the default values on optionals was also a thing I had to adjust on there.

00:48:39.240 --> 00:48:47.480
But if people want to learn all the stuff that we're talking about, you know, just talk Python.fm, go to courses, check out the MongoDB async course.

00:48:47.480 --> 00:48:49.720
It's all about Beanie and stuff like that.

00:48:49.720 --> 00:48:52.680
I was reading this course also and I can highly recommend it.

00:48:52.680 --> 00:48:53.240
They are.

00:48:53.240 --> 00:48:54.120
Thank you so much.

00:48:54.120 --> 00:48:57.320
One thing I do want to give a shout out to is I use Locus.

00:48:57.320 --> 00:48:58.680
Are you familiar with the Locus?

00:48:58.680 --> 00:48:59.400
Locus IO?

00:48:59.400 --> 00:48:59.720
Yeah.

00:48:59.720 --> 00:48:59.880
Yeah.

00:48:59.880 --> 00:49:01.480
What a cool, what a cool project.

00:49:01.480 --> 00:49:07.240
So this lets you do really nice modeling of how users interact with your site using Python.

00:49:07.240 --> 00:49:13.320
And then what you get is, I don't know if there's any cool pictures that show up on here in terms of the graphs,

00:49:13.320 --> 00:49:19.560
but you get really nice graphs that show you real time about how, you know, how many requests per second and different scenarios you get.

00:49:19.560 --> 00:49:25.960
And on this one, I'm pretty sure when I upgraded it to Pydantic 2 and ran it again,

00:49:25.960 --> 00:49:30.760
trying to think of all the variations that you knew, there could be something that has changed that I wasn't aware of.

00:49:30.760 --> 00:49:36.280
Like maybe, maybe I recorded on my M1 Mac mini and then ran it again on my M2 Pro Mac mini.

00:49:36.280 --> 00:49:39.640
So that could affect it by like a little bit as well, like 20%.

00:49:39.640 --> 00:49:49.400
But I think just so using Beanie and FastAPI and upgrading all those, those paths to Pydantic 2 and the respective Beanie and

00:49:49.400 --> 00:49:50.360
FastAPI versions.

00:49:50.360 --> 00:49:56.600
I think it went 50% or double fast, two times as fast, 100% faster just by making that change.

00:49:56.600 --> 00:49:57.800
So that was pretty awesome.

00:49:57.800 --> 00:49:59.000
This is great.

00:49:59.000 --> 00:49:59.000
Yeah.

00:49:59.000 --> 00:49:59.560
Yeah.

00:49:59.560 --> 00:50:02.200
I see, I see, I see this 50% also.

00:50:02.200 --> 00:50:09.000
And yeah, all I did is I just reran pip-tools to get the new versions of everything and reran the load

00:50:09.000 --> 00:50:11.640
tests and look how much faster they are.

00:50:11.640 --> 00:50:14.360
And so that's a real cool example of kind of what I was talking about.

00:50:14.360 --> 00:50:19.720
So yeah, if you want to see all this stuff in action, check out the MongoDB with async Python course.

00:50:19.720 --> 00:50:25.560
Thanks for coming on the show and updating us on Beanie and especially giving us this look into your

00:50:25.560 --> 00:50:28.840
journey of migrating based on Pydantic 1 to 2.

00:50:28.840 --> 00:50:29.320
I think that's really cool.

00:50:29.320 --> 00:50:29.720
Yeah.

00:50:29.720 --> 00:50:30.440
Thank you very much.

00:50:30.440 --> 00:50:31.640
Thank you very much for having me here.

00:50:31.640 --> 00:50:32.280
Of course.

00:50:32.280 --> 00:50:38.760
So before you get out of here, I got a PIPI project library to recommend to people something

00:50:38.760 --> 00:50:42.200
besides of course Beanie and Pydantic, which are pretty awesome and obvious.

00:50:42.200 --> 00:50:47.400
I would recommend to use motor. This is this is kind of by Mongo, but asynchroneous.

00:50:47.400 --> 00:50:51.320
Integrates with async and await perfectly, which is real, real nice.

00:50:51.320 --> 00:50:57.640
If you want to do something more low levels and then Beanie, then you have to at least meet with

00:50:57.640 --> 00:51:04.440
motor because this is really nice library. And even after that many years, it's still very actual.

00:51:04.440 --> 00:51:06.920
And what else? Honestly, I don't have anything.

00:51:06.920 --> 00:51:11.560
Yeah. Pydantic is good stuff. Awesome.

00:51:11.560 --> 00:51:15.320
Okay. I'll throw Locus out there for people. They can check out Locus. That's pretty cool.

00:51:15.320 --> 00:51:21.640
If you are going through the same process, you've got code built on Beanie or just Pydantic in general,

00:51:21.640 --> 00:51:26.440
and you want to see, you know, how does my system respond before and after?

00:51:26.440 --> 00:51:31.640
Locus is like ridiculously easy to set up, run it against your code, pip install, upgrade,

00:51:31.640 --> 00:51:36.200
run it again and just see what happens. I think that'll be a really good recommendation too.

00:51:36.200 --> 00:51:40.920
Yeah. Final call to action. People want to get started with Beanie. Maybe people out there already

00:51:40.920 --> 00:51:42.680
using Beanie want to upgrade their code. What do you tell them?

00:51:42.680 --> 00:51:48.680
Just pip install it. Everything would be, would work fine. So just try it. But at least you have to

00:51:48.680 --> 00:51:51.000
upgrade everything. So please write tests.

00:51:51.000 --> 00:51:51.000
Absolutely.

00:51:51.000 --> 00:51:59.080
And if something will go wrong, go to my Discord channel and me or other people will answer your

00:51:59.080 --> 00:52:04.280
questions. Sounds good. All right. Well, congrats on upgrading Beanie. You must be really happy to have

00:52:04.280 --> 00:52:05.720
it done. Yeah, I think. Thank you very much.

00:52:05.720 --> 00:52:07.480
Yeah, you bet. See you later. See you.

00:52:07.480 --> 00:52:14.760
This has been another episode of Talk Python To Me. Thank you to our sponsors. Be sure to check out

00:52:14.760 --> 00:52:19.800
what they're offering. It really helps support the show. Studio 3T is the IDE that gives you full

00:52:19.800 --> 00:52:25.880
visual control of your MongoDB data. With a drag and drop visual query builder and multi-language query

00:52:25.880 --> 00:52:31.960
code generator, new users will find they're up to speed in no time. You can even query MongoDB directly with

00:52:31.960 --> 00:52:39.160
SQL and migrate tabular data easily from relational DBs into MongoDB documents. Try Studio 3T for free

00:52:39.160 --> 00:52:46.840
at talkpython.fm/studio2023. Want to level up your Python? We have one of the largest catalogs of

00:52:46.840 --> 00:52:52.440
Python video courses over at Talk Python. Our content ranges from true beginners to deeply advanced topics

00:52:52.440 --> 00:52:57.640
like memory and async. And best of all, there's not a subscription in sight. Check it out for yourself

00:52:57.640 --> 00:53:03.560
at training.talkpython.fm. Be sure to subscribe to the show. Open your favorite podcast app and search

00:53:03.560 --> 00:53:09.320
for Python. We should be right at the top. You can also find the iTunes feed at /itunes, the Google Play

00:53:09.320 --> 00:53:17.160
feed at /play, and the direct RSS feed at /rss on talkpython.fm. We're live streaming most of our

00:53:17.160 --> 00:53:21.720
recordings these days. If you want to be part of the show and have your comments featured on the air,

00:53:21.720 --> 00:53:27.320
be sure to subscribe to our YouTube channel at talkpython.fm/youtube. This is your host,

00:53:27.320 --> 00:53:31.400
Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write

00:53:31.400 --> 00:53:41.080
some Python code.

00:53:41.080 --> 00:53:52.760
"The RSS feed on YouTube and check out the video. I'll see you next time. I'll see you next time. I'll see you next time."

00:53:52.760 --> 00:53:52.760
"The RSS feed on YouTube and I'll see you next time."

00:53:52.760 --> 00:53:53.160
"The RSS feed on YouTube and I'll see you next time."