#271: Unlock the mysteries of time, Python's datetime that is! Transcript
00:00 Time is a simple thing, right?
00:01 And working with it in Python is great.
00:04 You just import date time and, somewhat oddly, use the date time class from that module.
00:09 Oh, except there are times with time zones and times without time zones.
00:14 And why is there a total seconds, but no total minutes, total hours, or total days on time delta?
00:20 How about computing the number of weeks in a time span?
00:23 What if you wanted to iterate over the next 21 workdays, skipping weekends?
00:28 All right, we'd better talk about time in Python.
00:31 Good thing Paul Gansel is here.
00:33 He's a core developer who controls time in CPython.
00:37 This is Talk Python to Me, episode 271, recorded June 18th, 2020.
00:42 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem,
01:00 and the personalities.
01:01 This is your host, Michael Kennedy.
01:03 Follow me on Twitter where I'm @mkennedy.
01:06 Keep up with the show and listen to past episodes at talkpython.fm.
01:09 And follow the show on Twitter via at Talk Python.
01:12 This episode is brought to you by Brilliant.org and us with our online courses over at Talk Python Training.
01:19 Speaking of Talk Python Training, have you been thinking about taking one of our courses?
01:23 We're participating in the latest Humble Bundle deal for Python developers along with a bunch of other great educators and tool developers.
01:30 Until July 22nd, you can get $1,400 worth of Python goodies, including three of our popular courses for just $25.
01:41 Yeah, Humble Bundles are crazy.
01:43 That's $25 for our three courses and all those other things combined.
01:47 Just visit talkpython.fm/humble2020.
01:51 That's talkpython.fm/humble2020 altogether before July 22nd to take advantage of this offer.
01:58 Now, let's get to that interview.
02:00 Paul, welcome to Talk Python to me.
02:02 Glad to be here.
02:02 Yeah, it's great to have you here.
02:03 Really excited to talk about the meaning of time and all of that.
02:08 Time is really tricky, actually.
02:10 Time is one of those things that seems really obvious and simple until it's not.
02:15 And then until you start working with it and you're like, oh, actually that code's running on the server and that server we forgot,
02:20 you know, we didn't set the time zone to actually where it is.
02:23 So maybe it's UTC.
02:24 Maybe it's where the server's located or where our office is.
02:28 And all of a sudden you find yourself juggling all these things.
02:31 And especially if you're building anything that has to do with calendars or stuff like that,
02:35 that's going to be a real adventure.
02:37 Yeah, well, I will say that I did not set out to become like the date time guy,
02:41 but I think that kind of complexity, like the natural complexity of the subject has,
02:46 it's either drawn me in or what's the word for when something gets you stuck,
02:50 like in a Kragmire or like a Venus flytrap or something.
02:53 It's enticed you in and now it's got you trapped in its world.
02:57 Yeah, it certainly seems like a one-way function, like something barbed where you can move in one direction,
03:02 but not in the other direction.
03:03 Yeah, for sure.
03:04 Well, you know, it's really great that you're working on it.
03:06 And just for the listeners, so you're a Python core developer working on basically
03:10 times, date times, time spans, that kind of stuff in Python.
03:14 Yeah, I think I originally got into it with working on DateUtil.
03:16 So I've been maintaining that for about, oh God, now it's like six years.
03:21 And, you know, DateUtil has a lot of little minor extensions.
03:25 And then when PEP495 came around, the core developers showed up in DateUtil and said,
03:31 hey, we kind of used your library as a prototype for how this could work.
03:36 So maybe you want to actually implement PEP495.
03:39 And then I did that.
03:40 And then I had a relationship with the core developers.
03:43 And then I just kind of stuck around and started doing more stuff.
03:47 And eventually it's one of those things like you never set out to do it.
03:51 And I don't even have a job that's particularly date time heavy.
03:55 It's just like, if you start monitoring the issue tracker for something like DateUtil or
03:59 even CPython, and then specifically just focusing on any one thing, like you'll just learn a stupidly
04:05 large amount about edge cases and how everything works.
04:08 I can see how that happens.
04:09 And it's definitely one of those things that pulls you in.
04:11 Before we dive too much into all that, though, let's start with your story.
04:15 How'd you get into programming in Python?
04:16 I knew this question was coming and I was sort of dreading trying to answer it because
04:20 I feel like every time someone asks me how I started programming, I give them a slightly
04:24 different answer because I've had quite a bit of false starts.
04:28 Like I had when I was a kid, they taught us basic and I sort of remembered enough of that.
04:33 And then so I didn't go to high school.
04:35 But when I was sort of in place of high school, going to like a homeschooling center every once
04:41 in a while, I guess I shouldn't say every once in a while, I was regularly going to a
04:45 homeschooling center.
04:46 You know, I started learning C++.
04:48 And then weirdly, my friend became close friends with someone who was what they call
04:53 the wizard on a moo, which is a mud, but object oriented.
04:57 And mud stands for multi-user dungeon, I think.
05:01 You know, I don't know if the young people listening these days will know what a mud was,
05:05 but man, muds were super fun back in the early days.
05:09 Yeah.
05:09 See if you can describe a mud to people listening who have never played it.
05:13 Yeah.
05:13 So a mud is a text-based game that you would connect to via like Telnet or something.
05:18 It's kind of like Zork or some sort of, if you don't know what a mud is, you probably don't
05:22 know what Zork is either.
05:23 But you know, like it's a text-based game where you describe, it'll say, you know, like, oh,
05:27 you're in a room, you can go north or south and you'll go north, you know, you'll pick
05:31 up an object or something.
05:32 So it's like kind of a fantasy game or a Dungeons and Dragons type thing where it's all text-based.
05:37 And then you can also talk to other people there.
05:39 Yeah.
05:39 And there's usually, potentially hundreds of people all running around.
05:42 They might be in these rooms and you can have battles, you can go on quests.
05:46 One of the hallmarks, I think, is it's not super obvious what you can do.
05:50 It's not like, well, here are my five commands.
05:52 Like you kind of have to interrogate the game with words to figure out like, oh, there's actually
05:57 a rock and I could move the rock.
05:58 I didn't know that.
05:59 But now I, you know, that kind of stuff.
06:00 What I liked about it was it was, you know, as great as graphics are on games these days,
06:06 it really just leveraged your imagination.
06:08 And yet there were other people there and stuff.
06:10 I thought it was pretty cool.
06:11 Yeah.
06:11 Well, so I actually got into that stuff like after the heyday of MUDs, like when there was
06:17 like not that many people in there.
06:19 It was just my friend had this, like was running, it was basically a chat server that you could
06:24 just build whatever room you wanted.
06:26 And then like you could just directly program stuff.
06:28 And it was interesting because when you would actually play MUD games in say the early 2000s
06:35 or something, a decent fraction of the people or like a much higher fraction of the people
06:39 than you would think, than you would meet in your daily life were blind because that was
06:44 like the game was all textual.
06:45 And so it was one of those things where, you know, you're not really going to appreciate
06:50 World of Warcraft or something, but you could go on a MUD or something and then it'll describe,
06:55 it just describes everything in the room and all that kind of stuff.
06:58 So it was really interesting to see when like a decent fraction of the users are, you know,
07:04 just interacting with computers in a completely different way.
07:06 You see some interesting stuff.
07:07 If I think about this too much, I'm going to stop being the daytime guy and I'm going to
07:11 just take up a second career as the accessibility guy.
07:13 Yeah.
07:13 Yeah.
07:13 Yeah.
07:14 For sure.
07:14 Well, it is super interesting.
07:15 I hadn't really put that together, but of course, because it's pure text, you can have
07:19 a screen reader.
07:20 And like, once you have the screen reader, it's pretty much on par with everyone else's
07:23 experience.
07:24 That's super cool.
07:25 A cool bit of nostalgia there.
07:27 So you got into this Moo somehow that led you into programming.
07:31 Yeah.
07:31 So it was a programmable Moo.
07:32 So that was like, you could do object-oriented programming.
07:35 So I was just doing a little Moo scripting stuff.
07:39 And then later I took up Tickle TK.
07:41 I had this job that I had these very long shifts and nothing to do.
07:46 It was just kind of sitting around doing nothing.
07:49 And I could play on the computer.
07:50 So I bought a laptop and just programmed a notepad clone.
07:53 Then I went off to grad school, did a lot of MATLAB and some C programming.
07:57 And then at some point I was on some forum somewhere trying to make a point about something.
08:03 And I made a chart in MATLAB and I posted it.
08:08 And MATLAB has like terrible anti-aliasing, or at least it did at the time.
08:13 And someone was like, oh, what is this alias nonsense?
08:15 Like you should use MATplotlib instead.
08:18 So, and my friend had told me the one, the same one who had the Moo.
08:22 He had told me about this programming language called Python that was like interesting and,
08:27 you know, like natural language text or something.
08:29 Fewer curly braces.
08:30 Yeah.
08:31 So I just like, I was like, all right, I'll give it a try.
08:33 And then I really, really liked MATplotlib.
08:36 I rewrote my whole thesis or I guess I just wrote my whole thesis, made all my charts in
08:41 in MATplotlib.
08:42 And then when I went and graduated and got a job, there was this one day when all the MATLAB
08:47 licenses were taken on the, like you had a certain number of site licenses.
08:52 And if you ran multiple processes, it would take up two licenses.
08:56 And I just couldn't do any work that day.
08:58 So I was like, how about this?
09:00 I'll just rewrite all of our MATLAB scripts in Python.
09:03 And so then at work, I got to use Python.
09:06 And then, you know, I started doing open source and I don't think I've used MATLAB since I
09:11 left that job.
09:12 Isn't it interesting how these draconian licensing systems, like these licensed servers and other
09:18 super intense things that are like just there to punish you, even if you follow the rules,
09:24 you know, it's just, it's like in their world, the worst possible scenario is that in some edge
09:31 case, some user gets to run 11 copies instead of 10 when they only paid for 10.
09:37 Whereas, you know, they've lost all the goodwill and they've actually pushed you to just, you
09:41 know what, we're just going to go do something that doesn't work like this at all.
09:45 Those are really interesting situations.
09:47 That's one of the reasons I really like open culture is it just cuts down on friction so much.
09:52 Like there are just so many times when it's like, oh, if I get this image from Unsplash instead
09:56 of iStockphoto or whatever, it's just, I don't have to think about it.
10:00 I can use it however.
10:01 And that's why I tend to create a lot of open artifacts.
10:05 You know, I, my blog is all CC zero.
10:07 I'm just really trying to cut down on the friction of letting people use my work.
10:11 But, you know, I understand like my business model for myself is to sell my labor day to
10:18 day to big companies.
10:19 I understand how it would, you know, if you are a company, you may want to sell software.
10:24 I also totally love that.
10:26 I have to rent, you know, so many things via cloud services.
10:30 And I think part of that is because it's so easy to get around licensing restrictions.
10:35 So I don't know.
10:36 I'm of two minds, but I think wherever I can, just cutting down friction by using free and
10:42 open source licenses is, is just, just makes everyone happy.
10:45 Yeah.
10:46 Yeah.
10:46 Well, I mean, there could always, always be some sort of grace slack area, right?
10:53 Like, okay, you have 10 licenses, but we're not going to cut it off until you have 15 processes
10:57 right.
10:57 Yeah.
10:58 There's just like, when it's such a hard and fast rule, I don't know.
11:01 It just cuts me the wrong way.
11:02 All right.
11:03 So you talked about some of the stuff that you've done at work and you talked to, gave me the
11:08 sense that you're kind of just doing some consulting and helping out various companies on various
11:12 projects.
11:12 But what do you do day to day?
11:13 Oh, sorry.
11:14 Yeah.
11:14 No, I don't do consulting.
11:16 I work for Google.
11:17 So I, I, I'm more of a, less of a philanderer and more of a serial monogamist.
11:23 So I gotcha.
11:24 Yeah.
11:24 I worked for Bloomberg for about four years and now I work for Google and our team is
11:29 called a Corp Eng machine learning.
11:31 So Corp Eng is our internal infrastructure team that, that handles things like facilities
11:36 and like tickets.
11:39 Like if you, if you open tickets with people like HR, things like that, those are all within
11:44 Corp Eng and we build machine learning models for other people in Corp Eng for, to, to provide
11:50 them insights or to make their job easier, that kind of thing.
11:52 So this is like the Google scale of automate the boring stuff to help you do your work
11:56 a little better.
11:56 Yeah, definitely.
11:57 So, you know, my day to day work is playing around in Jupyter Notebook.
12:01 Well, okay.
12:01 Everything in Google is like its own proprietary thing or even like when it's not proprietary,
12:07 it's still its own thing.
12:08 So like, we usually use collabs.
12:10 Unfortunately, you know, I like to rag on people being like having their own proprietary versions
12:15 of stuff.
12:16 But like every time I try and do that with Google, like they have like stupidly good
12:20 reasons.
12:20 It just makes it so much less fun to make fun of our proprietary technologies when, when
12:26 it's like, oh, well, they do have native integration with like Google Docs and, and, Google drive.
12:31 So yeah, I guess it does make sense to use it.
12:34 Yeah.
12:34 Yeah.
12:35 Yeah.
12:35 Sure.
12:35 I mean, there is in the tech space and the developer space, certainly the not built here.
12:41 So we've got to build our own sort of obsession where people could take that too far.
12:45 And it's just a, just a drag.
12:47 Like we don't need to build our own like booking system.
12:51 We can just get a, get a calendar system somewhere else and things like that.
12:55 But when it's large enough scale and it has like built in advantages, sure.
12:59 It makes sense.
12:59 Yeah.
13:00 I mean, I just wish that my colleagues were less competent because then I can make fun of
13:03 them, but you know, now I just have to respect them.
13:07 Oh, what kind of a work environment is that?
13:10 Pretty good one.
13:11 This portion of Talk Python to Me is brought to you by Brilliant.org.
13:17 Brilliant has digestible courses in topics from the basics of scientific thinking all the
13:21 way up to high end science, like quantum computing.
13:25 And while quantum computing may sound complicated, Brilliant makes complex learning uncomplicated
13:29 and fun.
13:30 It's super easy to get started and they've got so many science and math courses to choose
13:34 from.
13:34 I recently used Brilliant to get into rocket science for an upcoming episode and it was a
13:39 blast.
13:39 The interactive courses are presented in a clean and accessible way and you could go from knowing
13:44 nothing about a topic to having a deep understanding.
13:47 Put your spare time to good use and hugely improve your critical thinking skills.
13:52 Go to talkpython.fm/brilliant and sign up for free.
13:55 The first 200 people that use that link get 20% off the premium subscription.
13:59 That's talkpython.fm/brilliant or just click the link in the show notes.
14:07 All right.
14:08 Let's talk about time.
14:09 Every programming language has to deal with dates and times and they all have their own
14:14 way of doing it.
14:15 And I would say Python's is pretty solid, right?
14:18 It's got a pretty good system there.
14:21 So there's a couple of interesting things, I guess, about time.
14:25 One is it only goes back so far in computer time, right?
14:29 Some of if you if you're talking in ticks or things like that.
14:33 So I don't know.
14:33 What do you think about the daytime story in Python now that you're down in it?
14:38 I'm actually a big fan.
14:39 I may not have a great breadth of experience outside of Python, but I think, you know, the
14:46 more I've learned about Python daytime, the more I understand the choices that were made.
14:51 Like one of the biggest issues with daytimes is that you often see people like, oh, I can't
14:56 believe daytimes are so terrible in Python because of X, Y, Z.
14:59 Like, why isn't why isn't every daytime UTC by default?
15:03 Like, why is it some other naive thing or or like, why do I have to attach a different time
15:09 zone to this or that?
15:10 And the issue is that there are multiple camps of people who have equally strong.
15:16 It should just be this way opinions, but in opposite directions.
15:21 So most of the people think UTC is the natural thing to do.
15:25 But I think those same people may not realize that when you're working with dates in the
15:30 future or you're working with civil times in general, UTC is actually a very bad choice
15:35 because generally speaking, UTC is good for.
15:40 So I say UTC is good and UTC is actually like the best that we have.
15:45 I don't love that UTC has leap seconds in it because it takes some nice monotonic constant
15:51 that you can, you know, that you can use as a reference and turns it into this weird thing
15:57 with its own edge cases where sometimes there are 60 and sometimes there are 61 seconds in
16:01 a minute.
16:02 Very annoying.
16:02 Really?
16:03 I had no idea.
16:04 Yeah.
16:05 every fourth minute has an extra second or something.
16:08 It can happen as many as two times per year.
16:10 And sometimes they, I think they'll either skip the 59th second or they'll add one second.
16:15 I think it's only ever that they've added one second.
16:18 It's for the purposes of leap seconds.
16:21 Leap seconds are, I think, useful for some things like astro navigation or something, but
16:27 they're incredibly important for astronomers.
16:29 I have no idea why they're built into the foundations of civil time of regular human does not need
16:36 to know to the second when the sun is going to be overhead in Greenwich.
16:39 Right, right.
16:40 You know, maybe it makes more sense to have like an astro time that takes those like peculiarities
16:45 into account, right?
16:46 Astronomy has its own whole big long set of ways of doing timekeeping.
16:50 Like they have sidereal times and solar time.
16:53 And, and I think they usually use the Julian calendar going backwards.
16:59 I don't really understand exactly why they do all that stuff, but if you really want to
17:03 dig into like weird timekeeping practices, astronomers are astronomers and weirdly astrologers.
17:08 Like something you, you may not know is that a decent amount of the like historical corrections.
17:16 And actually I think the original basis of a lot of the data in the time zone database
17:20 that everyone uses came from astrologers who were like deeply concerned with exactly what
17:27 second you were born at.
17:29 So like if you birth certificate says like three, 15 and 14 seconds in Yugoslavia in 1981, they
17:38 have to dig up the, like the old Soviet era Yugoslavian timekeeping records to find out what that
17:46 meant in UTC time so that they could find out whether Jupyter was in the house of Aquarius
17:51 or some, some crazy thing like that.
17:54 That is crazy.
17:55 I had no idea.
17:56 If you don't cut me off on these things, I will just go on as many tangents, like a weird
18:00 fact.
18:00 Tell me how they're still involved and then I want to dig into something else.
18:03 Oh, well, they just, it's still important for them.
18:05 So you'll still see corrections where they're very concerned with like times before 1970 because
18:11 the record keeping is a little shoddier.
18:13 And so the time zone database, you know, basically they say anything after 1970, we try and keep
18:18 it as accurate as possible before its best effort.
18:21 So you'll still see people making sure they're like, oh, well the, like exactly how did the
18:28 clocks move on the ground at this time?
18:30 It's like the people who are on the time zone mailing list are basically a bunch of Unix
18:36 people, the historians who are digging through, looking at historical stuff and astrologers who
18:43 are just weirdly concerned with exactly the minute you were born.
18:46 How interesting.
18:46 I had no idea.
18:47 All right.
18:48 So what I, what I want to dig into is this idea of naive time versus aware times.
18:55 Sure.
18:55 You can do interesting math.
18:56 Like you can subtract two date times, but I don't think you can subtract like a naive
19:01 time from an aware time and stuff like that.
19:02 Like you can't combine these things.
19:04 Can you not?
19:05 I don't remember.
19:05 I feel like I've seen an error from that before.
19:07 That is true.
19:08 So it's changed a little bit between Python two and Python three, but not to the point
19:13 where arithmetic is allowed between them.
19:15 So in Python two, there was this idea that a naive time was essentially just a representation
19:22 of a time as it would show up on a calendar.
19:25 So it's divorced from any notion of real time.
19:28 It's an abstract, perfect daytime that just is the proleptic Gregorian calendar.
19:34 And an aware time represents a specific timestamp.
19:37 So you can take something that's in UTC and you can turn it into local time in New York
19:42 or in Shanghai or any, anything like that.
19:45 You can get durations between things.
19:47 So that was the distinction.
19:49 And, you know, the idea was that it's kind of like a naive daytime was a daytime without
19:55 a unit and aware daytime is putting units on it.
19:58 So the number five is just a number.
20:00 Five feet is equivalent to one and a quarter meters or something like that.
20:05 Right.
20:05 Yes.
20:05 1.6 or something.
20:06 In Python three, it changed a little bit for reasons that are complicated and like probably
20:13 not worth getting into.
20:14 It's very complicated to create a time zone, like a TZ info that represents local time and
20:22 maintains all the semantics that you would hope.
20:24 If during an entire process where the local time might change over the course of the process.
20:30 And so one of the things that happened in Python three is that now instead of naive times being
20:37 unit list, they just represent local time.
20:40 So they still act mostly the same way.
20:44 But if you do something like you call dot timestamp on them, they will give you, they'll convert
20:50 it to UTC and give you the number of seconds since 1970, January 1st in UTC.
20:55 You can also do like as time zone and you can just directly, you can pass that a time zone.
20:59 It'll convert.
21:00 It'll assume that the daytime represents local time and then convert it into the time zone you
21:05 care about.
21:06 So at this point, local times are, or naive times represent local time, which is why UTC
21:13 now and UTC from timestamp are not functions you should use these days.
21:16 Okay.
21:17 Because they basically, they generate a naive timestamp without attaching UTC offset or like
21:22 without attaching the UTC object to it, but they do it as if that represents UTC.
21:26 So if you do UTC now and then dot timestamp, it'll give you a timestamp representing a different
21:32 time than now.
21:33 Like I'm in a minus eight, so it would subtract eight hours from it or something like that
21:38 or add eight hours rather.
21:39 It'll plus or minus eight hours, but it'll change it in some direction.
21:42 That's not just leave it alone.
21:43 It was UTC now.
21:45 So it just, it is that time, right?
21:47 Yeah, exactly.
21:48 Okay.
21:48 Interesting.
21:49 Yeah.
21:50 I always feel like daytime seems so straightforward until you get into here and then it's just
21:55 like fraught with education.
21:56 Like, wait a minute.
21:57 What?
21:57 I didn't know I couldn't use UTC now.
21:59 Why is it still there?
22:00 Right.
22:00 Without at least a warning.
22:01 It's even worse than you might think too, because the semantics of like subtraction
22:05 are made even more complicated.
22:07 And like the people who designed this library, like they weren't just like idiots.
22:12 They actually were like really thinking things through.
22:16 And so most of the time you won't even notice this kind of thing.
22:19 But when you learn about it, it's a bit like makes your head explode.
22:22 So there's subtract, like subtraction works differently when you do it between the same
22:29 time zone and a different time zone.
22:31 So if you have two dates and they're both in New York time, right?
22:35 America slash New York.
22:36 What it'll do is subtract them just based on the calendar.
22:41 It won't give you the amount, the total amount of time elapsed between those.
22:44 Even if like, so if it's like the day before daylight saving time transition and the day after
22:50 23 hours have elapsed between one o'clock and one o'clock the following day, it'll give
22:55 you 24 hours between those two.
22:57 If they're both in New York.
22:58 Okay.
22:59 Because that's how it works in the other direction.
23:01 If you add 24 hours to something, it'll just shift over the calendar by one day.
23:07 And I think the idea was that they wanted it to be the way it works in the other direction
23:12 is you add one day and then it'll create something that's one calendar day in the future.
23:16 And then it'll attach the same time zone object to the new daytime.
23:21 So if you then subtract it back, you'll get, you'll still get one day.
23:25 The problem is that like, you can't just sort of shift around on the calendar when you're
23:30 trying to do operations between two daytimes, like one in Chicago and one in New York.
23:34 And the relevant, like the most reasonable thing to do is to just get the total amount of elapsed
23:40 time.
23:40 You just convert them both to UTC and say how much time has passed.
23:43 Right.
23:44 The thing is that, the way they decide between inter zone and different zone, like same
23:49 zone and different zone is whether the objects are the exact same object.
23:53 So if you construct a new time zone object, if your time zone provider allows you to create
23:58 a new time zone object representing the exact same zone, it won't check if they're equal.
24:02 It'll just check if they're the same object.
24:04 This is not really a problem with PyTZ because PyTZ doesn't use normal daytime mechanisms.
24:09 And like basically literally everything you do will generate a new time zone object.
24:14 So everything is an inter zone comparison.
24:16 Yeah.
24:16 And with date util and with the new zone info module, I've implemented a cache.
24:21 So basically all of these zones are singletons anyway.
24:24 So nowadays it's basically like if you use the same INA key, you'll get same zone semantics.
24:31 And otherwise you'll get different zone semantics.
24:34 It's more complicated than you would think.
24:36 And it's really just all down to these overloaded concepts that are just everywhere in daytimes.
24:41 Like the idea of adding a duration is horribly ambiguous, right?
24:46 If you consider adding a month, right?
24:48 What does that mean?
24:49 Does that mean 30 days?
24:50 Does it mean 31?
24:50 Does it mean give me the next, the same day, the next month?
24:54 What if it's January 31st?
24:57 Do you go to the end of February?
24:58 Do you go to the beginning of March?
25:00 So for two days, if you add one month, does it all go to the same day?
25:03 Like, you know, do you just do one month, mod, however many days the difference is?
25:09 Like there's so many different options.
25:11 You know, you kind of have to pick one, but it's very frequently the case that
25:15 different users will have different, very strong expectations that it will do one thing or the other.
25:21 And for something so low level, like the datetime module, which is, you know,
25:25 essentially something like a standard, right?
25:28 It's trying to be basically a data type, an interchange type.
25:31 You just have to make choices that are hopefully going to be useful for the most,
25:37 the greatest number of people.
25:38 Yeah.
25:39 Well, it makes a lot of sense, but man, there are more and more edge cases.
25:42 The more we talk, I'm like, yeah, that's, that's tricky.
25:44 And you know, you think about, so if I was in Indiana, for example, like certain places
25:49 in the United States have different rules about following things like daylight savings, right?
25:54 So Arizona, Indiana, a couple others, but I pick on Indiana because they've got a county
26:01 by county system.
26:03 So depending on which county you live in, like you're actually, let me just read this a bit
26:07 from Wikipedia just so people know what I'm talking about.
26:09 It says, most portions of the state that were in the Eastern time zone do not observe
26:14 DTS daylight savings.
26:16 However, Floyd Clark and Harrison counties, which are near Louisville, Kentucky and Ohio
26:20 and Dearborn counties, which are near Cincinnati and Ohio unofficially observe it because they're
26:26 proximity to these cities, which do observe it.
26:28 So if I've, if I have like two day times, one in that county or not in that county, right?
26:35 It's in different time zones in different parts of the year based on like region.
26:40 Like what do you even do there?
26:41 Well, in the time zone database, the INA time zone database is also sometimes called the
26:46 Olsen database.
26:47 What they have is that, I mean, it's one of the reasons why names like us slash Eastern
26:52 and us slash Pacific are actually deprecated.
26:55 The way those keys work is that they usually go with a region slash the largest city in the
27:02 region that has continuously had used the same rules since 1970.
27:07 So America slash New York is the same one that you use in New York as in Boston.
27:12 Yeah.
27:13 Or in Virginia, anything like that.
27:14 If Massachusetts switches over to being part of a Atlantic standard time, then there will
27:20 be a new zone created called like America slash Boston and it'll cover Massachusetts going
27:26 back all the way to 1970 in Indiana.
27:28 They have their own like sub key.
27:29 So they have like America slash Indiana slash Indianapolis and then like five different states
27:35 there.
27:35 I think Mountain Time has a couple things like that as well.
27:38 Like Denver and Phoenix are separately, are handled separately.
27:41 I see.
27:42 So you have to be very precise about your, like your time zone info passed over, but then it'll
27:47 actually take care of that.
27:48 Yeah.
27:48 It definitely handles that in terms of historical times.
27:51 Like this gets to an issue that it's another one of these overloaded concepts, which is like
27:55 when we talk about a date time for some people, like if you're looking at server logs, people
27:59 are like, why would you not just use UTC all the time?
28:03 It's unambiguous, but those people are not implementing say a scheduling service, right?
28:08 Where, you know, like you and I scheduled to meet at a specific time to record this podcast.
28:13 Right.
28:14 If someone had changed daylight saving time, like if between the point where I scheduled it
28:19 and today, the United States said, we're in so many crises right now that we're just going
28:24 to like switch off daylight saving time.
28:27 I don't know why politicians love messing with time at the last minute, but they absolutely
28:30 do.
28:31 If someone had done that, then we would still want to meet at four o'clock because relative
28:37 to my work schedule and your work schedule and everyone's school schedules, that's still
28:41 the right time for us to meet.
28:42 But the mapping between four o'clock and UTC will have changed.
28:46 So what that means is like, if we had recorded that as saying, okay, this is going to be as a
28:52 daytime guy, I should always be embarrassed by this, but I can never do any daylight saving
28:55 time calculations in my head.
28:57 Hey, we have computers for this stuff.
28:59 Come on.
28:59 Yeah.
28:59 It's like eight o'clock UTC is when this recording is going to happen.
29:03 If all of a sudden the mapping between four o'clock and eight o'clock changes, you and I
29:06 will be very unhappy that they stored it in UTC and then converted it before showing it to
29:11 people.
29:11 So when you're dealing with the distinction I usually make is timestamps versus civil times.
29:16 So a civil time is like when you actually care what the clock on the wall says, then
29:22 you should use a civil time, which is you store exactly what the clock on the wall says.
29:26 And then the time zone you stored it in America, New York, something like that.
29:30 Right.
29:31 This is to get back to this point about what you do, like how you store it right now.
29:36 There's no way to solve exactly that problem, or there's not a great way to solve exactly
29:40 that problem.
29:41 Because what you really care about is like a little hard to pin down because, you know,
29:47 I'm in New York, you're in Portland, right?
29:49 Yeah.
29:49 So, you know, if one of us changes our time zone, if our relative time zones change, New
29:57 York changes its time zone offset, but Portland doesn't.
29:59 Which, you know, the meetings with two people, which one controls, which one matters, you
30:04 know, or like, I said, it's American.
30:06 Do you reorient it based on like the current understanding of time in New York and then break
30:11 Portland or do you stick with Portland and now you're out of sync?
30:14 I think the best you can do is you pick an INA zone and you just store it that way.
30:19 Usually that won't be a major problem.
30:21 If you're like super belt and suspenders, like storing a user's location, you say like,
30:26 where is this meeting or like, where's the rules that you care about?
30:29 And then you'd have to find a library that maps locations to time zones as a function of
30:35 time.
30:35 And then just make sure that that library is kept up to date.
30:38 That should sort of sort you out.
30:40 But I think the best you can do is to monitor the time zone mailing list and, you know, just
30:47 be aware if you see anything that's going to affect you.
30:51 Yeah.
30:51 Yeah.
30:51 Well, there's definitely a lot of little edge cases and that's pretty interesting.
30:54 So another thing that's interesting in the whole time space of Python, and I think it's
31:00 done pretty well, is time deltas, right?
31:02 And the fact that I can have two daytimes and I just subtract them and that gives me a time
31:07 delta object, or I can have a daytime and then add or subtract the time delta and that generate
31:11 a new daytime that's offset by that.
31:13 What's weird about time deltas that I should be learning about?
31:16 Because they seem pretty straightforward.
31:18 I would like to see them do a little bit more help for people, but this is already pretty
31:22 good.
31:22 The biggest thing semantically is an issue that I already alluded to, which is that they're
31:28 a little overloaded in that sometimes they represent an elapsed amount of time.
31:32 And most of the time, and usually they're interpreted when you like apply them to something, they're
31:37 usually interpreted as a calendrical delta.
31:41 So it's like, it'll give you, you know, if you add one hour to something, if you add one
31:46 hour to like 1.30 right before a standard time to daylight saving time transition, what you'll
31:52 get is some non-existent time that is right in the middle.
31:56 Like, you know, it's like a 2.30 that never happened.
31:59 Or you add one hour, like one minute to right before that you'll get, or you add one hour
32:05 and one minute in such a way that it spans that you'll get something that is nominally
32:10 one hour and one minute away.
32:12 But in fact, two minutes have elapsed, something like that.
32:15 Right.
32:15 Okay.
32:16 So that surprises people.
32:17 I think it tends to surprise people more on certain scales, like for very short time scales,
32:22 one minute, two minutes, something like that.
32:25 You're like, why isn't this working exactly as I expect when it's like one day, you're
32:29 usually trying to just get the same time the next day, something like that.
32:32 I don't know that there's a great solution to that.
32:35 One thing that I would kind of like to do, and I don't know that there's any way to do it
32:39 without breaking backwards compatibility is to separate out the concept of like a calendrical
32:45 delta and an absolute delta, which is like, give me this elapsed period of time.
32:50 You can certainly add a type that represents elapsed times.
32:54 And then when you add it to a daytime, it converts the daytime to UTC, does the addition
32:59 and then converts back.
33:00 But the issue there is that what you'd really want is you'd really want the results of those
33:07 interzone subtractions to return those elapsed time, you know, elapsed time time deltas, because
33:13 that's actually what's happening.
33:15 You know, you're actually getting an elapsed time time delta.
33:17 So if you take something that's like something in America Chicago minus something in America
33:22 New York, you'll get the absolute exact time.
33:25 And then if you add that to America New York and then convert the result back to America
33:29 Chicago, you might get a different value than the value that you started with.
33:33 So like that property is something that you would expect to be true.
33:38 Right.
33:38 That item potency of like, if I change it this way and then I change it back or that sort of
33:44 additive identity or whatever you call it there that like I make this change and I undo that
33:48 change.
33:48 It would be weird if it's like not back to where it started and stuff.
33:52 Right.
33:52 Yeah.
33:52 So, I mean, in that very particular edge case, it is a little bit lossy and I would really
33:58 like it if there were a bit that I could add.
34:00 And maybe you could like maybe everyone who is seeing this behavior is expecting the that
34:07 like reversible transformation.
34:09 And basically, we would just be fixing a bug for every single person.
34:13 But there may be people like you always hate to punish people who figured out the right thing
34:18 to do and then did it.
34:19 And then you just like change what the right thing to do is out from under them.
34:22 Yeah.
34:22 Once you've kind of put it in place, then it's really hard to change it.
34:27 Right.
34:27 I mean, you're kind of stuck.
34:27 Yeah.
34:28 I mean, that's the story of my life.
34:29 But you make slow and steady changes and you break things only when you absolutely have
34:34 to.
34:35 And hopefully you get a reputation for not breaking things just willy nilly.
34:39 And people take you seriously when you say, hey, we really want to break all your stuff
34:43 today.
34:44 Yeah.
34:44 Well, I think that that's actually one of the reasons that Python is going so strong.
34:49 Right.
34:50 You know, JavaScript is going strong as well.
34:52 But the challenge with JavaScript is like I spent all the time learning Angular and then
34:58 they threw away Angular 1 and they rewrote the API and they switched it to TypeScript.
35:02 So I got to start over.
35:03 Or, you know, like pick your framework du jour.
35:06 Like the stability of those things or the life cycle of those things is so short.
35:11 And, you know, the fact that you don't have to keep changing because your dependencies or
35:15 whatever changed.
35:15 That's like dependency being Python potentially.
35:18 That's a really good feature, I think.
35:20 Yeah.
35:21 I think sometimes also things have a short, tend to have a short shelf life in some situations
35:27 because people want to break a whole bunch of stuff.
35:30 And I guess I'm just glad that we have maintainers around who are willing to kind of put in the
35:36 work for the long haul to make it so that, you know, hey, this thing is still
35:41 supported and you don't have to migrate to a new framework every 30 minutes.
35:44 Yeah.
35:45 So one thing about daytimes that I want to ask, sorry, time deltas that I want to ask you
35:49 about because it drives me crazy is I find myself doing math way more than I think I should with
35:55 time deltas.
35:56 For example, if I have a time delta and I want to know how many days that that time delta
36:02 represents, that's a math problem for me, right?
36:05 It has a day's operation, a day's property or field, but it gives me unusual answers in some sense and not a
36:13 fraction of days, right?
36:15 So if I have something that's 30 seconds, I would like it to be whatever that is, like one one hundredth of a day.
36:21 Or I would like to be able to ask, how many hours are there?
36:24 How many weeks?
36:25 You can do that.
36:26 Months and stuff starts to get hard.
36:27 Okay.
36:28 How do I do it?
36:29 Because all I know about is total seconds, for example.
36:31 And then there's seconds and then there's microseconds and there's days.
36:35 Those accessors, like the days, seconds, microseconds thing, those are actually, time delta is essentially
36:43 supposed to be like a struct type object that just like holds a time delta, you know, and it has three
36:49 canonical components, which is days, microseconds, days, seconds and microseconds.
36:54 I understand that it's confusing that those accessors are like not the total number of days.
36:59 The way you're actually supposed to, this was added in Python 3.
37:02 The way you're supposed to get this information is time deltas can just be divided by other
37:06 time deltas.
37:07 So if you just have like a time delta and you want to know how many days there are, you just
37:11 divide it by time delta days equals one.
37:13 Oh, interesting.
37:14 So instead of going total seconds divided by 60, divided by 60, divided by 24, I would say
37:20 dt divided by time delta days equals one.
37:23 Yeah.
37:24 And you can just have any arbitrary value as the denominator there.
37:27 So it's actually, this is one of those situations where it's like not super discoverable,
37:32 but it's incredibly elegant.
37:33 Yeah.
37:34 And like you can, the fact that time, that total seconds even exists is a problem.
37:39 Well, it's not a problem, but it highlights this fact that it's not super discoverable because
37:44 the, when they added total seconds, they had already added the ability to divide by, they
37:50 had already solved this problem by adding the, by defining a dunder div.
37:54 Yeah.
37:55 And one core developer didn't know about that and added total seconds.
37:59 And then the guy who was, you know, maintaining daytime showed up after it was all merged and
38:03 stuff and was like, oh, I wish I had seen this before.
38:05 I would have said, please don't add this.
38:07 Yeah.
38:07 So, you know, every once in a while we get people who show up and they're like, can we
38:10 get total microseconds and total whatever?
38:12 And it's like, we don't want to make 40 different, you know, things that say quarters and months
38:17 and years and things like that.
38:19 Like just, you just have to divide it by a time delta and it works super well, especially
38:24 now that increasingly you are, people are working in Python three only code bases.
38:28 If you have a two, three compatible code base, you generally do have to like do total seconds
38:33 and then divide that by total seconds of whatever your time delta is.
38:37 Gotcha.
38:37 Okay.
38:38 Interesting.
38:39 Yeah.
38:39 Cause I was hoping there would be a total minutes, total hours, total days, and it could
38:44 just do that math that I talked about just in line.
38:47 Right.
38:47 And like, just so I don't have, so I could be more expressive and not do like a long series
38:51 of divisions.
38:52 You certainly can do that.
38:53 Like, I can understand why you would want that, I suppose, but then, you know, you only
38:59 have the enumerated units that, that are in there.
39:03 And it also sort of hides the fact that you can just do regular division.
39:07 I mean, you don't have a dozens method on, on integers to find out how many times 12 evenly
39:13 divides in it.
39:14 You know, it's a fair point.
39:16 I think that's the discoverability that is the challenge, right?
39:19 To know that you can actually do the division and it'll, cause now you say it like, sure,
39:23 it makes total sense to say, I want to know how many whole weeks are in there.
39:27 So I would say time delta days equals seven and then divide it.
39:29 And that would give me a great answer.
39:31 Right.
39:31 And I can pick my units to be whatever.
39:33 You can also do weeks equals one.
39:35 Yeah.
39:35 Yeah.
39:35 Yeah.
39:36 Weeks equals one.
39:36 Sure.
39:36 So there's a lot of flexibility there, but yeah, it's just, I don't see a lot of examples
39:41 using that, you know, when you look around at code stuff.
39:45 That's partly because people don't see it.
39:47 And partly because anything that a lot of the code that we look at now, especially open source
39:51 code has been two, three compatible for like the past five or 10 years or something.
39:55 And that idiom is not available in Python two.
39:58 So Python three, when you're Python three only, you know, it's kind of like you're starting
40:02 to see f-strings everywhere.
40:03 Hopefully all the cool kids now are just dividing their time deltas.
40:07 Well, they will be now, now that they hear about it on the podcast.
40:10 Yeah.
40:10 I thought you were going to say the thing about how the rapper and the, the stir, like the
40:16 string cast of time deltas is just absolute trash.
40:19 Like I really wish I could fix that.
40:21 I think it's going to be difficult to do so, but it's like, if you print a time delta that
40:27 is negative one hour, it tells you one day minus 23 hours or tells it to you in seconds.
40:33 Yeah.
40:34 The internal reputation is weird.
40:35 Like so for, I do that for one that is one second long.
40:38 It's negative one day, 23 hours, 59 minutes and 59 seconds.
40:43 If it has like a negative one second or something in it.
40:45 Yeah.
40:46 It's, you know, it computes to the same, but it's not very human.
40:49 Yeah.
40:49 I would probably do it a different way.
40:52 I would probably do it as like negative and then abs, like an absolute value.
40:56 I think they're worried about some ambiguity there, but I don't think anyone ever wants
41:00 what we got.
41:01 So yeah.
41:02 You've given like 99.99% of people something they don't want to avoid a tiny bit of possible
41:08 ambiguity.
41:09 It's right.
41:09 Yeah.
41:09 What are you going to do?
41:11 Well, so let's talk a little bit about some of the, the peps here.
41:15 So you've got pep495 and that one, is that one already done?
41:19 That one's already done.
41:20 Yeah.
41:21 So pep495 was implemented for Python 3.6 and that adds the fold attribute.
41:27 The background of this is that prior to Python 3.6 time zone support was not super great.
41:34 Basically, whenever you have an offset change that where the offset decreases, that means
41:39 that you are jumping back by an hour or by any amount.
41:43 And that creates an ambiguity, right?
41:45 So it's two o'clock and then you jump back one hour.
41:49 Hopefully it's two o'clock.
41:50 I actually found one place in, I think in Argentina in 1992, their daylight saving time
41:56 transition happened at midnight on February 29th.
42:00 Cause it wasn't bad enough to do it at midnight with that ambiguity.
42:03 They also had to do it during leap day.
42:04 So my goodness.
42:07 Yeah.
42:07 So it's like, anyway, so you do something like that, right?
42:11 And so instead of it being February 29th or March 1st, it's now February 29th and at
42:18 11 PM a second time.
42:20 So the way time zone handling works in daytime is, is, is actually quite elegant.
42:25 The idea is that you attach because a time zone represents a set of rules for like taking
42:32 the current for mapping between absolute times, like time and UTC and local time.
42:39 Like what time does it say on the clock?
42:41 So I think the original design intent was that you would have a TZ info and the TZ info has
42:47 these functions that you can call that give you information about the time zone as a function
42:52 of the daytime.
42:53 So it's super great.
42:54 You just write this function that maps your daytime to UTC and you're done.
42:59 Except that the daytime portion is sometimes ambiguous, right?
43:04 So if it's 1992 and it's 1130 in Argentina on February 29th, there are two different offsets.
43:11 There's two different sets of information that you could apply there.
43:14 And there's no function that you can write that just takes the naive portion of that daytime
43:19 and gives you the correct offset because it's ambiguous.
43:22 So the way PyTZ solved this problem was that they added this localized concept where what
43:29 they do is they, you create a time zone and then instead of just like attaching it directly,
43:33 you have to call localize on it.
43:35 And what PyTZ will do is it'll just attach a dedicated fixed time zone offset.
43:40 So it has a little switch in there where you can say, do I want to be on the daylight saving
43:45 time side or the other side?
43:47 This is a major source of bugs because people will just do what the docs say and attach PyTZ
43:52 time zones directly to the TZ info instead of letting PyTZ do it itself.
43:57 And also it kind of circumvents a lot of the other logic in there about a lot of assumptions
44:02 about like you're just supposed to have this one set of rules, not a fixed set of offsets.
44:06 And then it's also a bit of a problem because they specifically use DST, but there are situations
44:11 where you have transitions that are not due to daylight saving time.
44:15 Like someone just switches their base offset or something.
44:18 So that's where PEP495 came in.
44:21 They, what they did was they said, we're just going to take the naive portion of the date
44:25 time and we're going to add an additional component to it called fold.
44:28 And I can take two values, zero or one.
44:31 The zero represents the first, the offset that happened before the transition.
44:36 And then after the transition, you put fold equals one.
44:39 It's very elegant.
44:40 It allows you to write time zones that just take the naive portion of the daytime and then
44:45 just flip a switch if it's ambiguous based on the fold.
44:48 Okay.
44:49 Yeah.
44:49 That's, that's cool.
44:50 Yeah.
44:51 Just catching these little edge cases.
44:52 And then PEP615 you're working on as well.
44:56 And this one's in 3.9.
44:57 So I guess in a sense it's accepted because 3.9's in its beta world.
45:01 Yeah.
45:01 It was accepted and we merged it.
45:04 So it's going to be, you can get it in the betas now.
45:06 You can actually get it.
45:07 I have a backport of it implemented back all the way to Python 3.6.
45:13 It works in PyPy.
45:14 I've tried to make it, you know, my goal here is basically that I would really, really like
45:19 it if people could stop using PyTZ.
45:22 I think the maintainer of PyTZ would also agree with me on that.
45:25 It's basically PyTZ was sort of a hack around this issue.
45:29 They did a really great job on it.
45:31 Like, I think, I don't know that there's a better way to have done that, but well, you
45:36 know, other than just coming up with this fold idea yourself and that has its own problems.
45:40 In DateUtil, I also have a backport of PEP495 that works in 2.7.
45:47 So even if you have a Python 2.7 implementation with some combination of the zone info module,
45:53 which was added in PEP615 and DateUtil, you can actually get proper PEP495 support all
46:00 the way back to Python 2.7.
46:02 And yeah, you know, there's a bunch of reasons why you should use zone info.
46:07 One is obviously, you know, I think it's enough of a selling point to just say, you don't have
46:12 to do this localized, normalized stuff.
46:14 Like you can just use time zones like they were meant to use and it'll just work.
46:18 But there's also the fact that PyTZ doesn't have any support for the version 2.tzif files.
46:26 So the INA, the compiled versions of the INA time zone, they're all version 2 or 3 now,
46:32 like since 2005.
46:33 And version 2 and 3 is when they switched from using 4 byte integers to 8 byte.
46:39 So each time zone now starts with version 1 file, like the entire version 1 file.
46:44 And then after that, they have the version 2 file.
46:47 So all the stuff that supports version 1 files only still works with your version 2 and 3 files.
46:51 But version 2 and 3 files will go past 2038.
46:55 And they, instead of just like stopping at some point, they also have like a general rule.
47:01 So they'll say like, here's the rule for transitions after 2038.
47:05 Right.
47:06 We don't have data for this, but here's how it has been.
47:08 So just keep computing it like this.
47:10 Yeah.
47:10 I mean, the further into the future and actually the further into the past, like the distant past,
47:16 like just like what was the time is significantly less meaningful question to ask.
47:21 So right now, you know, what time zone, like what is the UTC for New York in 2040 is not a terribly like gazing into a crystal ball, right?
47:33 It's right.
47:34 You can have a guess.
47:35 It's probably a better guess than randomly choosing, but it's not going to be accurate.
47:39 Like it's not, it might be a bit of an XY problem if you're asking that question, but you know, it's still like, at least it'll do what you expect.
47:47 But as time goes on in the next 10 years, in the next 15 years, we're getting a lot closer to 2038.
47:55 You're going to need that kind of support.
47:57 Right.
47:58 So PyTZ doesn't have support for that.
48:00 It doesn't have support for sub-minute offsets.
48:03 Another thing that ZoneInfo does and DateUtil does, but PyTZ does not do, is that they will hit your system time zone information first.
48:12 So there's a way to configure it.
48:14 They have their own access to time zone.
48:16 They have their own time zone data in a sense that you can get if your system time zone data, if your system doesn't provide time zones.
48:23 And there are mechanisms for specifying where you want to get the time zone data from.
48:28 But with PyTZ, it ships the time zone data with it.
48:32 So you have to keep your PyTZ up to date just to get the data updates.
48:35 There's a way to change that where I think you like set an environment variable and tell it where your system time zones are.
48:43 And it will use that instead of its own thing.
48:47 But you have to know where your stuff is.
48:49 Whereas with ZoneInfo, it's built in.
48:52 So it'll find it in the most common places.
48:54 And then there's a mechanism for the person who distributes Python to you to tell the compiler where it's going to be.
49:01 So if you use some system Python, it'll always hit your system time zone, which is very helpful.
49:07 So, you know, there's all those reasons to not use PyTZ.
49:10 Yeah.
49:11 There's still reasons why you might be worried about switching over.
49:14 Like, for example, since PyTZ has its own weird interface, you know, there's going to be a lot of your users possibly who've taken your TZ info and they've called localize or normalize on some date time.
49:25 And if you change over from PyTZ, it's going to break all of them.
49:29 But I am aware of this kind of thing.
49:32 So I've also created a library called PyTZ deprecation shim, which is just a thin wrapper around ZoneInfo, except that it supports the PyTZ interface as close to the semantics that PyTZ has as possible.
49:47 And then it just raises a warning if anyone's doing anything PyTZ specific.
49:51 So it just works like a regular 495 thing.
49:55 It works in 2.7 and 3.6 onwards.
49:58 And if anyone's, like, using localize or normalize or whatever, it'll raise a warning.
50:02 So you can basically say, oh, for the next two years, I'm going to depend on PyTZ deprecation shim, and then I'm going to drop it in favor of ZoneInfo.
50:10 And hopefully everyone has not been ignoring all my warnings.
50:14 Yeah, that's good.
50:16 It's really nice to have that warning there.
50:17 So it gives you some time to deal with it.
50:20 All right, Paul, we talked about a lot of different things, a lot of interesting edge cases and scenarios.
50:26 Maybe give us the main takeaways, like, if you could give people listening some advice on the please do and please don't with time in Python, what would you say?
50:34 Yeah, so for don'ts, I actually have a couple of blog articles about this.
50:38 Like, one of the things I see the most often is people using UTC now thinking like, oh, I want something in UTC.
50:44 But UTC now and UTC from timestamp are not something you should be using in modern Python.
50:49 I would like to remove them.
50:51 But there are, like, occasionally these situations where they're useful.
50:55 Like, if you're immediately trying to, if you're just using UTC now to, and then dumping it right into ISO format, and you don't want the, like, the UTC offset in there.
51:07 Right.
51:07 It's not a super great way to do that.
51:09 I might just bite the bullet and deprecate those.
51:11 And then maybe, you know, have a wrapper function for it in dateutil.
51:15 But for the most part, you don't want to use UTC now.
51:19 You should use just daytime.now and then pass it the daytime.timezone.utc object, which is just a UTC singleton that's in a standard library.
51:27 Also, I don't think you should use PyTZ.
51:30 I think you should take all your PyTZ code and migrate it over to zone info or dateutil.
51:35 And start getting on the PEP495 bandwagon.
51:38 And I've got a little migration guide for that.
51:41 Nice.
51:41 We can put the links to those in the show notes.
51:43 Now, I just want to do, maybe acknowledge a couple of libraries that are on the outside of CPython, but still pretty helpful.
51:50 And the first one actually is Python-DateUtil.
51:53 That one's yours, right?
51:54 Yeah.
51:54 So I didn't write that, but I've been maintaining it for about six years.
51:58 That's got some interesting stuff.
51:59 Even after PEP615, the timezone module does still have some useful timezones in it.
52:06 Like it has, well, you know, I say useful.
52:09 What I mean is like available, like different types of timezones that are not in the standard library.
52:16 So like there's this TZiCal.
52:18 If you have timezones in this iCalendar RFC format, dateutil will handle that.
52:24 And you can also like use like these POSIX strings.
52:26 Like if you ever see like EST5 EDT and then like a, there's a POSIX standard for expressing recurring DST transitions.
52:35 So it has that and some Windows specific stuff as well.
52:39 You know what I use it for that I really like is its parse.
52:42 Its parsing is really good because it seems like it generally just handles time strings, which is great.
52:48 Yeah.
52:49 The parser is the part that I like the least, but that's mainly, I mean, because I have to maintain it.
52:54 And it's got so many rules that it's super easy to break.
52:56 So anytime someone shows me something where it's broken, it's like, I'm really afraid to change anything.
53:01 Because if I do, it'll break someone else's use case that they've been using it for like 10 years.
53:06 It's also like pretty slow.
53:09 So it's super useful if you have something that you know is a date time and it's in a pretty standard format and you just want it to like chew at it until it finds something that looks like a date time.
53:21 But it could be chewing for like 200 microseconds.
53:24 If you have something that's in ISO 8601 format, dateutil also has ISO parse, which will handle anything that's in a valid ISO 8601 format.
53:35 And then date time, I think the first thing I added to CPython was from ISO format, which is like the inverse of ISO format.
53:42 Okay.
53:43 That's cool.
53:43 That's on date time.
53:44 It's got our rule.
53:45 Dateutil also has our rule for recurrence rules, which are very useful.
53:50 It has relative delta if you want to do time deltas, but include years and months and things like that.
53:55 And the best one I saved for last is Easter.
53:59 It just tells you when Easter is.
54:01 You give it a year and it's like, this is when Easter is.
54:04 Oh, that's cool.
54:05 Yeah.
54:05 There's a couple of different holidays and stuff that are all sorts of all over the place that are tricky, like Hanukkah and stuff.
54:13 So the two others that come to mind, and I'm sure there's like 10 additional libraries that people are out there, let's see.
54:19 But this one, I just don't have them off the top of my head.
54:22 One is called Arrow.
54:24 Are you familiar at all with Arrow?
54:26 Yeah.
54:26 Yeah.
54:27 I've contributed to Arrow a couple of times.
54:29 Yeah.
54:29 It's pretty cool.
54:30 Some of the things, I guess, that stand out to me, like, that got my attention the most is that it has humanized.
54:36 So you can have a time and you say .humanize.
54:38 It'll say an hour ago.
54:39 And you say humanize.
54:40 An hour.
54:41 The locale is Korean or something.
54:43 And then you get the Korean statement of an hour ago and so on.
54:47 And that's pretty easy.
54:48 Pretty nice.
54:48 Yeah.
54:49 I mean, so for stuff like Arrow and Pendulum and Maya and DeLorean and all these things, I've never found a huge amount of use for them.
54:56 But I'm not, like, a huge, like, I don't actually, like, I don't know that I have the exact use cases.
55:01 I tend to write libraries and taking on big dependencies is not the easiest thing for me.
55:06 But I do know that people kind of swear by these libraries.
55:10 They really like them.
55:10 So I think that there's certainly something to them.
55:13 The biggest problem I have with them is that they tend to be, they tend to try and advertise themselves as, like, drop-in replacements for date time.
55:21 And so they then subclass date time and time delta.
55:25 Stuff like Pendulum advertises itself as a drop-in replacement, but then it breaks all the stuff that I want to break, you know.
55:32 So it's not actually a drop-in replacement.
55:34 You can't even construct it the same way.
55:36 And they tend to be slow as well.
55:39 I mean, this is partially because they're not written in C or not mostly written in C.
55:44 But, you know, to do simple time delta arithmetic in Pendulum is, like, 10 times slower than doing the same thing in C.
55:52 But, you know, if you're not handling a huge number of date times, it's probably fine.
55:58 But, of course, you know, we don't have human eyes and things like that in the core.
56:01 I think that those are very useful things.
56:03 I would tend, like, I would love to see a library that is kind of like Pendulum or Arrow, but instead of it being methods on a date time subclass, it's, like, just free functions, you know, utility functions.
56:16 Right.
56:17 You could pass it, built-in items.
56:18 Yeah.
56:19 Yeah.
56:19 I mean, I think that would be faster in a lot of ways.
56:22 Sometimes people don't care about fast.
56:23 Yeah.
56:24 Yeah.
56:24 Well, if you're building a library, especially if you're building CPython itself, it's a different level of concern.
56:29 Yeah.
56:30 If I'm rendering a web page and I've got 10 items and I want, like, the human expression of how long ago they were, I probably don't care.
56:36 Plus, you can benchmark.
56:37 Yeah.
56:38 Like, I can sort of, you can benchmark and see where the bottlenecks are.
56:41 I can't benchmark everything on PyPI and everyone's proprietary code bases and find out, like, you know, do some sort of optimization and find that it's, this is not the bottleneck anywhere that this code is being used.
56:53 Right.
56:54 Exactly.
56:55 It's a different scenario.
56:56 And, yeah, I do find that just using bare date times is useful for interop.
57:02 But if you're mostly working in your own stuff, like, you're just passing it around, doing stuff with it, they do have some, like, you know, people do like it, you know, they really do like the interfaces.
57:12 They like that the time delta things that they give you have total minutes and that they have human errors.
57:17 Right, right.
57:18 Yeah, they got, like, weeks and hours and stuff.
57:20 The thing that stands out as most unique about Pendulum is they have a duration period thing that is aware.
57:29 So I could have a date time span, the subtraction between two date times, the equivalent of a time delta.
57:37 And then you can say in weekdays and then for each thing in the period.
57:42 And it'll give you day one, day two, skip the weekend, day three.
57:45 So, you know, Thursday, Friday, Monday, Tuesday, as a thing you can go.
57:49 So if you're like, well, there's, it's this long, I need to know how many weekdays there are or how many weeks or something in this span.
57:56 That's an interesting concept.
57:58 Like in detail, we have this R rule, which is based on a data type described in RFC 5545 called the recurrence rule.
58:07 And it is super, like, it's a little more complicated than that.
58:11 You know, again, this is why I think these sorts of things could be done as free functions.
58:15 It's very interesting.
58:16 And like, it's a little too complicated for the average person to just sort of jump right into it.
58:21 But you can express all kinds of stuff like that and say, like, how many Tuesdays are there?
58:26 Like one of the talks I give on date you till, I think I say, like, how many times did like the guy who played Mr.
58:32 Miyagi, Pat Morita, like how many times did his birthday fall on a Tuesday during the Vietnam War?
58:37 You know, and I have like a simple R rule that is like standardized.
58:41 You can cast it to a string and pop it into Google Calendar.
58:43 And I can just easily do that with an R rule.
58:45 I say easily, but that's after maintaining an R rule library for use, right?
58:49 Like a random person.
58:51 Right, right.
58:51 But it does it.
58:51 I don't know.
58:52 It's like saying you can find that with a regular expression.
58:54 Yeah.
58:54 Right.
58:55 Yeah.
58:56 Yeah.
58:56 You can easily do that with a regular expression.
58:58 You can easily do that with a regular expression.
59:00 I will struggle with that regular expression, but it's fine.
59:03 But it's powerful.
59:04 It's cool.
59:05 All right.
59:05 Well, I think that's probably it for all the time we have.
59:08 We've run out of time talking about time, but there's been a lot to talk about.
59:12 It's very interesting.
59:12 Thanks.
59:12 So before you go, though, let me ask you the final two questions.
59:16 If you're going to write some Python code, what editor do you use?
59:18 I usually just dictate that to my secretary and just kidding.
59:23 That sounds good.
59:23 I usually use Vim.
59:25 I don't feel strongly about it, though.
59:26 So if anyone wants to, well, don't try and convince me about it, but also don't see that
59:31 it as an attack on your Emacs using Waze.
59:33 Like I just picked one at random.
59:35 Sure.
59:35 Sounds good.
59:36 And then notable PyPI package.
59:38 There's one that's really fun, but please don't like if you install it, do it in a virtual
59:42 environment or something.
59:43 It's called a break my Python.
59:45 You pip install it, and then it just makes it so that your Python is broken.
59:49 Like you can't, you don't have to import it to have your Python be broken.
59:52 You just do pip install break my Python.
59:54 And if you type Python, it won't work anymore.
59:57 It'll just break it.
59:58 That's break dash my dash Python.
01:00:01 All right.
01:00:02 So that's kind of an anti-recommendation.
01:00:03 That's interesting.
01:00:04 Very cool.
01:00:04 Funny.
01:00:05 If you want a real recommendation of something that's like fun and useful to use is I actually
01:00:10 do like pipX.
01:00:11 Oh, yeah.
01:00:11 PipX is cool.
01:00:12 I love pipX.
01:00:13 Yeah.
01:00:13 At this point, my main Python installation, well, my main Python installation stuff is
01:00:19 like super layered with all kinds of indirection.
01:00:22 So like everything is all pip and or not pip and pyenv environments.
01:00:27 So like I have 10 different pythons installed.
01:00:30 But even within those, I never pip install anything directly.
01:00:33 It's either like working in a virtual environment or if it's tools like talks or virtual env or
01:00:40 something like PEP 517.
01:00:42 Well, actually, that one doesn't really work.
01:00:43 Like glances or PT Python or any of those types of tools.
01:00:47 pytest.
01:00:48 mypy.
01:00:49 Yep.
01:00:49 Yep.
01:00:49 PyLint.
01:00:50 All that stuff.
01:00:51 You just do pipX install whatever.
01:00:53 You do it like you're going to pip install it.
01:00:55 But what it does is it will install each one of these tools into their own virtual environments
01:01:00 and then just put on your path the entry points that they expose.
01:01:05 So you do pipX install mypy.
01:01:08 And then if you try to like if you open your Python and try and import some mypy like thing,
01:01:13 it won't be there.
01:01:14 But if you just type mypy, it'll work as if by magic.
01:01:17 And this avoids the whole situation of like different tools stepping on each other's dependencies
01:01:23 or needing to update all that stuff.
01:01:24 It's really great.
01:01:25 Yeah, I really like it.
01:01:26 To me, it feels like homebrew for Python utilities and applications, right?
01:01:31 Like on my Mac, I can brew install MongoDB.
01:01:34 I can brew install FFmpeg, all these things that I might need.
01:01:39 And they'll be installed separately and maintained.
01:01:41 And you know when there's updates.
01:01:42 PipX is that for things that are tools based on Python that you would pip install them.
01:01:46 It's really nice.
01:01:46 Yeah, I mean, except brew also kind of maybe has that same problem, right?
01:01:50 Where because it's like a system package manager.
01:01:53 I mean, they handle it for you.
01:01:55 Yeah, it's a little more.
01:01:56 Yeah, it's less isolated.
01:01:57 Yeah, I agree.
01:01:58 Yeah.
01:01:58 Yeah.
01:01:59 I mean, as long as you're not using your brew Python, right?
01:02:01 Like if you don't open up Python that you brew installed.
01:02:05 Exactly.
01:02:05 Cool.
01:02:06 All right.
01:02:06 Well, PipX is a fun recommendation.
01:02:08 And Breakmypython is a good joke as well.
01:02:11 All right.
01:02:11 I'm looking forward to dig into that to see how it actually breaks your Python.
01:02:14 All right.
01:02:14 Well, thank you for being here.
01:02:16 Final call to action.
01:02:17 People want to get their time zones and time in order.
01:02:20 I mean, obviously, the recommendations you had, the do's and don'ts stand.
01:02:23 We'll put a link to those.
01:02:24 Anything else you want to leave people with?
01:02:26 I think it's pretty good.
01:02:27 Like I think I'm very interested in getting people to migrate over to Zone Info.
01:02:31 So especially right now when there's not a huge number of people knocking at my door
01:02:36 to ask about that.
01:02:37 But feel free to, you know, open some issues on there or send me an email or at me on Twitter
01:02:42 or something.
01:02:42 And yeah, like get out there and, you know, maybe like tweet or Instagram or TikTok or whatever
01:02:48 the kids do these days to all your friends about how great PEP615 is.
01:02:52 And hopefully we can get the word out that you don't have to use PyTZ anymore.
01:02:56 Yeah, absolutely.
01:02:57 Well, thanks for working on all this stuff and handling all these edge cases for people.
01:03:01 That way we can mostly ignore them except for when we can't.
01:03:04 Yeah.
01:03:04 Yeah.
01:03:05 Thanks.
01:03:05 See you later.
01:03:05 All right.
01:03:06 Bye.
01:03:06 This has been another episode of Talk Python to Me.
01:03:10 Our guest on this episode was Paul Gansel and it's been brought to you by Brilliant.org
01:03:14 and us over at Talk Python Training.
01:03:16 Brilliant.org encourages you to level up your analytical skills and knowledge.
01:03:21 Visit talkpython.fm/brilliant and get Brilliant Premium to learn something new every day.
01:03:27 Want to level up your Python?
01:03:29 If you're just getting started, try my Python Jumpstart by Building 10 Apps course.
01:03:34 Or if you're looking for something more advanced, check out our new Async course that digs into
01:03:39 all the different types of async programming you can do in Python.
01:03:42 And of course, if you're interested in more than one of these, be sure to check out our
01:03:46 Everything Bundle.
01:03:47 It's like a subscription that never expires.
01:03:49 Be sure to subscribe to the show.
01:03:51 Open your favorite podcatcher and search for Python.
01:03:54 We should be right at the top.
01:03:55 You can also find the iTunes feed at /itunes, the Google Play feed at /play,
01:04:00 and the direct RSS feed at /rss on talkpython.fm.
01:04:04 This is your host, Michael Kennedy.
01:04:06 Thanks so much for listening.
01:04:07 I really appreciate it.
01:04:09 Now get out there and write some Python code.
01:04:10 Bye.
01:04:11 Thank you.
01:04:31 Thank you.