#271: Unlock the mysteries of time, Python's datetime that is! Transcript
00:00 Time is a simple thing right? In working with it in Python is great. You just import a time and somewhat oddly use the date time class from that module. Oh, except there are times with time zones and times without time zones. And why is there a total seconds but no total minutes total hours or total days on time delta, about computing the number of weeks in a timespan? What if you wanted to iterate over the next 21 work days skipping weekends? All right, we'd better talk about time in Python. Good thing, Paul Ganzel is here. He's a core developer who controls time in C Python. This is talk Python to me, Episode 271, recorded June 18 2021. Welcome to talk Python to me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm at m Kennedy. Keep up with the show and listen to past episodes at talk python.fm and follow the show on Twitter via at talk Python. This episode is brought to you by brilliant org and us with our online courses over at talk Python training. Speaking of talk Python training, have you been thinking about taking one of our courses, we're participating in the latest Humble Bundle deal for Python developers along with a bunch of other great educators and tool developers. until July 22, you can get $1,400 worth of Python goodies, including three of our popular courses for just $25. Yeah, humble bundles are crazy. That's $25 for our three courses, and all those other things combined, just visit talk python.fm slash humble 2020 at talk python.fm slash humble 2020 all together before July 22. To take advantage of this offer. Now let's get to that interview. Paul, welcome to talk with me. Glad to be here. Yeah, it's great to have you here. Really excited to talk about the meaning of time. And all of that. Time is really tricky. Actually, time is one of those things that seems really obvious and simple. Until till it's not. And then until you start working with and you're like, Oh, actually that code running on the server and that server we forgot, you know, we didn't set the timezone to actually where it is. So maybe it's UTC. Maybe it's where the servers located or where our offices and all of a sudden you find yourself juggling all these things, and especially if you're building anything that has to do with calendars or stuff like that, that's going to be a real adventure. Yeah, well, I will say that I did not set out to become like the daytime guy. But I think that kind of complexity, like the natural complexity of the subject has either drawn me in or what's the word for when something gets you stuck, like in a quagmire, or like a trap or something. It's enticed you in and now it's got you trapped in its its world. Yeah, it certainly seems like a one way function like a something barbed where you can move in one direction, but not in the other direction. Yeah, for sure. Well, you know, it's really great that you're working on it. And just for the listeners, so you're a Python core developer working on basically time state times time spans that kind of stuff in Python. Yeah, I think I originally got into it with working on data tail. So I've been maintaining that for about God. Now. It's like six years. And you know, detail has a lot of little minor extensions. And then when Pep 495 came around, the core developers showed up in detail and said, Hey, we kind of use your library as a prototype for how this could work. So maybe you want to actually implement Pep 495. And then I did that. And then I had a relationship with the with the core developers. And then I just kind of stuck around and started doing more stuff. And eventually, it's one of those things like I never set out to do it. And I don't even have a job that's particularly daytime heavy. It's just like, if you start monitoring that the issue tracker for something like that you tell or even c Python, and then specifically just focusing on any one thing, like you'll just learn a stupidly large amount about edge cases and, and how everything works, I can see how that happens. And it's definitely one of those things that pulls you in. Before we dive too much into all that though, let's start with your story. How'd you get into programming in Python? I knew this question was coming. And I was sort of dreading trying to answer it. Because I feel like every time someone asked me how I started programming, I give them a slightly different answer. Because I've had, you know, quite a bit of false starts, you know, like I had, when I was a kid, they taught us basic, and I sort of remembered enough of that. And then so I didn't go to high school, but when I was sort of in place of high school doing went going to like a homeschooling center every once in a while. I guess I shouldn't say every once in a while I was regularly going to a homeschooling center. You know, I started learning c++. And then weirdly, my friend became close friends with someone who was what they called a wizard on a move which is a mud but object oriented and mud stands for multi user
05:00 dungeon, I think, you know, I don't know if the young people listening these days will know what a mud was. But man, muds? were super fun back in the early days. Yeah. See if you could describe a mud that people listening who have never played it. Yeah, so a mud is a text based game that you would connect to via like telnet or something. It's kind of like like soccer or some sort of if you don't know what a mud is probably don't know what Sark is, either. But you know, like, it's a text based game, where you describe, it'll say, you know, like, oh, you're in a room, you can go north or south, and you'll go north, you know, you'll pick up an object or something. So it's like kind of a fantasy game, or a Dungeons and Dragons type thing, where it's all text base. And then you can also talk to other people there. Yeah. And there's usually potentially hundreds of people all running around, they might be in these rooms. And you can have battles, you can go on quests. One of the hallmarks, I think, it's it's not super obvious what you can do. It's not like, Well, here are my five commands you'd like you kind of have to interrogate the game with words, to figure out like, oh, there's actually a rock and I could move the rock. I didn't know that. But now, you know, that kind of stuff. What I liked about it was it was, you know, as great as graphics are on games these days, it really just leveraged your imagination. And yet there were other people there. So I thought it was pretty cool. Yeah, well, so I actually got into that stuff. Like after the heyday of mods, like when there was like, not that many people in there, it was just my friend had this, like he was running, it was basically a chat server that you could just build whatever room you wanted. And then like, you could just directly program stuff. And it was interesting, because when you would actually play mud games in, say, the early 2000s, or something, a decent fraction of the people or like a much higher fraction of the people than you would think then you'd meet in your daily life. were blind, because that was like, the game was all textual. And so it was one of those things where you know, you're not really going to appreciate World of Warcraft or something. But you could go on a mud or something. And then it'll describe, it just describes everything in the room, and all that kind of stuff. So it was really interesting to see when, like, a decent fraction of the users are, you know, just interacting with computers in a completely different way. You see some interesting stuff. If I think about this too much, I'm gonna stop being the daytime guy. And I'm gonna just take up a second career, the accessibility? Yeah, yeah, yeah, for sure. Well, it is super interesting. I hadn't really put that together. But of course, because it's pure text, you can have a screen reader. And like, once you have the screen reader, it's pretty much on par with everyone else's experience. That's super cool. A cool bit in a soldier there. So you got into this move, somehow that led you into programming. Yeah, so it was a programmable move. So that was like, you could do object oriented programming. So I was just doing little move scripting stuff. And then later, I took up tickle TK, I had this job that I had these very long shifts, and nothing to do, it was just kind of sitting around doing nothing, and I could play on the computer. So I bought a laptop and just programmed a notepad clone. Then I went off to grad school did a lot of MATLAB and some C Programming. And then at some point, I was on some forum somewhere trying to make a point about something and I made a, I made a chart in MATLAB, and I posted it. And MATLAB has like terrible anti aliasing, or at least it did at the time. And so I was like, Oh, what is this alias nonsense, like, you should use matplotlib instead, so and my friend had told me, the one the same one who had the move, he had told me about this programming language called Python that was like, interesting. And you know, like natural language text, or something fewer curly braces. Yeah. So I just like was like, Alright, I'll give it a try. And then I really, really liked matplotlib, I rewrote my whole thesis, or I guess I just wrote my whole thesis, made all my charts in in matplotlib. And then when I went and graduated and got the job, there was this one day when all the MATLAB licenses were taken on the like, you had a certain number of site licenses. And if you ran multiple processes, it would take up two licenses. And I just couldn't do any work that day. So I was like, how about this, I'll just rewrite all of our MATLAB scripts in Python. And so then at work, I got to use Python. And then, you know, I started doing open source and I don't think I've used MATLAB since I left that job. Isn't it interesting how these draconian licensing systems like these license servers, and other super intense things that are like, just there to punish you, even if you follow the rules? You know, it's just, it's like, in their world, the worst possible scenario is that, in some edge case, some user gets to run 11 copies instead of 10, when they only paid for 10. Whereas, you know, they've lost all the goodwill, and they've actually pushed you to just say, you know what, we're just gonna go do something that doesn't work like this at all. Those are really interesting situations. That's one of the reasons I really like open culture is it just cuts down on friction so much like there's just so many times when it's like, oh, if I get this image from unsplash, instead of I stock photo or whatever, it's just, I don't have to think about
10:00 it, I can use it however, and that's why I tend to create a lot of open artifacts, you know, I, my blog is all cc zero, I'm just really trying to cut down on the friction of letting people use my work by you know, I understand, like, my business model for myself is to sell my labor day to day to big companies, I understand how it would, you know, if you are a company, you may want to sell software, I also totally love that I have to rent, you know, so many things by a cloud services. And I think part of that is because it's so easy to get around licensing restrictions. So I don't know, I'm of two minds, but I think wherever I can just cutting down friction by using free and open source licenses is is just just makes everyone happy. Yeah, yeah. Well, I mean, like, there could always, always be some sort of grace slack area, right, like, okay, you have 10 licenses, but we're not going to cut it off until you have 15 processes, right? Yeah. There's just like, when it's such a hard and fast rule, I don't know. It just cuts me the wrong way. Alright, so you talked about some of the stuff that doesn't work, you just tell and you talk to give me the sense that you kind of just doing some consulting and helping out various companies on various projects. But what do you do day to day? Oh, sorry. Yeah, no, I don't do consulting. I work for Google. So I'm more of a less of a philanderer and more of a serial monogamist. So gotcha. Yeah, I worked for Bloomberg for about four years. And now I work for Google in our team is called a core punch machine learning. So core punch is our internal infrastructure team that handles things like facilities and like tickets, like if you if you open tickets with people like HR, things like that, those are all within core punch. And we build machine learning models for other people in corporations for two, to provide them insights or to make the job easier, that kind of thing. So this is like the Google scale of automate the boring stuff to help you do your work a little better. Yeah, definitely. So, you know, my day to day work is playing around a Jupiter notebook. Well, okay, we're, everything in Google is like, its own proprietary thing, or even like, when it's not proprietary, it's still its own thing. So like, we usually use co labs, unfortunately, you know, I like to rag on people being like having their own proprietary versions of stuff. But like, every time I try and do that with Google, like they have like, stupidly good reasons. It just makes it so much less fun to make fun of our proprietary technologies. When, when it's like, oh, well, they do have native integration with like, Google Docs and Google Drive. So yeah, I guess it does make sense to use it. Yeah. Yeah, sure. I mean, there are a lot in the tech space in the developer space, certainly the not bill built here. So we got to build our own sort of obsession where people could take that too far. And it's just a just a drag, like, we don't need to build our own, like booking system, we can just get get a calendar system somewhere else and things like that. But when it's large enough scale, and it has like built in advantages. Sure. It makes sense. Yeah, I mean, I just wish that my colleagues were less competent, because then I can make fun of them. But you know,
13:06 I just have to respect them.
13:08 work environment is that
13:11 pretty good way.
13:14 This portion of talk by the enemy is brought to you by brilliant org. Brilliant has digestible courses in topics from the basics of scientific thinking all the way up to high end science, like quantum computing. And while quantum computing may sound complicated, brilliant, makes complex learning uncomplicated, and fun. It's super easy to get started. And they've got so many science and math courses to choose from. I recently use brilliant to get into rocket science for an upcoming episode, it was a blast. The interactive courses are presented in a clean and accessible way. And you could go from knowing nothing about a topic to having a deep understanding. Put your spare time to good use and hugely improve your critical thinking skills. Go to talk python.fm slash brilliant and sign up for free. The first 200 people that use that link, get 20% off the premium subscription, that's talk python.fm slash brilliant, or just click the link in the show notes.
14:07 Alright, let's talk about time. Every programming language has to do with dates and times. They all have their own way of doing it. And I would say pythons is pretty solid. Right? It's got a pretty good system there. So there's a couple of interesting things I guess about time one is it only goes back so far in computer time, right? So if you're talking antics, or things like that, so I don't know, what do you think about the daytime story in Python? Now that you're down, dude, I'm actually a big fan. I may not have a great breadth of experience outside of Python. But I think you know, the more I've learned about Python datetime The more I understand the choices that were made, like one of the biggest issues with the times is that you often see people like Oh, I can't believe daytimes are so terrible in Python because of XYZ. Like why is
15:00 And why isn't everyday time UTC by default? Like, why is it some other naive thing? Or? Or like, Why do I have to attach a different time zone to this or that. And the issue is that there are multiple camps of people who have equally strong, it should just be this way opinions, but in opposite directions. So most of the people think UTC is the natural thing to do. But I think those same people may not realize that when you're working with dates in the future, or you're working with civil times, in general, UTC is actually a very bad choice. Because generally speaking, UTC is good for, so I say UTC is good. And UTC is actually like the best that we have. I don't love that UTC has leap seconds in it, because it takes some nice monotonic constants that you can, you know, that you can use as a reference, and turns it into this weird thing with its own edge cases where sometimes there are 60. And sometimes there are 61 seconds in a minute very annoying. Really, I had no idea. Yeah, every fourth minute has an extra second or something. It can happen as many as two times per year. And sometimes they I think they'll either skip the 59 seconds, or they'll add one second, I think it's only ever that they've added one second. It's for the purposes of leap seconds. leap seconds are, I think, useful for some things like Astro navigation or something. But they're incredibly important for astronomers. I have no idea why they're built into the foundations of civil time. A writer can write does not need to know to the second when the sun is going to be overhead in Greenwich, right right here. Maybe it makes more sense to have like an Astro time that takes those like peculiarities into account right astronomy has its own whole big, long set of ways of doing timekeeping. Like they have sidereal times and solar time and, and I think they usually use the Julian calendar going backwards. I don't really understand exactly why they do all that stuff. But if you really want to dig into like weird timekeeping practices, astronomers are astronomers. And weirdly, astrologers, like something you may not know is that a decent amount of the like historical corrections? And actually, I think the original basis of a lot of the data in the time zone database that everyone uses came from astrologers, who were like, deeply concerned with exactly what second you were born at. So like, if you first certificate says us, like 315 and 14 seconds, in Yugoslavia, in 1981, they have to dig up the like the old Soviet era, Yugoslavia in Turkey timekeeping records to find out what that meant in UTC time, so they could find out whether Jupiter was in the house of Aquarius, or some, some crazy thing like that. That is crazy. I have no idea. If you don't cut me off on these things, I will just go on as many tangents like a weird, they're still involved in that I want to dig into something else. Oh, well, they just it's still important for them. So you'll still see corrections where they're very concerned with like, times before 1970, because the record keeping is a little shot here. And so the time zone database, you know, basically, they say anything after 1970, we try and keep it as accurate as possible before its best effort. So you'll still see people making sure they're like, Oh, well, the like, exactly how did the clocks move on the ground at this time? It's like the people who are on the timezone mailing list are basically a bunch of Unix people, the historians who are digging through looking at historical stuff, and astrologers, who are just weirdly concerned with executive The minute you were born Palin resi, I had no idea. Alright, so what I want to dig into, is this idea of naive time versus aware times, sure, you can do interesting math, like you can subtract to date times, but I don't think you can subtract like a naive time from an aware time and stuff like that. Like, you can't combine these things. Can you not, I don't remember, I feel like I've seen an error from that before those true. So it's changed a little bit between Python two and Python three, but not to the point where arithmetic is allowed between them. So in Python two, there was this idea that a naive time was essentially just a representation of time as it would show up on a calendar. So it's divorced from any notion of real time. It's an abstract, perfect daytime, that just is the proleptic Gregorian calendar. And an aware time represents a specific timestamp. So you can take something that's in UTC and you can turn it into local time in New York or in Shanghai or any, anything like that. You can get durations between things. So that was the distinction. And you know, the idea was that it's kind of like a naive day time was a daytime without a unit and aware daytime is putting units on it. So the number five is just a number
20:00 five feet is equivalent to one and a quarter meters or something like that, right? Yes, 1.6 or something. In Python three, it changed a little bit for reasons that are complicated and like probably not worth getting into. It's very complicated to create a time zone like a Tz info that represents local time and maintains all the semantics that you would hope if during an entire process where the local time might change over the course of the process. And so one of the things that happened in Python three is that now instead of naive times being unitless, they just represent local time. So they still act mostly the same way. But if you do something like you call dot timestamp on them, they will give you the converted to UTC and give you the number of seconds since 1970. January 1 in UTC, you can also do like as timezone and you can just directly, you can pass that a timezone it'll convert, it'll assume that the daytime represents local time, and then convert it into the timezone you care about. So at this point, local times are, are 90 times represent local time, which is why UTC now and UTC from timestamp are not functions you should use these days, okay, because they basically, they generate a naive timestamp without attaching UTC offset or like without attaching the UTC object to it. But they do it as if that represents UTC. So if you do UTC now, and then dot timestamp, it'll give you a timestamp representing a different time than now. Like I'm in minus eight. So it would subtract eight hours from it or something like that, or at eight hours. Rather, it'll play the plus or minus eight hours, but it'll change it in some direction. That's not just leave it alone. It was UTC now, so it just it is that time, right? Yeah, exactly. Okay. Interesting. Yeah. I always feel like daytime seems so straightforward until you get into here. And then it's just like, fraught with education. Like, wait a minute, what? I didn't know, I couldn't use UTC. Now, why is it still there, right, without at least a warning, it's even worse than you might think, to because the semantics of like subtraction are made even more complicated. And like the people who designed this library, like they weren't just like idiots, they actually were like, really thinking things through. And so most of the time, you won't even notice this kind of thing. But when you learn about it, it's a bit like makes your head explode. So there's subtract like subtraction works differently when you do it between the same timezone and a different time zone. So if you have two dates, and they're both in New York time, right, America slash New York, what it'll do is subtract them just based on the calendar, it won't give you the amount, the total amount of time elapsed between those even if like, so, if it's like, the day before daylight saving time transition. And the day, after 23 hours have elapsed between one o'clock and one o'clock the following day, it'll give you 24 hours between those two, if both in New York, okay, because that's how it works in the other direction. If you add 24 hours to something, it'll just shift over the calendar by one day. And I think the idea was that they wanted it to be the way it works in the other direction is you add one day, and then it'll create something that's one calendar day in the future, and then it'll attach the same timezone object to the new daytime. So if you then subtract it back, you'll get you'll still get one day. The problem is that, like, you can't just sort of shift around on the calendar when you're trying to do operations between two day times, like one in Chicago and one in New York. And the relevant like, the most reasonable thing to do is to just get the total amount of elapsed time, just convert them both to UTC and say how much time has passed, right? The thing is that the way they decide between inter zone and different zone like same zone, a different zone is whether the objects are the exact same object. So if you construct a new time zone object, if your time zone provider allows you to create a new time zone object representing the exact same zone, it won't check if they're equal, it'll just check if they're the same object. This is not really a problem with peitc. Because peitc doesn't use normal daytime mechanisms. And like, basically, literally everything you do will generate a new time zone object. So everything is an inter zone comparison. Yeah. And with detail, and with the new zone info module, I've implemented a cache. So basically, all of these zones are Singleton's anyway. So nowadays, it's basically like if you use the same ima key, you'll get same zone semantics. And otherwise, you'll get different zone semantics. It's more complicated than you would think. And it's really just all down to these overloaded concepts that are that are just everywhere in day times, like the idea of adding a duration is horribly ambiguous, right? If you consider adding a month, right, what does that mean? Does that mean 30 days? I mean, 31. Does it mean Give me the next the same day, the next month? What if it's January 31. Did you go to the end of February do you go to the beginning of March.
25:00 So, for two days, if you had one month, does it all go to the same day? Like, you know, you just ran like, do you just do one month mod however many days, the differences, like there's so many different options, you know, you kind of have to pick one. But it's very frequently the case that use, like different users will have different very strong expectations that it will do one thing or the other. And for something so low level, like the data module, which is you know, essentially, okay, something like a standard, right? It's trying to be basically a data type and interchange type. You just have to make choices that are hopefully going to be useful for the most the greatest number of people. Yeah, well, it makes a lot of sense. But, man, there are more and more edge cases, the more we talk, I'm like, yeah, that's, that's tricky. And you think about, so if I was in Indiana, for example, like certain places, the United States have different rules about following things like daylight savings, right, so Arizona, Indiana, a couple others, but I pick on Indiana because they've got a county by county system. So depending on which county you live in, like you'd actually let me just read this bit from Wikipedia, just so people know what I'm talking about. It says most portions of the state that were in the Eastern Time Zone do not observe DTS, daylight savings. However, Floyd Clark and Harrison counties, which are near Louisville, Kentucky, and Ohio and Dearborn counties, which are near Cincinnati, Ohio, unofficially observe it, because their proximity to these cities, which do observe it. So if I'm, if I have like today times, one in that county, or not in that county, right, it's in different time zones in different parts of the year based on like region, like, what do you even do that? Well, in the time zone database, the IMA time zone database is also sometimes called the Olsen database. What they have is that I mean, it's one of the reasons why names like us slash Eastern and us slash Pacific are actually deprecated. The way those keys work is that they usually go with a region slash the largest city, in the region that has continuously had used the same rules since 1970. So America slash New York is the same one that you use in like juniors in Boston, yeah, or in Virginia, anything like that. If Massachusetts switches over to being part of Atlantic Standard Time, then there will be a new zone created called like America slash Boston. And it'll cover Massachusetts going back all the way in 1970. In Indiana, they have their own like sub key. So they have like America, Indiana, Indianapolis, and then like five different states there. I think Mountain Time has a couple things like that, as well, like Denver and Phoenix are separately are handled separately, I see seems to be very precise about your your timezone info passed over. But then it'll actually take care of that. Yeah, it definitely handles that in terms of historical times, like this gets to an issue that it's another one of these overloaded concepts, which is like when we talk about a date time for some people, like if you're looking at server logs, people are like, why would you not just use UTC all the time? It's unambiguous. Yeah. But those people are not implementing, say, a scheduling surface, right, where, you know, you and I scheduled to meet at a specific time to record this podcast, right, if someone had changed daylight saving time, like if between the point where I scheduled it. And today, the United States said, we're in so many crises right now that we're just gonna, like, switch off daylight saving time. I don't know why politicians love messing with time at the last minute, but they absolutely do. If someone had done that, then we would still want to meet at four o'clock, because relative to my work schedule, and your work schedule, and everyone's school schedules, that's still the right time for us to meet. But the mapping between four o'clock and UTC will have changed. So what that means is like if we had recorded that as saying, Okay, this is going to be as a daytime guy, I should always be embarrassed by this by can never do any the same time calculations in my head. Hey, we have computers for this stuff. Khan. Yeah, it's like eight o'clock UTC is when this recording is gonna happen. If all of a sudden the mapping between four o'clock and eight o'clock changes you and I will be very unhappy that they stored it in UTC and then converted it before showing it to people. So when you're dealing with the distinction I usually make is timestamps versus civil times. So a civil time is like when you actually care what the clock on the wall says, then you should use a civil time, which is you store exactly what the clock on the wall says. And then the timezone you started in American New York, something like that, right? This is to get back to this point about what you did like how you store it. Right now, there's no way to solve exactly that problem, or there's not a great way to solve exactly that problem. Because what you really care about is like a little hard to pin down because, you know, I'm in New York, you're in Portland, right? Yeah. So you know, if one of us changes our timezone, if our relative time zones change, New York changes its timezone offset, but Portland doesn't which
35:00 rewrote the API, and they switched it to TypeScript. So I got to start over, or, you know, like, pick your framework does your, like the stability of those things, or the lifecycle of those things is so short. And, you know, the fact that you don't have to keep changing because your dependencies or whatever changed. That's like dependency in Python, potentially. That's a really good feature, I think. Yeah, I said, I think sometimes also, things have a short shoot tend to have a short shelf life in some situations, because because people want to break a whole bunch of stuff. And I guess I'm just glad that we have maintainers around who are willing to kind of put in the work for the long haul to write make it so that, you know, hey, this thing is still supported. And you don't have to migrate to a new framework every 30 minutes. Yeah. So one thing about daytimes that I want to ask, sorry, time deltas that I want to ask you about, because it drives me crazy, is I find myself doing math, way more than I think I should with time deltas. For example, if I have a time delta, and I want to know how many days that that time delta represents, that's a math problem for me, right? It has a days operation days property or field, but it gives me unusual answers in some sense. And not in a fraction of days, right? So if I have something that's 30 seconds, I would like it to be whatever that is, like, one 100th of a day, or I would like to be able to ask, how many hours are there? How many weeks, you know, starts to get hard. Okay, how do I do it? Because all I know about his total seconds, for example. And then there's seconds, and then there's microseconds and there's days, those accessors like the days seconds, microseconds thing, those are actually he timed out that is essentially supposed to be like a struct type object that just like holds a time delta, you know, and it it has three canonical components, which is days, microsecond, days, seconds and microseconds. I understand that it's confusing that those accessors are like not the total number of days, the way you're actually supposed to this was added in Python three, the way you're supposed to get this information is timed out. This can just be divided by other time deltas. So if you just have like a time delta, and you want to know how many days are you just divided by time delta days equals one? Oh, interesting. So instead of going total seconds, divided by 60, divided by 60, divided by 24, I would say DT divided by time delta days equals one. Yeah. And you can just have any arbitrary value as the denominator there. So it's actually this is one of those situations where it's like not super discoverable. But it's incredibly elegant. Yeah. And like you can the fact the time that total seconds even exists is a problem. Well, it's not a problem. But it highlights this fact that it's not super discoverable. Because yes, the Yes, when they added total seconds, they had already added the ability to divide by they had already solved this problem by adding the by defining Dunder div. Yeah, and one core developer didn't know about that, and added total seconds. And then the guy who was, you know, maintaining daytime showed up after it was all merged and stuff. I was like, Oh, I wish I had seen this before. I would have said, Please don't add this. Yeah. So you know, every once in a while, we get people who show up and they're like, Can we get total microseconds and total, whatever it's like, we don't want to make 40 different, you know, things to say, quarters and months and years and things like that, like just, you just have to divide it by time delta. And it works super well, especially now that increasingly you are people are working in Python three only code bases. If you have a two three compatible code base, you generally do have to like do total seconds, and then divide that by total seconds of whatever your time delta is. Gotcha. Okay. Interesting. Yeah. Cuz I was hoping that would be a total minutes, total hours, total days. And it could just do that math that I talked about, just in line, right in like two, so I don't have so it could be more expressive, and not do like a long series of divisions, you certainly can do that. Like, I can understand why you would want that, I suppose. But then, you know, you only have the enumerated units that that are in there. And it also sort of hides the fact that you can just do regular division. I mean, you don't have a dozens method on on integers to find out how many times 12 evenly divides in it. I you know, it's a fair point. I think that's the discoverability. That is the challenge, right to know that you can actually do the division and because now you say like, sure it makes total sense to say, I want to know how many whole weeks are in there. So I would say time delta days equals seven and then divide it and that would give me a great answer. Right? I can pick my units to be whatever. You can also do weeks equals one. Yeah. Yeah, we see. Although I'm sure there's a there's a lot of flexibility there. But yeah, it's just I don't see a lot of examples using that, you know, when you look around a code stuff, that's partly because people don't see it and partly because anything that a lot of the code that we look at now, especially open source code has been to three compatible for like the past five or 10 years or something. And that idiom is not available in Python two. So 531 you
40:00 Python three only, you know, it's kind of like you're starting to see f strings everywhere. Hopefully all the cool kids now are just dividing their time Delta's Well, they will be now now that they hear about it on the podcast. Yeah, I thought you were gonna say the thing about how the rapper and the the stir like the string cast have timed out. This is just absolute trash. Like, I really wish I could fix that. I think it's going to be difficult to do so. But it's like, if you print the time delta that is negative one hour, it tells you one day minus 23 hours or tells it to you in seconds. Yeah, the internal reputation is weird, like so for I do that for one that is one second long. It's negative one day, 23 hours, 59 minutes and 59 seconds. It has like a negative one second or something in it. Yeah, it's, you know, it computes to the same, but it's not very human. Yeah, I would probably do it a different way. I probably do it as like negative and then abs, like an absolute value. I think they they're worried about some ambiguity there. But I don't think anyone ever wants what we got. So yeah, you've given like 99.99% of people something they don't want to avoid a tiny bit of possible ambiguity, right? Yeah. But what are you going to do? Well, so let's talk a little bit about some of the peps here. So you've got Pep 495. And that one is that one already done, that one's already done. This. So Pep 495 was implemented for Python 3.6. And that is the fold attribute. The background of this is that prior to Python 3.6, timezone support was not super great. Basically, whenever you have an offset, change that where the offset decreases, that means that you are jumping back by an hour or by any amount, and that creates an ambiguity, right? So it's two o'clock, and then you jump back one hour, hopefully, it's two o'clock, I actually found one place in, I think, in Argentina, in 1992. Their Daylight Saving Time transition happened at midnight on February 29. Because wasn't bad enough to do it at midnight. Without integrity. They also had to do it during Leap Day.
42:06 So Oh, my goodness, yeah. So it's like, anyway, so you do something like that, right. And so instead of it being February 29, or March 1, it's now February 29. And at 11pm a second time. So the way timezone handling works in daytime is, it's actually quite elegant, the ideas that you attach, because it timezone represents a set of rules for like taking the current for mapping between absolute time, like time in UTC and local time, like what time does it say on the clock. So I think the original design intent was that you would have a Tz info. And the Tz info has these functions that you can call that give you information about the time zone as a function of the tick time. So it's super great, you just write this function that map's your date time to UTC, and you're done, except that the date time portion is sometimes ambiguous, right. So if it's 1992, and it's 1130, in Argentina, on February 29, there are two different offsets, there's two different sets of information that you could apply there. And there's no function that you can write that just takes the naive portion of the date time, and gives you the correct offset, because it's ambiguous. So the way peitc solved this problem was that they added this localized concept where what they do is they you create a time zone, and then instead of just like attaching it directly, you have to call localize on it. And what peitc will do is it'll just attach a dedicated fixed timezone offset. So it has a little switch in there where you can say, Do I want to be in the daylight saving time side, or the other side, this is a major source of bugs, because people will just do what the doc say, and attach pi Tz time zones directly to the Tz info instead of letting peitc do it itself. And also it kind of circumvents a lot of the the other logic in there about a lot of assumptions about like, you're just supposed to have this one set of rules, not a fixed set of offsets. And then it's also a bit of a problem because they specifically use DST, but there are situations where you have transitions that are not due to daylight saving time, like someone just switches their base offset or something. So that's where Pep 495 came in. They what they did was they said we're just going to take the knife portion of the daytime, and we're going to add an additional component to it called full. And I can take two values zero or one, the zero represents the first the offset that happened before the transition. And then after the transition, you put fold equals one. It's very elegant. It allows you to write time zones that just take the knife portion of the day time and then just flip a switch if it's ambiguous based on the foot, okay, yeah, that's, that's cool. Yeah, just catching these little edge cases. And then pepp 615 you're working on as well. This one's in three, nine. So I guess in a sense, it's accepted because three nines
45:00 It's beta world, yeah, it was accepted and we merged it. So it's going to be, you can get it in the betas now, you can actually get it, I have a backport of it implemented back all the way to Python 3.6. It works in pi pi, I've tried to make it, I, you know, my goal here is basically that I would really, really like it. If people could stop using pi Tz, I think the maintainer ficc would also agree with me on that, it's basically Pisces, he was sort of a hack around this issue, they did a really great job on it. Like, I think, I don't know that there's a better way to have done that. But well, you know, other than just coming up with this fold idea yourself. And that has its own problems, we in detail, I also have a backport of pet 495 that works in to seven. So even if you have a Python 2.7 implementation, with some combination of the zone info module, which was added in pet 615 and date you till you can actually get proper Pep 495 support all the way back to Python two, seven. And, yeah, that, you know, there's a bunch of reasons why you should use zone info. One is obviously the you know, I you know, I think it's enough of a selling point to just say you don't have to do this localized normalized stuff, like you can just use time zones like they're meant to use, and it'll just work. But there's also the fact that peitc doesn't have any support for the version two TCI files. So the IMA the compiled versions of the IMA timezone. They're all version two or three now, like since 2005. And version two and three is when they switched from using four byte integers to eight bytes. So each timezone now starts with version one file, like the entire version one file, and then after that, they have the version two file. So all the stuff that supports version one files only still works with version two and three files. But version two, and three files will go past 2038. And they instead of just like, stopping at some point, they also have like a general rule. So they'll say like, here's the rule for transitions after 2038. Right? We don't have data for this, but here's how it has been. So just keep competing it like this. Yeah, I mean, the further into the future, and actually, the further into the past, like the the distant past, like just like what was the time is significantly less meaningful question to ask. So right now, you know, what timezone, like, what is the UTC for New York in 2040? is not a terribly like gazing into a crystal ball, right? It's right, you're gonna have a guess it's probably a better guess, than randomly choosing, but it's not going to be accurate. Like, it's not, it might be a bit of an x y problem, if you're asking that question. But you know, it's still like, at least it'll do what you expect. But as time goes on, in the next 10 years, in the next 15 years, we're getting a lot closer to 2038, you're going to need that kind of support, right. So if it doesn't have support for that doesn't have support for sub minute offsets. Another thing that zone info does and date util those but peitc does not do is that they will hit your system timezone information first. So there's a way to configure it, they have their own access to time zone, they have their own time zone data, in a sense that you can get if your system timezone data if your system doesn't provide time zones. And there are mechanisms for specifying where you want to get the time zone data from. But with peitc, it shifts the time zone data with it. So you have to keep your peitc up to date, just to get the data updates, there's a way to change that where I think you like set an environment variable and tell it where your system time zones are. And it will use use that instead of its own thing, but you have to know where your stuff is. Whereas with zone info it's built in. So it'll find it in the most common places. And then there's a mechanism for the person who distributes Python to you to tell the compiler where it's going to be. So if you use some system Python, it'll always hit your system timezone. Which is very helpful. So you know, there's all those reasons to not use PI Tz. Yeah, there's still reasons why you might be worried about switching over like, for example, since pi Tz has its own weird interface, you know, there's going to be a lot of your users possibly who've taken your your Tz info, and they've called localize or normalize on some date time. And if you change over from pi Tz, it's going to break all of them. But I am aware of this kind of thing. So I've also created a library called pi Tz deprecation Shim, which is just a thin wrapper around zone info, except that it has it supports the PI Tz interface as close to the semantics that pi Tz has as possible. And then it just raises a warning if anyone's doing anything peitc specific. So it just works. Let's get a regular 495 thing. It works in two, seven and three, six onwards and if anyone's like that
50:00 Using localizer, normalizer, whatever, it'll raise a warning. So you can basically say, oh, for the next two years, I'm going to depend on peitc deprecation shin. And then I'm going to drop it in favor of zone info. So and hopefully, everyone has not been ignoring all my warnings. Yeah, that's good. It's really nice to have that warning there. So you, it gives you some time to deal with it. All right, Paul, we talked about a lot of different things. A lot of interesting edge cases and scenarios. Maybe give us the the main takeaways, like if you could get people listening some advice on the please do and please don't with time in Python, what would you say? Yeah, so for don'ts I actually have a couple of blog articles about this. Like, one of the things I see the most often is people using UTC now thinking like, Oh, I want something in UTC. But UTC now and UTC from timestamp are not something you should be using in modern Python, I would like to remove them. But they're like, occasionally, these situations where they're useful, like if you're immediately trying to, if you're just using UTC now to and then dumping it right into ISO format, and you don't want the like the UTC offset in there, right, it's not a super great way to do that. I might just bite the bullet and deprecate those, and then maybe, you know, have a wrapper function for it in date your tail. But for the most part, you don't want to use UTC now, you should use just daytime now and then pass it the daytime, that timezone that UTC object, which is just a UTC Singleton that's in the standard library. Also, I don't think you should use PI Tz, take all your itG code, and migrate it over to zone info or detail and start getting on the pet 495 bandwagon. And I've got a little migration guide for that. Nice. We'll put the links to those in the show notes. Now, I just want to do maybe acknowledge a couple of libraries that are on the outside of C Python, but still pretty helpful. And the first one actually is Python dash detail. That one's yours, right? Oh, yeah, so I didn't write that, but I've been maintaining it for about six years. That's got some interesting stuff. Even after Pep 615 the timezone module does still have some useful time zones in it, like it has, well, you know, I say useful, I, what I mean is like, available, like different types of time zones that are not in the standard library. So like, there's this Tz iCal if you have time zones in this icalendar RFC format detail will handle that. And you can also like us like these POSIX strings, like if you ever see like ESP five EDT, and then like a, there's a POSIX standard for expressing recurring DST transitions. So it has that and some some windows specific stuff as well. You know, what I use it for? that I really like is it's part it's Parsi. Yeah, really good. Because it seems like it generally just handles time strings, which is great. Yeah, the parser is the part that I like the least. But that's mainly I mean, because I have to maintain it. And it's got so many rules that it's super easy to break. So anytime someone shows me something where it's broken, it's like, I'm really afraid to change anything, because if I do, I'll break someone else's use case that they've been using it for, like 10 years. It's also like, pretty slow. So it's super useful if you have something that you know, is a date time. And it's a pretty standard format. And you just want it to like, chew at it until it finds something that looks like a data, but it could be chewing for like 200 microseconds. If you have something that's in ISO 8601 format detail also has ISO parse, which will handle anything that's in a valid ISO 8601 format. And then date time, I think the first thing I added to C Python was from ISO format, which is like an inverse of ISO format. So okay, that's cool. That's on daytime, it's got our rule details, or has our rule for recurrence rules, which are very useful is relative Delta, if you want to do time deltas, but include years and months and things like that. And the best one I saved for last is Easter. It just tells you when Easter is you give it a year. And it's like this is when Easter is Oh, that's cool. Yeah, there's a couple of different holidays and stuff that are all sorts of all over the place that
54:11 that are tricky, like Hanukkah and stuff. So the two others that come to mind and I'm sure there's like 10 additional libraries that people are out there. Let's see got it. But but this one, I just don't have them off the top of my head. One is called arrow. Are you familiar at all with arrow? Yeah, I've contributed to arrow a couple times. Yeah, it's pretty cool. It's some of the things I guess that stand out to me like that caught my attention the most is that it has humanized so you have a time you say dot humanize, it'll say an hour ago. You say humanize an hour, the locale is Korean or something and then you get the Korean statement of an hour ago and so on. And that's pretty easy. Pretty nice. Yeah, I mean, so for stuff like arrow and pendulum and Maya and DeLorean and all these things. I've never found a huge amount of use for them. But I'm not like a huge like, I don't actually like I don't know that
55:00 I have the exact use cases, I tend to write libraries and taking on big dependencies is not the easiest thing for me. But I do know that people kind of swear by these libraries, they really like them. So I think that there's certainly something to them, the biggest problem I have with them is that they tend to be, they tend to try and advertise themselves as like dropping replacements for date time. And so they then subclass the time and time delta, stuff like pendulum advertises itself as a drop in replacement, but then it breaks all the stuff that I want to break, you know, so it's not actually adopt proper replacement, you can't even construct it the same way. And they tend to be slow as well. I mean, this is partially because they're not written in C, or not mostly written in C. But you know, to do simple time delta arithmetic in pendulum is like, 10 times slower than doing the same thing in, in C. But you know, if you're not handling a huge number of daytimes, it's probably fine. That of course, you know, we don't have human eyes and things like that in the core, I think that those are very useful things. I would tend, like, I would love to see a library that is kind of like pendulum or arrow, but instead of it being methods on datetime, subclass. It's like, just free functions, you know, utility functions, right? You could pass it as the built in items. Yeah. Yeah. I mean, I think that would be faster. In a lot of ways. Sometimes people don't care about fast. Yeah, yeah. Well, if you're building a library, especially if you're building C, Python itself, it's a different level of concern. Yeah, if I'm rendering a webpage, and I've got 10 items, and I want, like the human expression of how long ago, they were, I probably don't care. Plus, you can benchmark, I can sort of, you can benchmark and see where the bottlenecks are, I can benchmark everything on pi pi, and everyone's proprietary code bases and find out like, you know, do some sort of optimization and find that it's, this is not the bottleneck anywhere that this code is being used. So Right, exactly. It's a different scenario. And yeah, I do find that just using bare daytimes is useful for interrupt, but if you're mostly working in your own stuff, like you're just passing it around doing stuff with it, they do have some like, you know, people do, like, you know, they really do. Yeah, like the the interfaces, they like that the time delta things that they give, you have total minutes, and they have human. Yeah, they got like weeks and hours and stuff. The thing that stands out as most unique about pendulum is they have a duration period thing that is aware. So I could have a date time span, the subtraction between two day times their equivalent of a time delta. And then you can say, in weekdays, and then for each thing in the period, and it'll give you day one day to skip the weekend, day three. So, you know, Thursday, Friday, Monday, Tuesday, as I think you can go, it's like, well, there's, it's as long I need to know how many weekdays there are, how many weeks or something in the span, that that's an interesting concept. Like in detail, we have this rule, which is based on a data type described in RFC 5545, called the recurrence rule. And it is super, like, it's a little more complicated than that, you know, again, this is why I think these sorts of things could be done as free functions. It's very interesting. And like, it's a little too complicated for the average person to just sort of jump right into it. But you can express all kinds of stuff like that and say, like, how many Tuesdays are there? Like one of the talks I give on detail? I think I say like, How many times did like the guy who played Mr. Miyagi Pat Morita, like, how many times his birthday fall on a Tuesday during the Vietnam War? You know, and I have like a simple rule that is like standardized, you can cast it to a string and pop it into Google Calendar. And I can just easily do that with an AR. I say easily, but that's after maintaining an oral library for you. Right, like, you're right. It does. I don't know. It's like saying you can find that with a regular expression. Yeah. Right. Yeah. I'll use easily Java radio expression, you can easily do that other expression. I will struggle with that regular expression, but it's fine. But that is powerful. It's cool. All right. Well, I think that's probably it for all the time we have. We've run out of time talking about time, but there's been a lot to talk about. It's very interesting. Thanks. So before you go, though, let's let me ask you the final two questions. If you're going to write some Python code, what editor Do you use, I usually just dictate that to my secretary. And
59:24 I usually use vim. I don't feel strongly about it, though. So if anyone wants to, well, don't try and convince me about it. But also don't see that as an attack on your Emacs using ways like I just picked one at random. Sure, sounds good. And then notable Python package. There's one that's really fun, but please don't like if you install it, do it in a virtual environment or something. It's called a break my Python, you pip install it, and then it just makes it so that your Python is broken. Like you can't, you don't have to import it to to have your Python be broken. You just do pip install break my Python and if you type Python, it won't work anymore. It'll just break that's break down.
01:00:00 My dash Python. Alright, so that's kind of an anti recommendation. That's interesting. Very cool. Funny if you want a real recommendation of something that's like fun and useful to use, I actually do like PIP x. Oh, yeah, Pip x is cool. I love that. But at this point I my main Python installation, my main Python installation stuff is like super layered with all kinds of indirection. So like, everything is all Pippin or pi in environments. So like, I have 10 different pythons installed. But even within those, I never pip install anything directly. It's either like working in a virtual environment, or if it's tools like talks, or virtual env, or something like Pep 517 Well, actually, that one doesn't really work like glances, or PT, Python, or any of those types of tools past my PI. Yep, yep, pi linked, all that stuff. You just do PIP x install, whatever you do, like you're going to pip install, but what it does is it will install each one of these tools into their own virtual environments. And then just put on your path, the entry points that they expose. So you do PIP it PIP x, install my PI. And then if you try to like, if you open your Python and try and import some my PI like thing, it won't be there. But if you just type my PI, it'll work as if by magic. And this avoids the whole situation of like, different tools, stepping on each other's dependencies, or needing to update all that stuff. It's really great. Yeah, I really like it. To me, it feels like homebrew, for Python utilities and applications, right. Like on my Mac, I can brew install MongoDB, I can brew install FFmpeg, all these things that I might need, and they'll be installed separately maintained. And you know, when there's updates, Pip x is that for things that are tools based on Python that you would pip install them. It's really nice. Yeah, I mean, it's a brew also, kind of maybe has that same problem, right. Where, because it's, it's like a system package manager. I mean, they handle it for you. Yeah, it's a little more Yes. is less is less isolated. Yeah, I agree. Yeah. I mean, as long as you're not using your your brew Python, right, like, if you don't, it's like open up Python that you brew installed. Exactly. Cool. All right, we'll get back to the fun recommendation, a hand break. My Python is a good joke as well. All right. I'm looking forward to dig into that to see how it actually breaks your Python. All right, well, thank you for being here. final call to action. People want to get their time zones and time in order. I mean, obviously, the recommendations, you have the do's and don'ts stand and we'll put a link to those anything else you want to leave people with? I think it's pretty good. Like I think I'm very interested in getting people to migrate over to zone info. So especially right now, when there's not a huge number of people knocking at my door to ask about that. But feel free to, you know, open some issues on there, or send me an email or add me on Twitter or something. And yeah, like, get out there. And you know, maybe like tweet, or Instagram or Tiktok, or whatever the kids do these days, tell your friends. But how great Pep 615 is, and hopefully we get the word out. But you don't have to use peitc anymore. Yeah, absolutely. Well, thanks for working all this stuff and handling all these edge cases for people. That way we can mostly ignore them, except for when we can. Yeah, yeah, thanks. See you later. But
01:03:07 this has been another episode of talk Python. To me. Our guest on this episode was Paul Ganzel. And it's been brought to you by brilliant org and us over at talk Python training. brilliant.org encourages you to level up your analytical skills and knowledge, visit talk python.fm slash brilliant and get brilliant premium to learn something new every day.
01:03:28 Wanting to level up your Python. If you're just getting started, try my Python jumpstart by building 10 apps course. Or if you're looking for something more advanced, check out our new async course the digs into all the different types of async programming you can do in Python. And of course, if you're interested in more than one of these, be sure to check out our everything bundle. It's like a subscription that never expires. Be sure to subscribe to the show, open your favorite pod catcher and search for Python. We should be right at the top. You can also find the iTunes feed at slash iTunes. The Google Play feed is slash play in the direct RSS feed at slash RSS on talk python.fm. This is your host Michael Kennedy. Thanks so much for listening. I really appreciate it. Get out there and write some Python code