#324: Gatorade-powered Python APIs Transcript

00:00 Python is used to solve a large and varied set of problems.

00:03 One of its core pillars is web APIs.

00:05 Another one is ML and data science.

00:08 Those two important pieces were brought together in an unexpected and yet magically futuristic

00:13 way by Rod Senra's team working with the Gatorade Sports Science Institute.

00:18 They created a patch that you wear while working out once or twice.

00:22 It analyzes your perspiration, combines it with other factors like running distance, sleep

00:27 quality, and more.

00:28 Then it provides recommendations using Python about how to get more effective fitness.

00:33 This is Talk Python to Me, episode 324, recorded July 8th, 2021.

00:38 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the

00:56 ecosystem, and the personalities.

00:58 This is your host, Michael Kennedy.

00:59 Follow me on Twitter where I'm @mkennedy.

01:02 And keep up with the show and listen to past episodes at talkpython.fm.

01:05 And follow the show on Twitter via at Talk Python.

01:08 This episode is brought to you by Sentry and Linode.

01:12 And the transcripts are brought to you by Assembly AI.

01:15 Please check out what they're offering during their segments.

01:17 It really helps support the show.

01:19 Rod, welcome to Talk Python to Me.

01:21 Thank you.

01:22 Super to have you here.

01:23 There's a lot of things you've been building for different companies through your work.

01:28 That's going to be really fun to explore.

01:30 I think one of the really neat things here is you're not just working for one company, working

01:35 on one team the way it works, but you're kind of interacting with a bunch of different projects

01:40 and types of technologies.

01:41 So you'll have a good, broad perspective on what went well, what didn't, what you would

01:45 change, and so on.

01:46 That's true.

01:47 Python is the one thing that stuck with me since 1997 across different companies, let's

01:52 say like five different companies, two different continents.

01:56 But Python was the one thing that remained the same.

01:59 That's fantastic.

02:01 A good choice indeed.

02:02 Well, let's start this conversation by talking about your story and how you got into programming

02:09 Python.

02:09 Python, it sounds like you were one of the early adopters.

02:12 What version of Python?

02:13 That was probably a version one type of thing.

02:15 It was 1.5.2.

02:17 It was in 1997.

02:19 At that time, I was almost finishing my undergraduate course in computer engineering, and I already

02:27 engaged in my master's post-graduation course.

02:30 What were you studying?

02:31 What was your degree?

02:32 Well, my undergraduate degree was computer engineering.

02:35 But for the master's thesis, I was studying computational reflection.

02:39 In the end, it's the line of research that led to Aspect J and other things.

02:45 But it was all about object-oriented protocols.

02:48 So how you organize object orientation in programming languages, those kind of things.

02:53 Okay.

02:53 Cool.

02:53 And Aspect J, that's like aspect-oriented programming?

02:57 Exactly.

02:57 Aspect-oriented programming was like a byproduct of the research in computational reflection.

03:03 At the time, we were, academically speaking, we were trying to figure out what can you do

03:09 with reflection.

03:11 Java was one of the languages that made it popular to do introspection and reflection

03:17 in programming languages.

03:18 Of course, we had that in Lisp since back to the 70s.

03:22 But Java made it known to the wide world world.

03:26 So then we became researching more formally.

03:29 And I started probing supports for reflective programming in programming languages.

03:36 And that meant not only Java, but at the time, Perl, Python, Tcl.

03:40 And once I saw Python, it felt like a glove to my hands.

03:45 And then I stuck with it.

03:47 Yeah.

03:47 Oh, that's really neat.

03:48 Maybe you could just talk a little tiny bit about what reflection is for people listening

03:53 who maybe haven't done a lot of Java or .NET.

03:57 They have it as well.

03:58 And in Python, we have it in the sense that you can go in and explore the types.

04:03 And so, you know, give it an object.

04:05 What if it's class?

04:06 And like, what is it?

04:07 It's meta class.

04:07 And you can like dig into it and even change it.

04:09 But I don't typically hear it referred to as reflection in the Python ecosystem.

04:14 Right.

04:15 Yeah.

04:15 Introspection became a more common term used.

04:19 But at the time, reflection was about creating programs that reason about programs.

04:26 So it was about this meta level reasoning.

04:29 And it was very popular to do non-functional aspects.

04:32 So you want to write your code and the code has the functional aspects, what it do, how it

04:39 like transform things.

04:40 And then the non-functional aspects like logging, persistency, monitoring, those kinds of things.

04:47 People were exploring how to detach, decouple those things from your actual program and do

04:53 that on a separate layer.

04:55 And that was what reflection was all about.

04:57 It was about having as inputs your own code and as outputs changing the behavior of your program.

05:05 And the way we attacked that at the time was by an interception mechanism.

05:11 So Python always had that.

05:13 I think they called the DOM buildry hook or something like that, where you could like intercept

05:19 like anything that was happening there.

05:21 We do have the bugging hook in the Sys module that allow us to like pause your computation and

05:29 see what's happening.

05:30 Other programming languages.

05:32 And at the time we're talking about the 90s, the popular ones were statically compiled

05:38 programming languages.

05:39 Right.

05:39 C, C++, they generally don't have this behavior.

05:42 Yeah.

05:42 Sometimes they had runtime type information when you compiled your source codes enabling that.

05:48 That's right.

05:48 I remember there was an RTTI.

05:50 RTTI.

05:51 And C++, but it was off by default.

05:54 Exactly.

05:55 But in the interpreted world, you had everything, right?

05:58 You had the interpreter during runtime.

06:01 Sometimes you had the compiler as well, which was true for Java, which was true for Python,

06:07 Perl, all the interpreted dynamically languages.

06:11 And then that became more of a thing.

06:13 Of course, in time, people realized there were other ways to achieve the same goals, which

06:20 were more effective because one of the main things with introspection and interception

06:26 is that everything that is happening in your program, it's kind of being reified, delivered

06:33 to this meta level and then handled as data.

06:36 So that is like a major break in speed, right?

06:42 So it was not the answer for everything, especially when you wanted to have like runtime performance.

06:49 But for things that were offline batch or analysis, it was not a problem.

06:53 And one thing that an exercise that I'd done at the time, which is kind of interesting, was

06:59 to introduce a debugger during runtime.

07:02 So because we had this reflection mechanism that works just like this, you could take possession

07:09 of an object and then any interaction with these objects would go to its guardian angel,

07:15 which I call the meta object of this object.

07:18 And then the meta object decided what to do.

07:20 To lie about what was happening, to just delegate back to that object down there, do something else,

07:28 become a proxy, solve the computation.

07:30 So you could do anything.

07:32 And in that case, we could install a debugger just for a given class or a given instance of your program,

07:40 and then propagate like a virus for anything either that class or instance touched.

07:45 So that was kind of interesting, a new dimension of programming, but became more as an academic exercise.

07:51 And the only thing that I remember that's kind of succeeded into the industry was aspect oriented programming of sorts.

08:00 Well, that's very interesting.

08:01 It seems to me a little bit like decorators.

08:04 Yes.

08:05 Decorators are wrappers.

08:06 But the thing is, they are not transparent.

08:10 So whenever you create a decorator or a proxy, what's happening is that the references you had to the original objects now need to point to the wrapper, to the decorator.

08:21 Right?

08:22 So the key insights for reflectional computation that we were exploring at the time was how do we make this proxy totally transparent?

08:31 So if you have a pointer or a reference to an object before it became reflective, it would still be valid.

08:41 So you can turn on and off those introspection mechanisms and interception mechanism, and it's totally transparent.

08:48 So to achieve that, we had to change the interpreter.

08:52 So that's how we've done it for Java.

08:54 And that's how I was exploring at the time for Python itself and Perl, to change the interpreter, to add the hook in there and do transparent wrapping.

09:04 Yeah.

09:04 It seems like some of the new peps might make it possible to plug in now rather than actually change it.

09:10 I know there's that JIT plugin that they're working in where you can intercept the method parsing compilation bits and stuff.

09:17 I don't remember the number.

09:18 Yeah.

09:18 I haven't explored, like, doing it to modern Python.

09:21 At the time, I did some exercises using the debug hook.

09:26 Yeah.

09:26 It kind of works, but it was kind of messy and super slow.

09:29 So I kind of stopped.

09:31 But it will be interesting to revisit that again.

09:34 I'm sure.

09:35 I do remember how interesting all the research was around that time.

09:39 And I feel like a lot of what's happening now is it's getting a little more structured, a little less dynamic with things like decorators and other types of wrappers and inner functions and stuff.

09:49 But yeah, quite an interesting time.

09:50 So how about now?

09:52 Maybe we can talk about what you're doing these days.

09:54 Nowadays, I'm a technology director at Working Co.

09:57 Working Co.

09:58 Working Co.

09:58 is a digital products company where we not only do designs and strategy, but we also do implementation and quality assurance for the projects that we take on.

10:11 So that's what makes us different from just an agency, a design agency, because the founders of Working Co.

10:17 realize that it's much better for the clients to hire a single entity to do the full digital products turnkey.

10:26 And then we can make sure that whatever we design, it's possible to build.

10:30 So that is the vision for Working Co.

10:32 We were very successful, I must say.

10:35 In a few years, Working Co. won many prizes.

10:38 It became famous for the Virgin America app for booking passages with Virgin America, I guess, five or six years ago.

10:47 But we're doing things for like Apple, Google, Chase, you name it.

10:52 I know that involves a lot of Python.

10:53 Does it involve other technologies?

10:55 It's worth it.

10:56 Probably other technologies as well, right?

10:58 Somebody comes and says, well, we want this thing designed and four of our Java shops don't make it for Java.

11:02 Whatever happens is usually there are projects that are just designed, projects that are designed plus front-end work.

11:08 Like projects that go all the way through to back-end.

11:11 Sometimes back-end is just like a mediator for some infrastructure that the clients already have.

11:17 But there are other times where we have to do everything, like the full digital platform for the client.

11:24 And in those cases, Python becomes a key technology for us.

11:29 Because what Python gives you most, in my experience, is optimizing for development time, right?

11:36 People are very much concerned about Python runtime efficiency.

11:40 But the key problem is development efficiency, right?

11:45 Most of the time, if it's a little bit slower, like in comparison, a Formula One car and a regular vehicle, one is much faster than the other.

11:56 We don't need Formula One cars most of the time, right?

12:00 That's right.

12:00 So it's like fast enough.

12:01 And that is the point.

12:03 So for many things, Python is fast enough.

12:06 But in terms of development time, Python is the Formula One vehicle.

12:10 It's like super fast.

12:12 It's super easy to throw things away.

12:14 It's super easy to explore.

12:15 It can touch every niche in computer size.

12:18 And I kind of have...

12:20 Yeah, exactly.

12:21 It has PyPI, right?

12:23 For example, you can install this and go.

12:25 Exactly.

12:26 So it was a key technology for us in those cases that we have an extremely agile cycle of development.

12:32 Lots of changes because design was sometimes moving along the way.

12:36 And we need to rebuild the backend infrastructure overnight.

12:39 So because for those scenarios, it was key.

12:43 I think the conversation around performance in Python is super interesting because there's just so many layers and variations.

12:50 Yeah.

12:51 And what are you trying to do?

12:53 Well, if you're trying to do a tight loop that does math, guess what?

12:56 Python's bad at that.

12:57 But maybe you shouldn't be doing that.

12:58 Maybe you should be using NumPy and the SciPy stuff.

13:01 And then all of a sudden, it's C speed again.

13:02 Yes.

13:03 Maybe you're doing some data-driven web API.

13:07 And it's like, well, it might be slow.

13:09 Well, actually, what you're doing is you're orchestrating exchange of JSON and talking to a database server or cluster.

13:15 And it's almost exactly the same speed as if it were written in C because you're mostly waiting on the database and waiting on the network.

13:21 And there's all of those things.

13:23 And then on top of it is this thing that you talk about that often gets ignored.

13:27 There's a really interesting story that was recounted in Mike Driscoll's Python interviews book about the competition between Google Video and YouTube and how Google Video had like 100 C++ engineers.

13:40 And YouTube, this little startup, had 20 Python developers.

13:45 And YouTube was just blowing away all the Google engineers because they could add features faster.

13:50 And if Google would do something, they could copy it, re-implement it really quickly.

13:53 And so Google fixed the problem by buying YouTube.

13:56 It's still Python to some degree there.

13:59 So that's a really good point.

14:01 Yeah.

14:01 My experience, there are two factors.

14:04 One factor that you just mentioned is having it under control when it grows.

14:09 Right?

14:09 So software is like a living thing.

14:12 You start a project.

14:13 It gets, it's born.

14:14 It starts to grow.

14:16 Right.

14:16 Does it become a sequoia or does it become blackberry bushes?

14:19 Exactly.

14:20 This portion of Talk Python, I mean, is brought to you by Sentry.

14:25 How would you like to remove a little stress from your life?

14:27 Do you worry that users might be having difficulties or are encountering errors in your app right now?

14:32 Do you even know it until they send that support email?

14:35 How much better would it be to have the error and performance details immediately sent to you,

14:41 including the call stack and values of local variables and the active user recorded in that report?

14:47 With Sentry, this is not only possible, it's simple.

14:50 In fact, we use Sentry on all the Talk Python web properties.

14:53 We've actually fixed a bug triggered by a user and had the upgrade ready to roll out as we got their support email.

15:00 That was a great email to write back.

15:02 We saw your error and have already rolled out the fix.

15:04 Imagine their surprise.

15:05 Surprise and delight your users today.

15:08 Create your Sentry account at talkpython.fm/sentry.

15:11 And if you sign up with the code talkpython2021, it's good for two months of Sentry's team plan,

15:17 which will give you up to 20 times as many monthly events as well as other features.

15:22 So just use that code talkpython2021 as your promo code when you sign up.

15:29 So it has this life cycle, right?

15:31 At some point, depending on its history, it gets out of control.

15:36 So if you didn't have the proper team, the proper guidance, it can become the problem.

15:43 Not the problem it was supposed to solve, but the software itself becomes a problem.

15:48 And then people tend to, oh, let's throw this away and get something else because this is unmanageable.

15:53 And I don't think it's an intrinsic problem on any programming language or infrastructure.

15:58 It's more between the matching of did the team that was doing this really master the technology at the time.

16:08 If they do, it could be Fortran, COBOL, C++, or Python.

16:12 You have good results.

16:14 If they didn't, they might get into this one-side, one-way road in the opposite direction.

16:20 Then they are screwed, right?

16:22 So this is one thing.

16:24 The other thing, it's all about flexibility and the performance that you mentioned.

16:28 For example, in this latest project that we are tackling, the GX project, we have it all.

16:33 We had CPU-bound problems that we are tackling, IO-bound problems.

16:38 And in both cases, Python was behaving just fine.

16:41 We have like 16,000 users right now.

16:45 We have three API instances.

16:47 And when it's a performance problem, we change the algorithm, we introduce caching, we're using MongoDB for scalability, and we have zero performance issues so far.

16:58 But because we have the software under control, we know what's happening in there.

17:04 Technology is not a mystery.

17:05 Then it's easy to pinpoint what's wrong and then replace it for a new version fast.

17:12 And that's it.

17:13 It's bringing results.

17:14 That's a fantastic point.

17:15 You know, you've got the architectural considerations as well as just the raw technology, right?

17:20 Oh yeah, for sure.

17:21 It's a single thing.

17:22 That's why I use the metaphor of like a living organism, because it's on an ecosystem.

17:27 Your software is not an island anymore.

17:30 In the past, we had like this release to the desktop, install, and it's all on your machine.

17:37 But today, it's interacting with the operating system, with the cloud, with the user.

17:43 So it's alive, right?

17:45 It's a dynamic ecosystem.

17:46 So it's all about those interactions and understanding the dynamics on each of these interfaces, and then optimizing for the Feng Shui, for the flow.

17:56 Very cool.

17:57 And I want to dive into this first API, because I think it's super interesting, and people are going to be really surprised by it, I'm sure.

18:04 Oh, that's good.

18:04 Yeah, it'll be fun.

18:05 But before I do, I do want to ask you one more sort of big picture question.

18:09 You're working with Work & Co., you're working with all these clients as this digital agency doing this end-to-end work, which I think makes a lot of sense.

18:17 What is the trends for Python that you've seen across the last five years or so?

18:24 I see.

18:24 What has stood out to you from these conversations?

18:27 Maybe you had five, ten years ago, you're having now, you're thinking about the future.

18:30 The two things that are my reality, at least, and that doesn't mean like, it's a broad world that we have out there.

18:37 So there are many cases, right?

18:39 So I'll speak just from my experience in the last three years.

18:42 So there are two things that come to mind.

18:45 One of them is this mediator thing.

18:49 So because everything is in the cloud, because you're not writing any more software from scratch,

18:55 you have to do integration, like gluing.

18:58 Python, back in the early days, became famous as a glue language.

19:03 And it was kind of gluing things in the desktop.

19:05 Now Python is gluing things in the cloud.

19:09 Because Python is such a versatile language in terms of a huge ecosystem of libraries,

19:15 because the language itself allows you to compute like anything, it became an excellent technology for a glue language.

19:22 And for us, it's critical, because any project we're dealing with, okay, we have to talk to a database, for sure,

19:31 but we also have to talk to a forecasting weather service.

19:35 We have to talk to a push notification service.

19:38 We have to talk to analytics system platform.

19:42 So you need to send signals there.

19:44 So this ecosystem thing is great for that, because you find the connectors, you find the drivers, you find examples.

19:53 So all of that speeds up the cycle of development and gives you confidence that you achieve whatever you need in success.

20:01 In terms of ecosystems, still, we're not talking about just the computer right now.

20:06 So we did a project for Google, the new store in Google.

20:09 There are some embedded devices there.

20:12 So that's another niche, embedded systems, IoT, things for the television.

20:18 We did things for Marriott, where it was like embedded on a setup box.

20:22 So this is one thing that I think we're going to see more and more often.

20:25 Now that we see like CPUs in your lamps, and people playing boom in their lamps.

20:30 Right.

20:30 The other thing, it's the data-driven aspect.

20:33 So one of the flags that I carry at Working Co is the data-driven design.

20:38 So data-driven design is like, okay, we are not doing design just based on the inspirational thing from designers and their creativity and their capacity for innovation, but anchor in data from the real world.

20:52 Sometimes that approach goes all the way through conceptual design into the end product, which we are seeing with these machine learning, deep learning-based products.

21:05 Email model in production behind Flask or something.

21:08 Exactly.

21:09 So that's another trend that Python is great, right?

21:15 Not only for the business analytical aspects, but also for the pipelines to build models in production.

21:23 Very neat.

21:23 The way you described Python as a glue language is super interesting compared to the way it's traditionally been described.

21:31 Traditionally, glue language meant something like, well, I've got a C library and I'm talking to Linux.

21:38 And so I could write this stuff in Python that'll do some shell stuff with Linux and that it'll also like pull in the C API and just move that data from here to there and go.

21:48 And it's come across a little bit as a second class thing.

21:52 It's like the very best scripting language you can imagine.

21:55 And it's kind of not perfect for apps, but well, we'll do this.

21:59 But what you described was, we're going to take the database, we're going to take these APIs, we're going to take the web requests, we're going to glue that together.

22:07 I mean, that is the application.

22:09 And it's like, it's thinking, yeah, it's thinking of, well, really, what is a modern API app other than a thing that takes a little data from an inbound request,

22:20 maybe pulls some, pushes something over to Celery, grab something out of the database, calls these APIs and bundles it back up as a response and sends it back with a status code.

22:29 That's the entire application.

22:30 And yet this idea of gluing these pieces together is a really interesting way to think of it.

22:35 That's what I learned.

22:35 It's about like gluing systems and gluing data.

22:39 And of course, these things are connected, but sometimes you can have gluing data in a asynchronous way, right?

22:47 In batch processing or local processing or for analysis or human interaction.

22:52 But it's still like gluing data from several sources.

22:55 And there's this other approach, which is like gluing live systems.

22:59 And of course, data is flowing across them, but also the connectivity aspect is key, right?

23:05 Can I authenticate and talk to all of these systems?

23:09 In the way they expect to interchange information.

23:12 The real power is how well does it perform that combination, not how well does that one millisecond versus two milliseconds in the actual WebView method runtime behave, right?

23:23 Yes.

23:24 It's super interesting to see it that way.

23:25 All right, well, let's jump into the first API that we're going to talk about.

23:30 Right.

23:30 Yeah.

23:30 So this has to do with Gatorade, which is the sports drink company.

23:35 And they have created this really interesting idea of the quantified self for fitness here with this thing called the GX Sweat Patch.

23:46 Exactly.

23:47 This is something that blew my mind when I saw it first.

23:49 Yes.

23:50 This is something that is like really in the beginning.

23:52 We launched this in production in March 1st this year.

23:56 It has been in development since 2019.

23:59 And the research inside Gatorade goes even far back.

24:03 So what they are trying to do, Gatorade has this subunit called GSI, the Gatorade Sports Science Institute.

24:12 I had the opportunity to visit one of their physical instances in Sarasota, Florida in 2020.

24:19 And they do like amazing work there, like bridging sports science, like students, actual experiments.

24:27 And they are helping to develop this concept of how do you bring some of the metrics used by high performance athletes to a more general, broad audience?

24:40 Right.

24:40 We've probably all seen pictures of Olympic athletes or, you know, pick your favorite sport athletes with a couple of doctors or researchers around with a clipboard and like a big breathing thing.

24:53 And they're on like a workout bike or they're doing and they're like studying all these different things.

24:58 Right.

24:58 And they get really interesting feedback on, well, under this situation, this is where you're hitting your limits in cardio or in like breathing at altitude or something.

25:10 And for the rest of us, we just put on shoes and run or, you know, get on a bike or whatever it is.

25:16 And we just have no idea.

25:16 In a specific case of the, what you're showing now on the screen, the sweat patch, this thing that you can put in your forearm and it's going to capture your sweat in your micro pores in your skin.

25:29 And then you lead that sweat into two channels.

25:32 One channel, which is the orange channel that fills up in a zigzag form.

25:37 It's the volume of sweat you're producing.

25:39 The other channel is a slightly bar on the lower side that will react in colors to reflect the amount of sodium that is in your sweat.

25:52 And those two things, they will be super critical to understand how much fluid you're losing from your body while you're performing some sports activity and the concentration of sodium you're losing as well.

26:06 Because sodium, as everybody knows, it's critical for the performance of muscles, the potassium sodium pump that exists and control all muscles.

26:16 So learning about those things can help you improve your performance.

26:21 And of course, this is more impactful if you are a professional athlete, but I think everybody can benefit from it.

26:27 And the biggest challenge was to take that out of the lab where you mentioned, Michael, where you have an apparatus that is easier to capture all the signals and bring that out in the field where anybody can use under any conditions.

26:44 And that was the challenge that we took up to help Gatorade to achieve that.

26:48 And it's had a lot of obstacles for us to overcome.

26:53 And Python was super helpful because not only it had the tools for us to build APIs, but also there's a ton of formulas that we need to compute to achieve that.

27:04 And Python had NumPy, Pandas, and Stats, and all the tooling that we needed to make it happen.

27:12 So let me just describe this really quickly for people who are watching.

27:16 It's like a band-aid, maybe two and a half inches.

27:20 Yeah, like four fingers.

27:21 Three or four fingers, something like that.

27:22 Maybe about the size of the palm of most people's hands.

27:25 You stick it on to your arm and it changes these colors and reads out.

27:30 And then you scan it with this iOS app and it gives you an analysis that you're talking about here.

27:35 Yeah.

27:35 Exactly.

27:36 The app, of course, serves for other purposes as well.

27:39 So you can like track your workouts.

27:41 It does integrate multiple sources of information.

27:44 Garmin, Strava, HealthKit.

27:46 And then it creates this timeline of events in your day that understands if you're like doing multiple workouts a day, very early in the morning, very late on the evening, and how they interfere in each other.

28:00 Because if they are too close apart, maybe you shouldn't drink anything between the two of them.

28:04 If they are too far apart, maybe you need supplemental hydration.

28:08 So all of those effects, we perform that analysis in Python at the back end.

28:12 And the app becomes like the avatar that conveys that information to the athlete in real time.

28:20 Yeah.

28:20 When I first heard about this, I thought it specifically would just be like, well, Gatorade recommends this.

28:24 So here's how much Gatorade you should buy.

28:26 You should get the lime flavor, not the cool or whatever.

28:29 But it takes in data from, like you said, like Apple HealthKit and Garmin and these other things as well.

28:36 How do you get that data out of it?

28:37 That's a great question.

28:38 So the plan was to go as broad as possible.

28:41 We even investigated Fitbit and other providers.

28:44 But for the MVP, we went for three sources.

28:49 Two external sources, Strava API and Garmin API.

28:53 So if you're a Strava user or a Garmin user, you can, in a web view, connect to Garmin and Strava through our app.

29:00 And then we created this data feed from Garmin and Strava into our system.

29:06 Like some OAuth.

29:07 Backend sort of API type of thing?

29:09 Okay.

29:10 Yeah.

29:10 And then it starts popping up in the app.

29:12 You don't need to.

29:13 The app becomes a read-only thing.

29:15 You just look into it and you perform your exercise.

29:18 But the information comes by through the backend.

29:21 So these are the main channels.

29:22 So these are the main channels.

29:22 But we also talk to Apple HealthKit, the platform in the iOS that consolidates information from multiple apps.

29:32 Health related, of course.

29:34 And then in that case, if you have like a sleep tracker or other sports tracker.

29:39 For example, I use RunKeeper.

29:40 So I go to that application.

29:43 I allow that application to export information to HealthKit.

29:46 And then, of course, Apple Health, the app, will see it.

29:50 But also any other app registered as a reader for the HealthKit platform.

29:56 And that's how we grab and consolidate all your activity, sleeping and workouts, into this single timeline.

30:04 And then we provide recommendations on top of that.

30:07 Right.

30:07 So you might be able to correlate workout with sleep.

30:10 Yes.

30:10 How well you're sleeping or something like that.

30:11 Yes.

30:12 That's one of the things we do.

30:13 That was one of the challenges.

30:15 Dealing with this events timeline became this, we needed a time series database to do time series analysis.

30:22 Because time is super relevant.

30:24 We have to handle conflicts.

30:25 Because you may have like many trackers reporting the same thing.

30:30 So the same physical event comes as multiple digital events.

30:34 Sometimes there are applications that break up a single event into multiple events.

30:39 Sleep trackers typically do that.

30:42 So for us, you have a single night of sleep.

30:45 You go to bed and you wake up.

30:47 Right.

30:47 But actually what happens is multiple cycles of sleep that you wake up in the middle of night.

30:53 But we don't handle them logically as multiple sleeping periods.

30:58 For us, logically, it's a single sleeping period.

31:00 So this is another thing we have to handle in the system.

31:03 Yeah.

31:03 It seems really useful, actually, the more I hear about it.

31:05 So let's talk about some of the tech behind the scenes for this one.

31:09 Cool.

31:09 There's probably some data science side.

31:11 There's obviously the API and database side.

31:14 What do you got going on here?

31:15 Okay.

31:15 So in the beginning, we went down to GSSI, the Gateway Sports Science Institute.

31:20 And together with their scientists, we started to create a model.

31:24 Not only for the way the sweats patch works.

31:28 The sweats patch was developed by a Boston company named Epicor, the physical patch.

31:33 And also the driver that kind of captures the basic information from the patch.

31:39 That is embedded on the mobile app.

31:41 And then when you take a picture of the patch, we do some image processing and we extract the two bits of information that we want.

31:49 The volume channel.

31:50 That's probably something like Swift or something on the app, right?

31:53 Today, it's a C# Xamarin.

31:55 But it was a modern restriction from PepsiCo.

31:59 They were already using that technology.

32:01 And it was supposed to be a framework cross-platform.

32:05 So later, when we decide to go to Android, it would be a possibility to reuse that framework.

32:11 So that was a constraint that we had to accommodate for.

32:14 But those modules are Objective-C and Swift, for sure.

32:19 They are native.

32:20 Yeah.

32:20 And so what we do after we have that, we have to translate whatever you read, which is your local sweat rate.

32:28 How much your forearm is sweating.

32:30 A single micropore in your forearm.

32:33 And we have a statistical model developed by GSSI to translate that into your whole body sweat rate.

32:40 So that is the first machine learning, statistical learning bit that is embedded on the system.

32:47 And we need to take into consideration what's the weather like?

32:51 What's your weight?

32:53 What type of sports you are doing?

32:55 What's the humidity?

32:55 Yeah.

32:56 Humidity we are not using right now because it's hard to capture.

32:59 So we had to create a less accurate model.

33:02 But for sure, humidity is critical.

33:04 But it was a product decision to leave that one out.

33:08 Because of the facility to grab the information, to capture that.

33:12 Right, right.

33:12 And once we have your whole body sweat rate, that acts as a crystal ball for future workouts.

33:20 So when you're performing a new workout, you don't need to use the sweat patch again.

33:24 You can use the sweat profile, the crystal ball, to predict, given the conditions of this new workout, how much you're going to sweat.

33:33 And then base recommendations on that.

33:36 So, of course, if there are some constraints that if they do not match, you have to do a new sweat test, create a new profile for those new conditions.

33:46 For example, I create a patch for running and now I'm doing bike.

33:50 So it's a different activity.

33:51 It should be best if I create a new profile.

33:56 This portion of Talk Python to me is sponsored by Linode.

33:58 Visit talkpython.fm/Linode to see why Linode has been voted the top infrastructure as a service provider by both G2 and TrustRadius.

34:07 From their award-winning support, which is offered 24, 7, 365 to every level of user, to the ease of use and setup,

34:14 it's clear why developers have been trusting Linode for projects both big and small since 2003.

34:20 To pull your entire application stack with Linode's one-click app marketplace or build it all from scratch and manage everything yourself with supported centralized tools like Terraform,

34:30 Linode offers the best price-to-performance value for all compute instances, including GPUs as well as block storage, Kubernetes, and their upcoming bare metal release.

34:40 Linode makes cloud computing fast, simple, and affordable, allowing you to focus on your projects, not your infrastructure.

34:48 Visit talkpython.fm/Linode and sign up with your Google account, your GitHub account, or your email address, and you'll get $100 in credit.

34:56 That's talkpython.fm/Linode, or just click the link in your podcast player's show notes.

35:01 And thank them for supporting Talk Python.

35:03 How does the app know what you're doing?

35:07 Do you tell it?

35:08 Just right now I'm doing this, so take a scan.

35:10 You have two options.

35:11 One of them, the app asks the user, when you're scheduling manually a workout, you can say, I'm planning to use a sweat patch on this particular workout, so be prepared.

35:21 That's one option.

35:22 The other option is, after you finish a workout, the app asks you, did you use a sweat patch?

35:28 So if you did, let's go with the flow of scanning.

35:31 If you're not, there is a fallback mechanism called way in, way out, where we can compare your rates and subtract them and see how much fluid you lost.

35:42 The downside of this is that we do not capture sodium loss.

35:46 So you cannot take sodium in consideration.

35:48 So when it gets to the server side, what's the API framework?

35:53 So the app is talking to the backend all the time.

35:56 We kind of use the backend for frontend metaphor of sorts.

36:00 So the API not only does the reasoning for the whole system, but sometimes it even helps the app a little bit with layout.

36:09 So the app talks to the backend to get user profile, to get the timeline of events, and then it renders those timeline of events, captures additional information, for example, your motivation and your fatigue, and reports that information to the backend.

36:24 The sweat scans and manually schedule workouts.

36:28 Everything else happens at the backend.

36:30 By everything else, what do we mean?

36:32 Grabbing weather information.

36:34 So translating your latitude and longitude into a temperature, if your workout was outdoor.

36:40 One thing.

36:41 Sending push, scheduling push notifications to remind you that you have this recommendation or there's an upcoming workout.

36:49 Things like that.

36:50 It also computes the local to whole body transformation.

36:54 It manages your sweat profiles and it triggers some product recommendation engines that will tell you not only Gatorade products,

37:03 but also general foods that could be suitable for your nutritional needs.

37:09 So we can say, for example, if you need this amount of carbs, this amount of protein, maybe you should take a little bit of caffeine or casein.

37:18 And then we give it a list of generic food like rice, coffee.

37:24 And then you can plan accordingly to fulfill those recommendations.

37:28 All of that comes from the backend.

37:30 And the backend is Flask?

37:31 The backend is Flask.

37:33 We were in doubt in the beginning being between FastAPI and Flask.

37:38 I wanted to fight the problem domain, not the technology.

37:44 So I decided to go fully synchronous.

37:46 And because in that case, it's super easy to debug.

37:49 It's much less prone to like problems.

37:52 I was not concerned with performance in the beginning.

37:54 What we learned was that Flask was performance enough for all our needs.

38:00 So that went well.

38:01 So that went well.

38:02 We had a separate API just for integrations with Garmin and Strava.

38:06 After we entered production in March, we realized that the team was too small to maintain it.

38:13 So we merged the two APIs into a single monolith.

38:18 But those two APIs, one of them was asynchronous.

38:21 So we did that asynchronous API with Quart.

38:24 And that was the one talking to Garmin and Strava because that one was purely IO bound, not CPU bound.

38:31 Right.

38:31 You're entirely waiting on Strava and Garmin and the internet.

38:36 And so you should be able to scale that many, many times out because all you're doing is waiting on their APIs.

38:41 And you're completely at the mercy of their performance picture as well.

38:46 Right.

38:46 So Async makes a lot of sense, but it turns out it wasn't needed.

38:49 Yeah.

38:50 The thing that we realized was that, of course, there are tradeoffs.

38:53 So if you go for a job interview and you present that as like a conceptual problem, I think the answer is no.

39:00 Do it asynchronously.

39:01 Do it as a separate API because then you can scale independently.

39:05 You have better IO throughput, lower latency, etc.

39:08 But in the real world, you have to balance all those things with the size of your team, the resources that you have, other external conditions.

39:17 So in the end, we decided to consolidate everything into a single technology in a single stack because it was simpler.

39:25 If we need to like to onboard new people trained instead of knowing Flask and Cart, now they only need to know a single framework.

39:33 And because scalability was not a problem, we were using like Kubernetes in production and it's like horizontally scalable.

39:40 We decided to reverse that and build a monolith, single stack, Flask all the way.

39:47 And that's what we have today.

39:49 How interesting.

39:49 So you decided you can just solve it by running more worker processes for container and then just running more containers if you need to.

39:57 Exactly.

39:57 That was a better solve for the conditions we had in the project.

40:01 I think that makes a lot of sense.

40:02 There's so often these recommendations of using microservices, breaking stuff into a bunch of pieces, having just the right technology for just this slice of what you're doing.

40:15 And then you've got your app.

40:17 Your app is talking to the different services.

40:19 Now you're trying to coordinate.

40:21 It just gives me chills to even think about it.

40:23 Releasing this through the Apple app approved process and coordinating that with the versions of multiple APIs.

40:29 Exactly.

40:30 That sounds so bad.

40:31 It becomes a nightmare really fast.

40:33 And in my mind, my experience is that microservices is an answer for a given team size.

40:40 So if you have like 100 developers like, I don't know, or a thousand developers like Netflix or Shopify, then it makes every sense in the world to break it up in individual components because you have individual teams.

40:53 Conway's law, right?

40:55 The software you build reflects the architecture of the people in the company and how they build the software.

41:02 But when you have a very tiny little team, the monolith is great.

41:05 It simplifies everything.

41:07 So that's like a lesson we learned that we kind of over-engineered in the beginning, trying to go with two APIs, and we took the route of bringing the monolith.

41:19 I think that makes a lot of sense for small teams.

41:21 And here's the thing, if you run into performance problems that really needs async stuff, you know, when you considered to use Flask and not FastAPI, FastAPI was brand new.

41:31 And who knows if it would survive another six months or if it would go the way of other really promising projects like Jepronto or something, which as far as I could be wrong, but as far as I know, it hasn't gotten a ton of traction.

41:44 It was really exciting for a while, and it just kind of fizzled out, right?

41:47 Yes.

41:48 You want to build on that.

41:49 So I think Flask is a totally reasonable choice.

41:51 But I guess what I was going to say is, you know, it's not that different if you need to re-translate that, if you're going to convert that to FastAPI.

41:57 Like, that's a thing you as a team of a couple of people could do in a few days.

42:02 Yes.

42:02 And it would be fine.

42:03 There were other, like, circumstances.

42:05 For example, as a big fan of Talk Python to me, I watched the episode on FastAPI.

42:11 I watched the episode on Pydantic.

42:13 And I knew of the symbiosis between the two of them.

42:16 But for us, we started with Kerberos as a schema validation technology.

42:22 Which comes out of the Eve project from my friend Acola, by the way.

42:25 Yeah.

42:25 And we evolved into using Marshmallow, replacing Kerberos.

42:29 We compared with Pydantic.

42:31 And at the time, we didn't want to, like, extract Pydantic from FastAPI and use FastAPI with Marshmallow.

42:38 So we went with Flask that was, like, not opinionated about what the schema should look like.

42:43 And we went with Flask and Marshmallow.

42:46 Nice.

42:46 Some other interesting building blocks that you highlighted is Pint, P-I-N-T for units.

42:53 Pint is super cool.

42:54 Tell people a bit about Pint.

42:55 Exactly.

42:56 So one of the things that we're doing on this particular project was it's supposed to be international.

43:03 And it's, like, heavy on the physics.

43:05 So we're dealing with rates and concentrations.

43:09 There's a lot of chemistry going on.

43:12 We're talking about the metric system, U.S. customary, imperial systems.

43:17 So converting between units was going to be a big part of the system.

43:23 And then we find out about Pint.

43:25 I was, like, in the Pint community for over 20 years.

43:28 I shook hands with Guido von Russell in 2005 when I translated his tutorial to Portuguese.

43:35 But I never heard of Pint in those 20 years.

43:38 But when I had the need, I did a quick research and I found exactly the solution that we wanted.

43:45 So Pint uses the excellent object orientation that we have in Python to transparently create

43:53 this, like, new integers and floats.

43:55 They're just not numbers.

43:57 But they also have a unit together with them.

44:00 It's so fantastic.

44:01 Let me maybe describe this little example on the Pint homepage.

44:04 So if I wanted to have three meters plus four centimeters, instead of saying three times 100 plus four or vice versa, you know, divide by 100.

44:13 You have three times meter plus four times CM.

44:18 And then what you get back is a quantity, which is 3.04 meters.

44:21 It's fantastic.

44:22 Exactly.

44:23 You could even do something horrible like three meters plus seven inches if you had to.

44:27 And it becomes even more powerful when you're talking about different kind of dimensions.

44:32 For example, if I want a concentration and I'm going to like to divide mass per volume, and then I need to make sure that my calculation makes sense.

44:42 So Pint allows you to do that.

44:44 So you do those conversions and the units are preserved.

44:49 And because the units are preserved, it's easier to test.

44:52 It's easier to compute.

44:53 So it's really a lifesaver that saves us a lot of time while coding this API.

45:00 Yeah, this is not something that in my world I do anything with these days, really.

45:05 But if I did, boy, I'd be all over Pint.

45:07 That thing's cool.

45:07 Also, there was Unit.

45:10 Yes.

45:10 I've also just recently heard about, and I don't know really how they can, UNYT, but this is also something similar in that regard as well.

45:20 Yeah, we, at the time, and I'm talking about early 2020, we evaluated a couple.

45:26 I don't remember if Unity we checked, but there were three others besides Pint.

45:33 But in the end, Pint sound the most robust one.

45:36 But I'll check Unity again.

45:37 Yeah.

45:37 I guess, what's your assessment now that you've been actually using Pint?

45:41 It's good?

45:41 We're super happy.

45:42 If we have like zero issues with Pint, it saved our lives time and again.

45:47 Yeah.

45:47 Another one that you'll run into is Time.

45:49 So you used Pendulum for that.

45:51 I was even a little bit resistant in the beginning because I was using date time forever.

45:57 And of course, members of my team were suggesting, hey, why don't we use Pendulum?

46:01 It has a really nice interface.

46:03 And the deal breaker for me was that the Pendulum object, it inherits from the regular daytime objects.

46:13 So they are completely interchangeable.

46:14 The one thing I didn't want to happen was to have like these two kind of libraries dealing with time.

46:21 And in that case, daytime is standard in the standard Pint library.

46:26 Something else external would be a cause of concern for me in my experience.

46:32 But in that case, it was seamless.

46:34 So I can use either Pendulum or daytime.

46:36 It doesn't matter because they all inherit from the same root.

46:41 So that was key.

46:42 And to be honest, it was convenient, especially for time zones.

46:48 But remember that that was before, I think it was on Python 3.9, that we had better support for time zones.

46:56 And at least in the standard Python.

47:00 And at that time, I think Pendulum was having a better way to handle with time zone conversion.

47:06 That was key for us in terms of the solution.

47:09 We never know where our athletes are going to be in the world.

47:11 So that was the key reason.

47:13 And we're happy with Pendulum.

47:16 Yeah, this is super neat.

47:17 You know, I didn't really put it together when I was looking at this before, but the time zone stuff is quite interesting.

47:22 Like something that's always a challenge that I deal with is when I'm working with some of the web apps that I have,

47:30 or other things, just where is the server versus where is the person accessing the server?

47:37 And it turns out to be way more annoying than you think, right?

47:40 Like another thing we're going to talk about is using MongoDB for the backend on this.

47:44 And so am I.

47:45 And I'm just a super fan of MongoDB.

47:47 It's been such a nice way to make fast, easy to maintain apps.

47:51 But it stores stuff at like UTC, whereas the server, I think, is in Eastern time zone.

47:56 And I'm in the Pacific time zone.

47:58 And so if I want to pull something up, so this event is going to happen then, or it happened at what time?

48:02 It's not easy to say, well, that was an hour ago.

48:06 Like something so simple as here's a list of activities.

48:08 And this one was an hour ago.

48:10 It's challenging.

48:10 And this is really cool.

48:11 So you can say, instead of daytime now, I can say pendulum.now and then pass in the base time zone.

48:16 And I say, well, what is now in, you know, inbound time zone or something like that, right?

48:21 Would be really, really nice.

48:22 The strategy we took, which is kind of traditional strategy, is that at all the edges of the system, we convert to UTC.

48:30 And inside, we're just reasoning on UTC.

48:34 And only when we're kind of exporting information through this membrane of the system back to close to the user, then we translate back to their time zone.

48:43 However, it gets tricky when we're talking about not real time, but nominal time.

48:49 What is nominal time?

48:50 For example, I wake up at 7 a.m.

48:54 So 7 a.m. for me, it's a nominal time.

48:57 It's not anchored on any place on Earth.

49:00 It's like the time that I'm supposed to wake up.

49:04 The nominal time becomes trickier because it's not anchored on a particular place.

49:10 So you have to deal with those kind of things a little bit differently.

49:13 But other than that, Pendle was very ergonomic in terms of their API.

49:19 It's super easy.

49:20 You can keep it on your mind.

49:22 And it's compatible with date time.

49:24 So it was a plus.

49:25 We're happy.

49:26 That's a really neat point because if you've got some other API that takes a date time, you can just pass in the Pendle time and it just, it is a date time.

49:35 Yeah.

49:35 Right?

49:35 So you don't have to major, oh, I forgot to convert here.

49:38 So it's broken.

49:38 So the lander crashed into the ground because, you know, whatever.

49:41 Exactly.

49:42 All those units and conversion types are weird.

49:44 Yeah.

49:44 And the arithmetic.

49:45 Right?

49:46 So I think that is the other aspect that Pendle makes a little bit easier.

49:50 The arithmetic one is like, as a day subtract minutes, this duality between a point in time and a delta that we do have on like the time and time delta and all the Python API.

50:03 But Pendle also helps you in that sense to do those conversions in terms of like scale and arithmetic.

50:11 So it was really interesting.

50:13 Yeah.

50:13 One quick thought I'll throw out there that because this was such a surprise to me.

50:16 I just had, it's so undiscoverable, but it's good once you know it.

50:21 Yeah.

50:21 It's good to know it.

50:22 Let's put it that way, at least.

50:23 That when you're doing, forget Pendle for a minute, if you're just doing raw date times and time deltas.

50:28 So time deltas always have a seconds, right?

50:31 Yeah.

50:31 So I create a time, a difference between two times or I take a time and add a time delta.

50:36 I get a new time delta or you get new time.

50:38 But if you have a time delta and it's in seconds, which is basically the only option you get.

50:42 And you want to know, well, I need this in hours.

50:45 I would always just go two total seconds divided by 60.

50:48 Exactly.

50:48 Divided by 60 again.

50:50 What you can do is create another time delta and say time delta hours equals one and then divide one by the other.

50:56 And then you get a number of hours.

50:59 Or if you want like weeks, time delta is seven days and you divide your time delta by that and it'll give you the days.

51:05 And that's really handy.

51:06 But boy, is it hard to discover to know that that's possible.

51:09 Exactly.

51:09 And that's where I think Python shines as well.

51:13 Because since the beginning, one of the arguments in favor of Python, it was when people said Python is pseudocode that runs.

51:22 They were telling you Python brings you closer to the domain of the problem.

51:26 Right.

51:27 And that matters.

51:29 Syntax sugar matters.

51:31 Because the way you express software, it means you make less mistakes.

51:36 So when you read it, you fully understand it.

51:39 Yeah.

51:39 It's easy to do a review.

51:41 It's easy to do maintenance.

51:43 It's easy to throw in and write it again.

51:45 That matters.

51:47 So it's not just about if you know the logic, if you know the theory, but in the way you express the logic and the theory matters a lot.

51:56 Absolutely.

51:57 Totally agree.

51:57 All right.

51:58 Let's talk one more element here on this one.

52:03 And that's the database side of things.

52:05 Why did you choose MongoDB?

52:07 What's your experience working with it?

52:09 That's a great question.

52:10 That's an example.

52:10 That's a great question.

52:11 So I came while I was at the university, I was working with the database group and I had a lot of experience with like database.

52:19 I played with object oriented database in 1993, the French object database called O2.

52:25 As time went by, we had Zope and Zodb.

52:29 I played a lot with Zodb.

52:31 I even made a tiny little contribution to Zope.

52:33 So I've been exposed to NoSQL before NoSQL became a thing.

52:38 But of course, SQL was still there.

52:40 But what we wanted was, again, development time.

52:44 So as a director at Working Co., my main consideration is, can I make this project fit the budget and the schedule before I exactly know what we have to do?

52:55 Because details will only be captured at the end of the conceptual design phase.

53:01 But at that point in time, the money is defined, the schedule is set.

53:06 So it's like, okay, I'll build the pyramids, but I don't know what's going to be like bringing those blocks of rock across the Nile.

53:13 So because of that, I wanted to go for flexibility first.

53:17 And going for a NoSQL database, it gives you speed in terms of development time, right?

53:24 So you can put anything in there.

53:26 Mongo is storing information in the same formats as the applications are consuming then, which today are like...

53:34 You don't have that object relational impedance mismatch.

53:37 You just have it's in Python.

53:38 Now it's in Mongo.

53:40 Now it's back in Python.

53:41 It's the arms of the world.

53:43 I always was let down by arms.

53:47 Because in the beginning, when you know nothing, they are great because you don't need to learn the actual language of the database.

53:54 After you evolve, they become a barrier because they are never as sophisticated as the data manipulation language of the database, right?

54:05 And then you say, oh, why they don't support this and that?

54:08 My challenge has been more operational with ORMs is they're pretty good.

54:13 And a lot of times, if you have the right indexes, you know, the speed can be okay.

54:16 If you make sure you do the joins instead of the lazy N plus one type of queries, you'll be okay.

54:23 But you build up your classes and everything's easy.

54:25 And then you want to add a field to a class.

54:29 You want to create a relationship.

54:30 Well, now you try to run your app.

54:32 It doesn't just not work well.

54:33 It fully crashes until you do a database migration.

54:36 And are you doing that migration in production and staging and dev?

54:41 And then what's the downtime story as you roll out the changes to your multiple servers that all got to talk to that database?

54:47 It's like, ah.

54:48 And for me, something like Mongo that is just so much more likely to adapt.

54:52 Like I haven't run a database migration or anything like that for years.

54:57 You know, people talk about Mongo for like it's web scale and it's all this data.

55:01 And that's great.

55:01 I mean, like my database probably has seven gigs of data for like talk by done training and the podcast.

55:07 And that's a non-trivial amount of data, but it's not so much about the data.

55:11 It's about the flexibility.

55:12 Like it's easy to make it fast and it doesn't require DevOps at the extreme to do it well.

55:19 You know?

55:19 That is a great point because managing the database.

55:22 I mentioned DevOps.

55:23 So because Mongo, they also provide the Atlas service where you have like database as a service.

55:29 That was key for us because then we can like forget about this.

55:33 We simply connect to Atlas.

55:34 They manage our cluster.

55:36 They have the dashboards, everything in place.

55:38 It was one click of a button to migrate from one version of Mongo to another version of Mongo.

55:44 So those things are priceless when you're doing like quick-paced development.

55:49 There was another reason though.

55:50 In the beginning of this project, PepsiCo asked us to use Azure.

55:55 So that meant we were doing our Python services within Azure and we had to do a database within Azure.

56:02 And we went for CosmoDB because CosmoDB was compatible with Mongo.

56:07 If at any point in time we needed to move out of CosmoDB, we would have Mongo as a fallback.

56:12 And that's exactly what happened.

56:14 So a couple of months, there were some political changes within PepsiCo.

56:19 They decided to go like AWS out of Azure and then boom, we had MongoDB in Atlas and we had like the same cluster in AWS for Python.

56:30 We were happy.

56:31 Totally agree.

56:31 Fantastic.

56:32 So I know we're going to talk about a bunch of APIs, but we spent all our time talking about this one.

56:38 Sure.

56:38 But I think this is really a super interesting thing to dive into.

56:44 We can touch on some of the other ones maybe a little bit, but I guess before we move off of this,

56:48 like what's one of the big takeaways you took from this project here?

56:51 Oh, that's such a big question.

56:53 There are like many things.

56:55 The biggest perhaps is at the core of the agile mindset.

57:00 After you implement stuff, it's really when you start to understand the problem.

57:05 It's really, really hard to kind of design on paper and then it just works.

57:11 So what we did for this project was many, many cycles of software rewrite.

57:17 When the project started back in 2019, like over the weekend, when they presented me the problem over the weekend, I was able to, on a Friday afternoon, Saturday and Sunday, to work on a tiny little full-blown solution.

57:32 I had a timeline.

57:33 I had a recommendation system.

57:35 I had recommendations.

57:36 It was kind of mockery a little bit, but I was able to cook that up in three days, show that to the clients and say, this is more or less what you're going to do.

57:47 And they really understood what was in there.

57:49 And then we threw all of that away and built like the new version, like on Troy, many cities built one on top of the other.

57:56 So that's what happened on this API.

57:58 Like we're doing microservices, like let's do Monolith.

58:02 We're doing Cosmo.

58:03 Let's do Mongo.

58:04 We're doing Quartz.

58:06 Let's just do Flask.

58:07 We're doing Kerberos.

58:08 Let's do Marshmallow.

58:10 Being able to throw things away and kind of rebuild, refactor, the power of refactoring made us to have a system that's always performing, that's manageable, that's minimal technical debt.

58:24 And I think that was the key learning that I was able to fully apply in this long-term projects, like two years.

58:31 That is not common for a working coach to have this two long projects.

58:36 Usually our project is like three months, six months, maybe a year.

58:39 But in the case of Gatorade, it's a retainer project.

58:43 We're for the long run with them.

58:45 And in this case, you really need to evolve the platform.

58:49 It's like maintenance-driven development.

58:52 Does that actually give you a different mindset when you're creating?

58:55 You're like, I am going to have to live with this.

58:58 Not they.

58:59 We're going to have to live with it.

59:00 I think that is the most difference between what I consider seasoned developers and new developers.

59:09 We have like junior developers that they are very competent.

59:13 They are very capable.

59:15 They master really quickly new technologies.

59:18 But the difference is to fully understand the impacts of decisions, not just based on theory, but based on social events that may or may not happen, based on what the future may bring.

59:30 And experience counts a lot in that sense.

59:33 And the maintenance-driven development mindset, when you know you're the one that you're going to do the maintenance, changes things a lot.

59:42 You think about stability, robustness, task coverage in a completely different way.

59:48 I totally, it seems like it absolutely would if you're the one who's got to live with it.

59:52 Because you go to some of these consulting projects and they're like, oh, we really want to use this specific database.

59:57 We want to use this framework, but in this odd way.

01:00:00 And you're like, well, they're the customer.

01:00:01 They're always right.

01:00:02 But if it's going to be you, you're like, yeah, we better build this the way that we're going to be happy in a year.

01:00:06 You know, there's that, it reminds me of that funny quote, like, always code as if a person who ends up maintaining your code will be a violent cycle path.

01:00:15 That's where you live, right?

01:00:16 Very good. That's good advice.

01:00:17 Yeah, a little bit morbid, but do think about that you'll have to live with it, I guess.

01:00:23 And I think that talks to the trend they call boring codes, right?

01:00:27 There's a bunch of like speeches and talks on the internet.

01:00:30 People talk about, oh, right, boring codes.

01:00:32 Codes are easy to understand, easy to do maintenance.

01:00:34 It has this mindset that when you're learning things and you're not concerned about production stuff, you are much more bold to try new things, right?

01:00:45 Because then the project is done, you move away.

01:00:48 So there's no cost, there's no risk, there's no penalty for trying new stuff that may prove inadequate.

01:00:56 But when your success measurement is considering the maintenance phase, the long-term success of the project, the mindset is a different one.

01:01:05 It's what is stable, what's guaranteed.

01:01:08 If you're chasing the shiny new thing, you could end up saying, well, I'm going to build this six-month project.

01:01:12 I'll do that in React.

01:01:13 And this one is going to be in Vue.

01:01:14 And this one will be in Angular.

01:01:15 And this is going to have this backend and that backend.

01:01:17 And then all of a sudden, you end up maintaining every JavaScript front-end framework in the world.

01:01:22 Exactly.

01:01:23 And yeah, you don't want to be like that.

01:01:25 I think the other thing you touched on is really interesting.

01:01:28 One of the things that I try to preach a lot is it often doesn't matter too much how you get started so much as you get started on a project.

01:01:37 I see a lot of people saying, you know, they'll get frozen.

01:01:40 They're like, well, I just can't decide between Django and Flask or FastAPI or this or Postgres or Mongo.

01:01:47 And they just, they think and they think and they think and they don't get anywhere.

01:01:51 Like you highlighted with your, I did it over the weekend prototype.

01:01:54 A lot of times those people who are stuck, if they just fully internalize, like you can just go on a path and then evolve it with refactoring and change it and throw it away and rewrite it with more knowledge.

01:02:05 The time you spent worrying about what to do, you would already have a working example, probably.

01:02:09 And then you have so much more information to build from and decide from.

01:02:13 I think that's a really important takeaway.

01:02:15 God is in the details.

01:02:16 Nobody stumbles and mouthings.

01:02:18 It's the little things that make your fault.

01:02:20 That's a good quote.

01:02:21 I like it.

01:02:22 Nice.

01:02:23 All right.

01:02:23 Well, I think rather than diving into the other APIs that we're going to cover, I think maybe just this deep one into the DXSWet patches is probably a little bit more constructive and useful.

01:02:34 So maybe we'll leave it here for the APIs.

01:02:36 Cool.

01:02:36 Really, really neat work.

01:02:37 A lot of moving parts, right?

01:02:39 You've got your mobile apps.

01:02:40 You've got your mobile apps in Xamarin.

01:02:41 You've got your ML models.

01:02:44 You've got the APIs, the other external API integration, the other app integration.

01:02:49 A lot of stuff going on here, right?

01:02:50 So I think it's a pretty good case study.

01:02:52 It is.

01:02:52 One bit that we didn't mention, I'd like to really quickly mention, is that we have another engine for product recommendations that use constraint problem solving.

01:03:04 And this was, we're reusing a module, a colleague of mine called Gustavo Niemeyer.

01:03:10 He's a core developer for Python, now working for Canonical.

01:03:13 And he was with me in Europython 2005.

01:03:17 And on Europython 2005, he presented a Python module that now we're using to solve those constraint-based problem optimization.

01:03:26 And that was really cool because it's a really powerful mechanism.

01:03:30 And the problem we were solving was that you have all of these needs, right?

01:03:35 Needs for carbs, needs for protein, needs for hydration, needs for electrolytes.

01:03:40 And then you have all of these products that have different combinations of those elements.

01:03:45 So what is the optimal amount of products and types of products?

01:03:50 And what is the minimal amount that you need to kind of fulfill all your nutritional and hydrational needs?

01:03:57 And for that, we use Python Constraint.

01:04:00 There are other engines there, but we want it to be like fully Python solution.

01:04:05 And that is another hint that I want to share with people.

01:04:09 That is, it solves like a big problem and it's really easy to use.

01:04:13 And this is Python-Constraint on PyPI?

01:04:16 Yes, exactly.

01:04:17 This one you have on screen, Python-Constraint.

01:04:19 Looks really cool.

01:04:20 It reminds me a little bit of linear programming.

01:04:22 It is.

01:04:22 The difference is, well, actually, you have linear programming like simplex, where you're kind of going into the plane and trying to optimize numerically.

01:04:32 This is one kind of thing.

01:04:33 This other one is more on exploring a discrete space of solutions.

01:04:39 So you define your variables, the domain of values for each of these variables, and the constraints that validate, is this combination of values a solution to the problem or not?

01:04:50 If not, it's refuse.

01:04:52 So then it's a search space.

01:04:54 Yeah.

01:04:54 So there's like a chess problem, the rook.

01:04:57 Exactly.

01:04:58 Which clearly is not a continuous problem.

01:05:01 It's a very discrete.

01:05:01 The rook can be in one of, you know, 100 places or whatever on a chessboard, and that's it.

01:05:06 Very cool.

01:05:07 One final thing on this project here.

01:05:09 Now that you've created it, how many people are keeping it going and working on it?

01:05:14 What's the team size at the beginning and the end?

01:05:16 I mean, we talked about the monolith versus microservices, but not really the details there that made you decide the monolith side.

01:05:22 That's a great question.

01:05:22 So first of all, all the time we're talking about the GX consumer.

01:05:27 So for like you and me, we also have another app that talks to the same API called GX Teams.

01:05:33 And the GX Teams is for practitioners, coaches, personals, that they are managing a group of athletes.

01:05:40 So this is another thing that Gatorade launched, I guess, this month.

01:05:44 Both apps are up to the same API that we call GX Rack Engine.

01:05:48 So just for GX Rack Engine, our development team is today for people.

01:05:55 It's me, João Gomes, Nicolas Moya, Rodrigo Merlo, and Sachari Moreno.

01:06:01 So it's me plus four.

01:06:02 That is the whole backend team.

01:06:05 For frontend, for PM, strategy, business, product, then it's kind of gross, right?

01:06:13 But the things that we talk about in the Python world, we're talking about five people.

01:06:18 You know, that's super common.

01:06:19 If we look at the Python developer survey from the PSF and we go down and look at the team size,

01:06:24 the average team size, 75% of the time was two to seven people.

01:06:30 Yeah, it's the pizza rule, right?

01:06:32 Yeah.

01:06:32 If you get out to what I would call the microservice side, you know, that's maybe 20 people or so.

01:06:39 And 20 people or more on the team size is only 4% of all software developers doing Python.

01:06:45 So keep that in balance when you hear about like, you know, how Netflix or Google or Instagram is doing some amazing thing, right?

01:06:52 That context doesn't necessarily apply to like your context.

01:06:55 Exactly.

01:06:55 And they break down.

01:06:56 I worked for global.com, which is the biggest news company in Brazil.

01:07:01 We had a audience of 4 million people like daily going through like our websites, etc.

01:07:08 And the teams internally, they were like four or five people.

01:07:13 We none above seven because it becomes unmanageable.

01:07:17 The communication overhead becomes so high that it's not worth it.

01:07:22 Super interesting.

01:07:22 And I think we're going to leave it there, but I'm going to ask you the two final questions.

01:07:25 Before I do that, let me just call out a couple comments from the live stream.

01:07:29 Go ahead.

01:07:29 Black and White said, this video is so great to be frank.

01:07:31 Thank you.

01:07:32 And Vincent, hello, hello.

01:07:33 Love your show.

01:07:34 Keep up the work, please.

01:07:35 All your guests are on point.

01:07:37 Very interesting.

01:07:37 Yeah.

01:07:37 Thank you all for being here.

01:07:39 That's great.

01:07:39 All right, Rod.

01:07:40 Working on these projects and others, you're going to write some Python code.

01:07:44 What editor would you fire up?

01:07:45 Well, that's a great question.

01:07:46 So I'm a big fan of PyCharm.

01:07:48 So I've been using PyCharm forever.

01:07:51 It has some features that I kind of love, like class hierarchy, like usages of functions,

01:07:57 the debugging.

01:07:59 It's super well polished.

01:08:02 So I use PyCharm for professional work.

01:08:05 But having said that, I use everything else.

01:08:08 So I dabble with Visual Studio Code, great tool.

01:08:12 I use Vim a lot when I'm on the shell or doing stuff.

01:08:16 And of course, I use Jupyter Notebooks as well.

01:08:19 And sometimes I code stuff in Jupyter Notebooks, straight on Jupyter Notebooks.

01:08:23 And it's like lots of code.

01:08:25 All of those.

01:08:26 Fantastic.

01:08:26 I agree with all that.

01:08:27 And notable PyPI package.

01:08:30 Well, we mentioned like a couple there, right?

01:08:32 Like Pint and Pendulum and Python Constrain.

01:08:37 One that I heard in your show and I explored a little bit is called Rich.

01:08:42 So I think it was Brad Cannon that mentioned Rich, but I'm not sure right now.

01:08:46 But that was really awesome as well.

01:08:49 Really interesting.

01:08:50 So these are like my picks.

01:08:52 Yeah.

01:08:53 Rich is coming along as quite an interesting project.

01:08:56 It's going, you know, Will there is doing such interesting work.

01:08:59 And it just has super opened up what you can do in the terminal, I think, in a much more

01:09:04 approachable way.

01:09:05 Have you seen Textual?

01:09:07 No, I haven't.

01:09:08 I think that Textual is like a layout engine for Rich.

01:09:11 Oh, that's cool.

01:09:12 That is coming on here.

01:09:13 So what you can do is you can break up your terminal and to have like a toolbar, like a

01:09:19 left docking thing, a footer, and then a main area.

01:09:22 And then you render into each of those with Rich.

01:09:25 Fancy.

01:09:25 You can do like even arrow keys and only like move the main window section.

01:09:30 Oh, it's yeah.

01:09:31 There's a lot of cool stuff going there.

01:09:33 So super cool.

01:09:33 Yeah.

01:09:34 It's awesome.

01:09:34 All right.

01:09:35 Well, final call to action.

01:09:36 What's your final advice for people building these APIs thinking about these decisions for

01:09:41 their own projects?

01:09:42 Well, one I already have given, it's like be hands on, like try it out, see how it feels,

01:09:47 get your hands dirty.

01:09:49 This is crucial in my mind.

01:09:51 The other one is start thinking about maintenance driven development.

01:09:54 So when you start, because even if it's not you, somebody will have to like give maintenance.

01:10:01 So at least balance the costs and benefits of exploration and the advantages and downsides

01:10:09 of reusing well-established technology.

01:10:11 Right.

01:10:12 Look for the Zen, the yin-yang, the balance on these forces.

01:10:15 Good advice.

01:10:16 Definitely second it.

01:10:17 Rod, thanks for being on the show.

01:10:18 Thank you so much, Michael.

01:10:19 Yeah, you bet.

01:10:21 This has been another episode of Talk Python to Me.

01:10:23 Our guest on this episode was Rod Senra.

01:10:26 It's been brought to you by Sentry and Linode and the transcripts are brought to you by Assembly

01:10:30 AI.

01:10:30 Take some stress out of your life.

01:10:32 Get notified immediately about errors in your web applications with Sentry.

01:10:36 Just visit hawkpython.fm slash Sentry and get started for free and use the promo code

01:10:42 hawkpython2021 when you sign up.

01:10:46 Simplify your infrastructure and cut your cloud bills in half with Linode's Linux virtual machines.

01:10:50 Develop, deploy, and scale your modern applications faster and easier.

01:10:54 Visit hawkpython.fm slash Linode and click the create free account button to get started.

01:10:59 Transcripts for this and all of our episodes are brought to you by Assembly AI.

01:11:03 Do you need a great automatic speech to text API?

01:11:06 Get human level accuracy in just a few lines of code.

01:11:08 Visit hawkpython.fm slash assembly AI.

01:11:11 Want to level up your Python?

01:11:13 We have one of the largest catalogs of Python video courses over at Talk Python.

01:11:17 Our content ranges from true beginners to deeply advanced topics like memory and async.

01:11:22 And best of all, there's not a subscription in sight.

01:11:25 Check it out for yourself at training.talkpython.fm.

01:11:28 Be sure to subscribe to the show, open your favorite podcast app, and search for Python.

01:11:32 We should be right at the top.

01:11:34 You can also find the iTunes feed at /itunes, the Google Play feed at /play,

01:11:39 and the direct RSS feed at /rss on talkpython.fm.

01:11:43 We're live streaming most of our recordings these days.

01:11:46 If you want to be part of the show and have your comments featured on the air,

01:11:50 be sure to subscribe to our YouTube channel at talkpython.fm/youtube.

01:11:54 This is your host, Michael Kennedy.

01:11:56 Thanks so much for listening.

01:11:57 I really appreciate it.

01:11:59 Now get out there and write some Python code.

01:12:00 I'll see you next time.

01:12:21 Thank you.