#251: Building and UX Testing Azure's Python SDK Transcript
00:00 What does it take to build a Python library that will be used by a large number of developers?
00:04 This happens all the time in open source.
00:06 Projects take off and become wildly successful.
00:09 What if you could sit down with the developers using your library and see how they take to it?
00:14 Well, over on the Azure team, Kate Olsewska and Johan Stenberg do incredible user testing for Azure's Python SDK.
00:22 Just to give you a taste, imagine a room with a computer, a couple of developers unfamiliar with the API,
00:28 a one-way mirror, and lots of feedback.
00:31 This is Talk Python to Me, episode 251, recorded Monday, January 27th, 2020.
00:38 Welcome to Talk Python to Me, a weekly podcast on Python, the language, the libraries, the ecosystem,
00:57 and the personalities.
00:58 This is your host, Michael Kennedy.
01:00 Follow me on Twitter, where I'm @mkennedy.
01:02 Keep up with the show and listen to past episodes at talkpython.fm.
01:06 And follow the show on Twitter via at Talk Python.
01:08 This episode is brought to you by Springboard and Clubhouse.
01:12 Please check out what they're offering during their segments.
01:14 It really helps support the show.
01:16 Kate, Johan, welcome to Talk Python to Me.
01:18 Hi.
01:19 Thank you.
01:20 It's great to be here.
01:21 Yeah, it's great to have you both on the show.
01:23 I'm really excited to talk about creating Python APIs and how you communicate those and document
01:30 them for Azure.
01:31 Azure is so huge.
01:35 And I haven't even done anything with Azure for a couple of weeks, probably.
01:38 Maybe even a couple of months, counting some vacation time in there.
01:41 And there's just so many things.
01:44 So these libraries are super far reaching.
01:45 So it's going to be fun to dig into all those, I think.
01:48 Now, before we do, though, let's start with your stories.
01:51 How do you all get into programming in Python?
01:53 Kate, let's start with you first.
01:54 Me first.
01:55 Okay, sure.
01:56 So my story is relatively short.
01:57 I started with Python as the first language that I picked up in my first intro to programming
02:04 class.
02:04 And it kind of stuck with me throughout.
02:06 Yeah, cool.
02:07 And did you start out in computer science just planning to go down that path?
02:10 Or did you study something else?
02:11 I was initially studying economics.
02:13 And kind of in a lot of data science that economists do, you know, there's a lot of existing
02:18 software.
02:19 But at some point, I figured there's more flexibility if you do some things on your own.
02:24 And that's when I started to pick up more of data science through Python.
02:27 And that's what led me into it.
02:28 Yeah, I never really thought of economics as having this super strong data science side
02:34 to it.
02:34 But the more that I talk to data scientists, a lot of them come from economics background.
02:39 It's pretty interesting.
02:40 Yeah, there's a lot of analysis that's done on like really large scale.
02:44 And there is a huge need.
02:46 And mostly what happens is that a lot of economists just work with data scientists to help them provide
02:52 the data and the analysis, while they just provide kind of contextual information.
02:58 But in the end, it's most efficient if you can do some stuff on your own.
03:02 Yeah, absolutely.
03:03 It's definitely great if you can solve your own problems and not go asking for help, right?
03:07 Yep.
03:07 Johan, how about you?
03:08 How do you get in programming in Python?
03:10 Actually, I did not start until five years ago in Python specifically.
03:14 At the time, we were looking at rebuilding the client libraries.
03:18 And specifically in that case, the command line interface for communicating with Azure.
03:24 We had one that was written in JavaScript.
03:27 And we decided that that needed some overhaul.
03:31 So we looked into doing that in Python.
03:33 So that's how I got started with Python.
03:35 Right.
03:35 And what you're talking about, this is not just the libraries that maybe Python people
03:39 might use.
03:40 This is the command line interface for working with Azure as a Java developer or as whatever,
03:45 right?
03:46 It doesn't matter what language you are.
03:47 Like, this is the language you guys decided to implement it in or move to, at least.
03:52 Correct.
03:52 So it gave me my first introduction to both Python, the language, but also the client libraries
03:57 that we had to use in order to implement the CLI.
04:00 But you're absolutely correct.
04:01 That was an application that we wrote in Python.
04:04 Right, right.
04:05 Where were you coming from before then?
04:07 What language?
04:08 Not surprising.
04:10 A lot of people at Microsoft write code in C#, in C++, although I haven't done much
04:17 C++ in the last decade or so.
04:19 But I've also written code in Visual Basic.
04:21 I'm probably one of the few people that have written Visual Basic.net code that ships as
04:27 part of Visual Studio.
04:28 So, but C# primarily.
04:30 Yeah, yeah.
04:31 What's the story of Visual Basic.net these days?
04:33 Is that still a thing?
04:34 I mean, I know technically it's supported, but it feels like the main story that Microsoft's
04:39 telling around their own internal languages is C#.
04:41 Most users, I'm actually not up to speed.
04:45 I haven't been on the Visual Basic.net team.
04:47 I left that in 2005 or something like that.
04:50 So I've kind of lost contact with my friends over in VB.net land.
04:54 Sure.
04:55 No, no worries.
04:56 How'd you get into programming in the first place?
04:58 I'm from Sweden originally.
04:59 We had a toy store in Stockholm, the capital of Sweden, and they had a wall full of home
05:06 computers is what they were called at the time.
05:08 Commodore 64, Svick 20.
05:09 As opposed to the machine sized ones?
05:12 As opposed to the PCs that became popular later on and all of that.
05:17 And there was always something fascinating about being able to tell a machine what to do.
05:23 Although a kind of archaic way that you have to tell it what to do would wrote things in
05:28 basic at the time.
05:28 But just having playing with it and getting it to print your name over and over again.
05:33 I think that was my first program as it wasn't Hello World at the time.
05:37 It was your one was here.
05:39 It was Hello, Johan.
05:40 But yes.
05:42 I don't know.
05:43 From that point on, I realized that that's what I'm going to do when I grow up.
05:46 That's what I did when I grew up.
05:47 Yeah, that's super cool.
05:49 Well, both of those are really great ways to get into programming and cool stories.
05:53 I guess another follow up question before we move off this intro bit here.
05:58 So Johan, you had said that your first exposure to Python was when you guys rewrote the CLI
06:06 in Python, which is good.
06:08 But then you also, to implement that, had to use all the Python libraries that other people
06:16 who wanted to automate Azure or talk to parts of Azure with Python had to use as well.
06:20 What was that experience like?
06:21 Was it like, oh, these are great libraries.
06:23 They're really well put together.
06:25 It was like, wait a minute.
06:26 This seems different than all the other stuff I see online.
06:29 It looks more like C# in Python.py files.
06:33 I have no idea.
06:34 I'm asking.
06:35 What was it like?
06:36 It was a little bit all over the place.
06:39 One thing that we do here at Microsoft is quite a few of what we consider to be the control
06:45 plane.
06:45 That is how you create resources in Azure.
06:48 Those are auto-generated from Swagger or OpenAPI, which means that they are auto-generated libraries
06:54 with all of the good and bad that comes with that.
06:57 Other libraries were what we consider to be the data plane that is actually using the service
07:03 software provision that, uploading blobs, things like that.
07:06 They were hand-authored.
07:07 So there was a wide variety of how, what the client libraries looked like, what patterns
07:14 they used in there.
07:16 So it was a little bit all over the place.
07:18 Yeah.
07:18 So maybe just define Swagger and OpenAPI for folks listening.
07:21 Not everyone's API developer.
07:23 Ah, sorry.
07:24 Sometimes I forget.
07:25 That's part of my life as well is reviewing REST APIs.
07:28 So...
07:29 Yeah, no worries.
07:29 You're deep in it.
07:30 Yes.
07:31 Deep in it.
07:32 It's a way of describing a REST API.
07:34 So basically, OpenAPI is an open standard that allows you to describe your endpoints, what
07:42 kind of data that they can accept and what kind of data that they will return if you call
07:47 them.
07:47 So it's something like you guys have maybe implemented your endpoints, your REST endpoints
07:52 and, I don't know, ASP.NET Web API or whatever you guys chose.
07:56 And then you added Swagger to it.
07:58 And then everyone else is like, hey, we need libraries for these.
08:00 We can auto-generate some Python libraries or JavaScript libraries or whatever it is you
08:06 can auto-generate off of Swagger.
08:08 Is that sort of how it went?
08:09 Yes.
08:09 So it's both that.
08:10 And we still do that.
08:11 We have, there's a requirement for all Azure services to describe their REST APIs using
08:17 OpenAPI.
08:18 And it has a tool chain both to generate libraries as well as documentation, reference documentation
08:24 and tooling for trying to ensure consistency among the REST APIs.
08:28 Yeah.
08:29 Oh, that sounds really fun.
08:30 Now, next thing related to this, I want to ask you what you do day to day.
08:35 And we got a sense of what you do quite a bit there, Johan.
08:39 But maybe give us the quick flyover and then Kate can tell us what she's up to as well.
08:44 Yeah.
08:45 So the two main areas of responsibility that I have is I'm the architect for the Azure client
08:52 libraries for Python.
08:53 So I'm responsible for the overall design guidelines that we have that are publicly available.
08:58 I'm trying to help the people on my team figure out what the common patterns should be, what
09:04 their handling should look like, what authentication should look like.
09:07 So that's one half of my job.
09:09 The other half of the job is to look at Azure's service APIs and try and ensure that we have
09:15 as much consistency as we can on that side as well.
09:18 Is it across different services or across different languages for a given service?
09:23 It's across different services for on the service API side of things.
09:27 The cross languages is a really interesting question because we have, I'm the architect
09:32 for Python.
09:33 We have, I have counterparts for JavaScript, TypeScript, Java for C#.net, obviously.
09:39 So what we are looking at is we're looking at these guidelines.
09:42 The capabilities have to be roughly the same for all languages.
09:45 If a service exposes a feature, all languages need to be able to expose that feature.
09:50 If a service has some specific failure modes or some quirks to it, everybody has to deal
09:56 with that.
09:56 So we're having discussions both how to best implement it in Python, but also making sure
10:00 that for all languages that we provide client libraries, that they all have the same capabilities
10:05 as well.
10:06 Right, right.
10:06 Yeah.
10:07 That's got to be an interesting intention.
10:08 Kate, how about you?
10:10 What do you do today?
10:11 I'm a program manager on the same team, but on my side of things, we like to call ourselves
10:17 just Azure developer experience, because one aspect of having that great experience with
10:22 the libraries is having great libraries.
10:25 But another is also making sure that the experience you get when you come to Azure as a developer
10:31 is that everything is straightforward.
10:32 If you go to the portal, we find the information that you need to get started and everything else.
10:37 So we like to work with other teams across Microsoft and across Azure to ensure that we bring that
10:43 consistency in other places as well.
10:45 My main job is to bring the voice of the developers using Azure into the designs that we have in
10:52 all of these places.
10:53 Cool.
10:53 That sounds really fun.
10:54 And you guys have some really neat stuff that we're going to dig into that you're using over
11:00 there.
11:00 The scale of Azure, kind of like I was saying before, it's a little bit, I mean, I guess if
11:06 people use AWS, they probably have a similar feeling.
11:08 Although I feel like Azure is even more so these days.
11:11 You just go there and it's like, you know, this huge menu of so many different choices.
11:17 It's a lot to keep track of and help people discover, right?
11:21 Yeah.
11:21 That's actually one of the main issues that we see for people who are just trying to, you
11:25 know, play around with these platforms.
11:27 And I think that applies across, you know, both Azure and AWS and everywhere else.
11:32 There's just so many different services to begin with that it might be kind of overwhelming
11:36 to know where you need to go.
11:38 So I think eventually our goal is to make sure that that choice becomes easier over time.
11:43 Have you guys thought of things like a beginner view or something, you know, for lack of a
11:50 better word, something that hides a lot of the stuff and just says, I'm a web developer
11:55 in this language.
11:55 Just show me the 10 things that I might care about.
11:59 Yeah.
11:59 I think we're trying to create some sort of tracks based on what we believe people might
12:03 be developing in our documentation so that it's easier to consume.
12:07 But I think, you know, maybe that's a good idea.
12:09 Just have a very simple beginner view.
12:11 Yeah.
12:12 Like show me everything.
12:13 I don't know.
12:13 It just, my experiences there has been, it's like, wow, I'm on a bit of a treasure hunt,
12:19 you know, to go through things here.
12:21 Yeah.
12:21 Very cool.
12:22 Now, you guys are also, Kate, you're also involved in some of the documentation and
12:26 tutorials and things like that.
12:28 Johan, are you as well?
12:29 Yes.
12:30 Yes, I am.
12:31 Yeah.
12:31 Actually, in our team, we were trying hard to have a greater push for creating better documentation
12:38 overall.
12:39 I think one of the problems with the auto-generated libraries is that that sometimes ends up being
12:44 not the best documented library in the end.
12:46 And that's why all of our developers really focus and spend time on writing better comments
12:53 and including more instructions for people to get started with.
12:56 Yeah.
12:56 Some of those auto-generated documentation things, it can be really tricky because it could
13:01 say like, you know, the function could be log in.
13:04 The documentation might be logs in user or upload uploads file, right?
13:10 It's just like one or two words or it just takes the name with a function and breaks it
13:14 out into like English-like words instead of compound words or whatever.
13:17 And it's just like, yeah, you can tell that, okay, this documentation, this is not going to
13:21 be helpful for me, right?
13:23 So it's good to have something really, you know, custom there, right?
13:26 Yeah, exactly.
13:27 Yeah.
13:27 One of the challenges, I guess, let me ask you, Kate, how much are you focused on Python
13:32 specifically and how much are you focused sort of cross-language working on this stuff?
13:36 I am focused on all languages to an extent because, you know, if we're creating some standards
13:41 for documentation that should apply across all of them, Python shouldn't be getting some
13:45 extra treatment just because it's Python.
13:47 I don't see why not.
13:49 Maybe that's just me.
13:51 Sure.
13:52 Let's document just Python, nothing else.
13:54 I was more asking, like, are you specifically on like the team that does the Python parts
13:58 and there's other people doing the same thing for JavaScript or are you like overseeing it
14:03 for all the languages?
14:03 Our split in terms of who cares about what is pretty complex on the PM team.
14:09 My focus was for Python in a sense of helping with user studies and helping with some telemetry
14:15 searches.
14:16 But in terms of all of these kind of overarching themes, I work with all of the languages.
14:21 Okay.
14:21 Yeah.
14:22 Yeah.
14:22 Cool.
14:22 This portion of Talk Python to Me is brought to you by Springboard.
14:28 Springboard offers online data science and machine learning education through their Machine
14:33 Learning Education Career Track program.
14:35 This is similar to an online machine learning boot camp with the difference that this career
14:40 track follows a project-based learning methodology where students work towards creating their own
14:45 portfolio of machine learning models.
14:47 And every student is paired with a machine learning expert who provides unlimited one-on-one mentorship
14:52 throughout the program via video conferencing.
14:55 This program was built for software engineers.
14:58 So to be eligible, you must have at least one year of experience.
15:01 You'll get a machine learning job guaranteed.
15:03 Springboard is the first company and still the only company in the U.S. to offer a machine learning
15:09 engineer career guarantee for students interested in changing careers.
15:13 That means either you get a job in the industry or your tuition is reimbursed.
15:17 So to get started today, just visit talkpython.fm/springboard or click the link in your
15:23 podcast player.
15:26 One of the challenges I can see here right away and your joke, Johan, sort of touches on it a little
15:33 bit.
15:33 I mean, I know you're joking, but you've got some endpoints, some cloud stuff that you want
15:39 to control and do things.
15:41 And every language has its own idiomatic way of working, right?
15:47 You know, Python has Pythonic code.
15:49 Other languages have, you know, whatever, whatever they call it.
15:54 Right.
15:54 But there's always an idiomatic way of doing things.
15:56 And sometimes that really lends itself to doing something quite different than other languages.
16:02 How do you trade off trying to make it so that somebody who's a C# developer working
16:08 with a Python developer, it's not like two different worlds, but also it doesn't feel like somebody
16:13 put C# or JavaScript into a .py file.
16:17 Yeah, you're absolutely right.
16:18 that's a constant tension.
16:20 And we've had plenty of discussions among the architects for the various languages here
16:24 on how to accomplish exactly that.
16:27 And the way that you iterate through things is different a little bit in Python than it
16:33 is in C#.
16:35 For Java, we have builder patterns for things.
16:38 A lot of interfaces, a lot of builders, a lot of patterns over in the Java world, for sure.
16:42 Yes.
16:42 So it is a challenge.
16:44 We're trying to make sure that we can talk when we talk about the abstract concepts, for
16:49 the most part that you can understand.
16:51 You can basically say if you talk about a specific thing, a specific noun, be that the blob or something
16:56 like that in one language, that you can map it down to what that means.
17:00 If I look at code in a different language, exactly what it looks like is going to be slightly
17:04 different because otherwise you're stuck with something that is going to be looking like
17:09 a specific programming language.
17:11 We certainly had that problem when I started here that we had some jokes.
17:15 Some of the usability studies that we had, the participants were arguing about what language
17:20 the original library was written in or the developer, what language they preferred.
17:25 Could they see the abstraction leaking through and go, I know, I think I know what this is.
17:30 Yes.
17:30 There was one of the early Python library usability studies that we had where they argued whether
17:35 or not it was a C# or Java developer that had written the Python library.
17:39 And it made me feel sad.
17:41 Neither of those are the right answer.
17:44 I don't know what the answer is, but that's not right.
17:46 Right.
17:47 Right.
17:48 Yeah.
17:49 Kate, you're back there laughing.
17:51 Do you have some stories like that that you experienced as well?
17:54 Yeah.
17:54 I mean, as Johan mentioned, being kind of blamed of over C#ying all of our libraries
18:00 was one of the, I think, initial kind of triggers that also made us want to explore the space
18:06 we're in and launch our team to begin with to make sure, you know, people no longer have
18:11 these doubts anymore.
18:12 You know, it's interesting, right?
18:14 Like, I'd like to hear what the user testing sort of uncovered for this.
18:19 But to me, when I use a library, I'm trying to write some program and then I'm like, oh,
18:24 I'm going to bring this library into the program.
18:26 And it's clearly not following the idioms of the language.
18:29 You can just tell.
18:30 You're just like, ah, there is something.
18:32 This, you just get this feeling like this wasn't really made for me.
18:36 It was kind of thrown together so I could still work with it.
18:40 But it's not for me.
18:41 You know what I mean?
18:42 As a whatever developer, right?
18:44 If you're Python and you're consuming something that looks like C# or you're a C#
18:50 developer, you're consuming something that looks like C++ or whatever it is.
18:53 It's not a great feeling like, oh, this is a this is the place they want me to be, right?
18:58 Yeah, no, absolutely.
19:00 I do think so.
19:01 Something that I've learned working with the other architects for the different languages
19:05 that each programming language community has its own personality as well.
19:09 And I found the Python users are generally extremely pragmatic, which is something that I very much
19:16 appreciate.
19:17 Yeah, I would say that's a pretty good classification, right?
19:21 They don't bring all these design patterns and these extra structures and whatnot in just
19:26 for the sake of saying it's well designed or well factored.
19:28 It's like, we'll do that if we need to.
19:30 But let's just put a function in a file and try that first.
19:33 Something like that, right?
19:34 Yep.
19:35 So, yeah, there are lots of other other discussions on what is the best way of doing React in some
19:42 of the other language communities.
19:43 I have not seen too many of those things show up in Python discussions.
19:48 Yeah, that's a tough challenge.
19:49 How do you guys, either of you or both of you handle this?
19:53 Because, yeah, every language has its culture and its community, but you can only immerse yourself
19:58 so deeply in so many different cultures, right?
20:01 And really get the zen of it.
20:04 So, how do you keep track of the other ones?
20:06 Or is that just you work with the other folks that are more those language experts or whatever?
20:11 The way our team is structured is that we have four separate teams with each of them has an
20:16 assigned PM, an assigned architect, and an engineering lead in the entire engineering kind of team.
20:22 We're really trying to become experts in that particular language.
20:26 So, it becomes easier to track this as, you know, we have kind of separate entities that are caring about each language.
20:33 Yeah, they have that expertise or whatever, right?
20:35 And they can just express that, sort of speak for their community at the meetings.
20:40 Yep.
20:40 That is my role on the architect board that we have for the Azure SDKs.
20:45 I try to bring the Python voice into the discussion and telling them where we are going to do something different
20:52 and what the reason for that is.
20:54 Yeah, for example, not from the Python universe of things, but from JavaScript, we were wondering what would be the best way to present, you know, JavaScript documentation.
21:03 Should it be Node?
21:04 Should it be JavaScript?
21:06 So, we just brought in.
21:08 I have no idea about anything JavaScript related.
21:10 So, I just brought in our engineering lead and our architect and kind of let them let me know better about what the community thinks about that particular space.
21:19 Yeah, interesting.
21:20 Well, there's just, you know, so many APIs and then you have to, you know, sort of cross product that with how many different languages are consuming those APIs.
21:30 It's really interesting.
21:32 So, one of the things I think would be fun to talk about is the test setup that you guys have, Kate, for user studies and observations.
21:41 Basically, the way we just approach trying to create a library to begin with is, you know, we have to learn better about the service itself.
21:49 And we start out by, you know, writing initial preview versions of the library with guidance from the architects, as well as closer work with the service team.
21:58 Once we have some sort of preview to work off, we set up some user studies.
22:04 We have a couple of labs in our building where we invite a lot of developers to come for, I think, two hours at a time.
22:16 Where we set up some basic tasks for them to complete using the preview library that we have.
22:21 You might give them the library to, like, blob storage and say, okay, use Python to upload this file to Azure blob storage.
22:28 So, or something like that, right?
22:30 And then just let them go?
22:31 Yeah, exactly.
22:33 So, we try to, you know, start with some simple tasks to see how these, you know, initial steps are going and then try to proceed to more complicated tasks later on.
22:43 And then basically see how they proceed with that.
22:47 We have kind of two different separate types of studies that we do.
22:50 One is focusing on the design of the API itself, where we often do not provide much documentation and see kind of intuitively if the naming makes sense and where do people end up searching for them.
23:02 Yeah, just auto-complete driven discovery.
23:04 Go with that and see what they do.
23:06 Right.
23:07 So, in that case, it kind of helps us to better understand what are, if, you know, our conventions of naming are discoverable and easy versus the other type of studies we have is to go more end-to-end.
23:19 And once we have some sort of set of docs to go off, see how people actually go about discovering it and seeing if there are any gaps that we have.
23:27 Yeah, quite interesting.
23:29 So, you're both checking the libraries and the API, but you're also checking maybe the documentation and the tutorials in that second case.
23:37 Yeah, exactly.
23:37 Because there is a lot of existing documentation for things that existed before us.
23:42 And making sure that, you know, all the places where people actually go to find it are updated and are not misleading is a big challenge overall.
23:50 Yeah, yeah, for sure.
23:51 I mean, I can't speak for Azure in general.
23:54 I know Azure's changed a lot.
23:55 So, that old tutorials don't make any sense anymore.
23:59 But in the Python space, that is a challenge.
24:02 You'll see tutorials written for Python 2 that'll tell people, like, well, go pip install virtual ENV so that you can create a virtual environment.
24:10 And it's like, no, no, no, that's already built into Python.
24:13 It's a module, VNV.
24:14 Yeah.
24:15 We don't need these things.
24:15 Like, there's these other steps.
24:17 Or even there's new ways to install Python into Windows, right?
24:21 Like, from the Windows Store and whatnot.
24:22 So, it's got to be a constant challenge of, like, that stuff was put there to help people.
24:28 And yet, it's kind of in the way now.
24:31 It'd be better if you didn't put anything.
24:32 And they might just have to go search for it and maybe discover the right way.
24:36 Yeah, I think for Python specifically, there has been an effort on our side to also try to consolidate all sorts of guidance, exactly like you mentioned, with a virtual environment that is consolidated into guidelines on how we are supposed to write tutorials.
24:51 I think it's still at a draft stage, but it's something we're definitely looking more into to make sure that, you know, the guidance we provide for the tutorials and similar texts also makes sense for the set of developers and for the language.
25:04 Yeah, this question for either of you.
25:06 How do you all deal with telling people to get started in Python?
25:09 Because on Windows, until recently, they didn't have the Python 3 command, right?
25:15 Steve Dower got the new version of Python into the Windows Store, and it comes with Python 3.
25:20 So, that's one thing that you can just say, always type Python 3 to get started.
25:23 But I've always found it very tricky to say, if you're on Windows, you do this.
25:28 If you're on Mac or Linux, you do that.
25:29 And, like, you kind of have to replicate that either-or bit through everything.
25:34 How do you all deal with that?
25:35 Today, it is either-or kind of for your operating system go here kind of thing.
25:40 We're doing less of now, and I think what it used to do more of in the past was that every getting started document had all of the instructions on how to set up Python first,
25:50 which meant it was not until page 3 or something like that that you got to what we're actually trying to tell you.
25:56 So, now, most of the getting started documents are that, assuming that you have Python installed.
26:02 And, by the way, if you don't know how to do that, go here.
26:05 Right.
26:05 So, you just have that one little blurb at the top of the document.
26:08 Yeah, that makes a lot of sense.
26:09 Now, I do want to come back to this lab that you have, Kate, because I think it's really interesting from a couple of angles.
26:16 But let me try to describe what I think the lab looks like, because I've seen some pictures and whatnot of it in action.
26:24 But this might not be the same thing.
26:25 So, the one that I saw, there was basically a room with some computers.
26:31 Some developers can go sit down there and either, like, group program or do separate programming.
26:36 But then there's also a one-way mirror, and you could sort of screen share.
26:40 You could watch what they're doing on the screen from behind the mirror or somewhere else
26:46 and sort of critique, you know, how well your library is holding up to them, poking at it, and things like that.
26:52 Is that basically what it is?
26:53 It is pretty much how it is.
26:54 So, we do try to bring two developers at a time to do pair programming.
26:58 We've kind of noticed that it's much easier for people to be more open about what they feel about the libraries and the experience overall.
27:07 It's kind of more natural to speak out and comment and see how it feels.
27:12 It's also usually easier to unblock the other person if one of them doesn't have experience in something.
27:19 Yeah, I can imagine from a user testing angle that if there's more than one person,
27:23 person that are working on it, they're going to just communicate verbally with each other because it's weird to just sit there and work with not, like, sharing what you're up to.
27:32 But that's exactly, like, what you would like them to do for you to understand what they're doing, right?
27:37 Exactly, exactly.
27:38 So, even some, you know, more random comments are always great.
27:41 And we've had some people be very open about what they thought at the time of, like, oh, you know, this is crap.
27:48 Like, this needs to change.
27:49 I don't understand why I'm even looking at this.
27:52 And kind of having another person there, I think, makes it less awkward for them.
27:56 Because you can imagine sitting in a tiny, tiny room, you know, just you and a computer in front of you, knowing that you're being observed by X amount of eyes.
28:05 It's a pretty stressful situation if you're on your own.
28:08 Yeah, the other person will help you forget that there's all those eyes or thoughts thinking about what you're doing.
28:13 Pretty interesting.
28:14 On our side of things, we also try to bring in, you know, a lot of people.
28:17 So, all of the devs that are actually working on the library try to always come to the study to see how their libraries are performing.
28:25 So that it's also easier for them to kind of understand how, you know, how users interact with the libraries.
28:31 And to understand where the struggles are.
28:34 And makes it kind of easier for them to understand how important it is to make it really good.
28:40 Yeah, yeah.
28:41 You think, oh, that's no big deal.
28:42 People will figure that out.
28:43 We don't need to do this.
28:44 And then you see four people stuck for 15 minutes trying to do it.
28:47 You're like, oh, actually, okay.
28:49 I see.
28:49 Exactly.
28:50 How often do you run these studies and how many people go through them?
28:53 We are pretty much booked all the time.
28:56 I think we have user studies, if not every week, then every other week, depending on what we're currently working on.
29:02 So we have usually, we manage to do, I think, two groups of studies per day, which is four people then per day.
29:12 Sometimes if there is something in particular that we're trying to test out, we can use two.
29:17 So that would mean most eight people a day.
29:19 Yeah.
29:20 Okay.
29:20 Well, the reason I think this is so interesting is because there are so many people creating packages and libraries and Python for others to use.
29:28 And many of them are extremely, extremely popular, right?
29:32 You know, over on PyPI, we have 215,000 libraries at the time of the recording.
29:37 You can go over to place up PyPI stats, I think.
29:41 Mm-hmm.
29:41 And you can check that out.
29:42 You can see the most downloaded libraries, right?
29:44 It's pretty cool.
29:45 There's a bunch that are like over, you know, well over a million downloads a day.
29:50 So there's these libraries and they're just being used in so many, by so many people in so many places.
29:56 And yet my feeling is that most people who build these libraries have very little visibility into how people feel about them until they raise to the level of like a GitHub request or a bug sort of complaint or something like that.
30:11 Right.
30:11 And having this discoverability and ability to observe people working on your libraries, it must be really interesting.
30:19 Yes, absolutely.
30:20 I mean, it's very gratifying when you hear someone talking about the library that you built in a positive way.
30:26 Yeah.
30:26 It can be very humbling if you thought that you had a great idea that turns out that no one understands how to use or may even have other choice words for the experience that they feel like they were exposed to.
30:40 Yeah, yeah, sure.
30:41 I had a rough day at work.
30:42 People hated what I built.
30:43 I thought it was great.
30:44 That's pretty harsh, but it does allow things to get pretty polished, I suspect.
30:48 So, Johan, do you have some examples of things that you've changed or realizations you've had through some of Kate's studies?
30:56 One of the experiences is you touched on the tab completion kind of discovery of things, but the Python community is extremely diverse in how they use things.
31:10 I think about half of the people that we've had in the usability study did not use any form of IDE at all.
31:18 Yeah.
31:18 When they did things.
31:19 Yeah.
31:19 So you let them pick their IDE.
31:21 It's not like, hey, we're all using Visual Studio Code because it'll work with all the languages and it's consistent or whatever.
31:26 You said, if you want to use Emacs, knock yourself out.
31:29 If you want to use PyCharm, go for that.
31:31 Is that how it went?
31:32 The last couple of studies, I think we set it up using VS Code, but we've certainly had PyCharm.
31:37 We've had people that even if you have VS Code in there, they only use the, well, they use VS Code as an editor at most.
31:44 And then they dropped into the terminal window and run everything interactively or run things through REPL immediately.
31:51 The use of a debugger, similar thing there.
31:55 I think there was one quote that one of the participants had that, yeah, I use a debugger for simple things, but if things get complicated, then it's pretty print all the way.
32:06 It's so funny.
32:06 I'm laughing because to me, it's like the opposite, right?
32:09 I'll print out something like, oh, this is getting hard, man.
32:11 I'm going to have to use a debugger and go figure this thing out.
32:13 Yeah, but it's an insight into how people feel and what people use and how important it is to make sure that things work well, that you have a good, that you implement Dunderstir and REPR so you can print out an object and you get some useful information.
32:29 Right, right.
32:30 In fact, people use it that way.
32:32 So understanding the importance of that is something that I've come to appreciate more.
32:36 Right, right.
32:36 Because if you just print it out, especially if you're in the non-IDE situation and you don't know what the properties are, and maybe you got something like a blob response object, it would just say blob response, add memory address, such and such, right?
32:51 As opposed to here's the URL or the bucket and whatever, right?
32:55 Like, I think that actually is a good point in general for libraries to consider the Dunderrepper and Sturr stuff so that not just if you actually just want to print it out, but those appear all over the place, right?
33:07 Those appear in debuggers.
33:09 They appear potentially in log files.
33:11 There's all kinds of stuff where having that is just like, it's just delightful.
33:16 You're like, oh, I can just see what this is right here.
33:19 I don't have to do extra work because somebody put in like a nice little repper that's good enough to know what's happening.
33:25 Right.
33:26 So there are many of those small things that are there.
33:30 I'm trying to reduce the number of types that we have in our libraries now because in order to call an API, if you want to call that from the REPL window, it's much easier to just use scalar types, for example.
33:40 And we have the wonderful keyword-only arguments in Python that makes life easier, whereas in JavaScript, I would create option objects with lots of properties on it.
33:49 Right, because in the option object, you could have the default value, but in Python, that's just part of the function.
33:54 Right.
33:55 So these kind of things that have changed a little bit in how I look at libraries.
34:02 This is interesting about these sort of compare and contrast libraries, trying to do exactly the same thing with people that know the system really well.
34:09 I think that's super interesting.
34:10 And I feel like a lot of that pragmatism that Python has is because there's a lot of flexibility in the lower-level stuff, right?
34:19 Like keyword-only arguments with default values and stuff allow you to avoid certain patterns or a global variable in a module is kind of like a singleton.
34:30 That's what you need.
34:31 You don't need a whole class for that and so on.
34:33 What was your experience coming from these different languages and seeing that across teams?
34:38 Is that about right or how has it been?
34:40 Absolutely right.
34:41 I still review Python code from other teams that come in and you can see where they're coming from, where they have extremely deep class hierarchies and things like that.
34:50 It makes sense if you wrote code in C# or Java, but type safety is not the same thing in Python as it is in C# where you will get the compiler error if you try to call something that doesn't exist.
35:02 There are other patterns that you can introspect things in Python that makes it more powerful.
35:09 This portion of Talk Python to me is sponsored by Clubhouse.
35:12 Clubhouse is a fast and enjoyable project management platform that breaks down silos and brings teams together to ship value, not features.
35:19 Great teams choose Clubhouse because they get flexible workflows where they can easily customize workflow state for teams or projects of any size,
35:27 advanced filtering, quickly filtering by project or team to see how everything is progressing,
35:32 and effective sprint planning, setting their weekly priorities with iterations and then letting Clubhouse run the schedule.
35:39 All of the core features are completely free for teams with up to 10 users.
35:43 And as Talk Python listeners, you'll get two free months on any paid plan with unlimited users and access to the premium features.
35:50 So get started today.
35:51 Just click the Clubhouse link in your podcast player show notes or on the episode page.
35:56 Kate, one thing I want to come back to because I do think it's really interesting and it's a challenge to this testing that you talked about is this not using an IDE or something that has some kind of
36:09 completion story for working with the code that Johan talked about.
36:14 What do you do if somebody wants to use one of those non-assisting editors or you're working with a language that doesn't provide enough information to actually get that even in tooling that would otherwise show it,
36:32 like some parts of JavaScript or something like that?
36:34 So as I mentioned before, we have a couple of different types of studies that we have.
36:39 So in cases where we can't rely on people discovering things on their own, we provide them with documentation.
36:45 We set up a couple of experimental documentation sites, which we can iterate on pretty quickly where we just direct people to a site and they can look at reference documentation.
36:57 I see.
36:58 So here's the SDK functions and types and whatnot.
37:01 Go look at that if you don't actually want to say dot and see the list or maybe an addition.
37:07 Yeah.
37:07 What about code that you all put into documentation?
37:11 Right.
37:12 You've got documentation for Python.
37:14 You've got documentation for JavaScript.
37:16 You both have these examples.
37:17 These libraries are changing over time.
37:20 The endpoints are changing potentially and, you know, extending or having better ways to do things.
37:25 How do you deal with that kind of stuff?
37:27 And do you have somebody go through and just go, all right, time to retry all the tutorials?
37:32 Or what do you do there?
37:33 Okay.
37:33 So there is a couple of different things we were trying to do.
37:37 So all of our packages live on our GitHub.
37:41 And one thing that we have already implemented for a couple of languages, but not all, is exactly as you mentioned.
37:49 It's kind of challenging to keep control of all of the code snippets and samples we have all across docs, all across all of other places.
37:59 So one thing that we are trying to do is to write tests that, you know, we test all the time on an ongoing basis and basically be able to pull out snippets from these sources into our readmes and then eventually into docs.
38:14 So that's something we have been putting a lot of work in.
38:18 Already, most of our stuff in our GitHub should be fresh because of this.
38:23 And we're now trying to ensure that the same thing happens when you're going to docs as we cross kind of reference our repositories.
38:30 Yeah, that's really cool.
38:31 You know, the challenge is that the little code snippet might need other stuff set up all around it, right?
38:37 Right.
38:37 The good thing is that usually in terms of the setup that's around it, that doesn't change as often because it's most, you know, having an instance of a service running or some sort of setting up the environment, which shouldn't change that often.
38:50 Yeah, I'm thinking of like a tutorial that says, first, import this thing.
38:54 Then a little bit later, it says, oh, make sure you set this connection string.
38:56 And then a little bit later, it says, here's how you download a file or whatever, right?
39:01 And you want to test the, here's how you download the file bit.
39:03 Like the stuff that comes before it still has to happen, but it's not part of that snippet.
39:08 Do you guys have examples like that you run into?
39:10 Yeah.
39:11 Yes.
39:11 So we do have that.
39:12 We have the, we're using things for generating a lot of this.
39:16 So using the literal includes.
39:18 So you can basically have a comment in the code that says this, this snippet or this part of my function or method corresponds to this example in there.
39:27 So when you run through the whole thing, it will do the setup and you can spread.
39:31 I see.
39:31 It'll like chain them together or something like that.
39:34 Yeah.
39:34 Yeah.
39:34 Okay.
39:35 Yeah.
39:35 The challenge is to keep everything up to date, as you alluded to.
39:38 If there's new functionality, we can, it's relatively easy or not impossible to make sure that things continue to work, to make sure that things are using the best practices.
39:49 If new best practices have come to light is much more challenging.
39:53 Right.
39:54 There's an old way of doing things.
39:55 Of course, it still works because you wouldn't break it for old software, but you don't want them to do that anymore.
40:01 Right.
40:01 Right.
40:01 So writing these examples, it's a little bit like giving someone a puppy.
40:06 It's going to be work forever.
40:08 That is true.
40:09 Yeah.
40:10 You got to walk it every day, even in the rain.
40:12 Yeah.
40:13 How about testing these libraries?
40:15 What do you all do for making sure that your library up to Azure or whatever service it's talking to is still good?
40:24 That's challenging.
40:25 We were talking at the Python meetup here in Portland about this.
40:30 And it's like, what if I wasn't connected and I wanted to run my test?
40:34 How do I even get that to work?
40:36 Or how do you test against real cloud environments, but you don't actually want to create a whole bunch of cloud environments?
40:43 You just want to see if the API is still hanging together or something like that.
40:46 It seems like a challenging thing to mock out all of Azure.
40:50 Yes.
40:51 So for what we're using is that we're using a recording framework.
40:55 So basically record all of the requests and responses.
40:58 Oh, interesting.
40:59 Okay.
41:00 Because it's very challenging.
41:02 And I, especially since we want to be able to take contributions from the public, we want them to be able to write tests and run tests, obviously.
41:11 But as with all clouds, it costs money to run things in the cloud.
41:15 And we don't want to charge.
41:17 Yeah.
41:17 Some are cheap, but some of them are really quite expensive to just do a little bit with.
41:21 Right.
41:22 If you're going to provision, I don't know, a certain type of VM with a GPU because you need to test the GPU a bit.
41:27 I don't know.
41:27 All right.
41:27 That could be expensive.
41:28 Yeah.
41:29 So what we have is that we make these recordings available so that you can run tests against these recordings.
41:35 It's still challenging to come up with new tests if you want to contribute to new tests.
41:39 Honestly, we don't have a good solution for that today.
41:42 But that's, we're using these recordings as much as we can.
41:45 And then we run, we call live tests on a nightly basis to make sure that things still work when actually using the services.
41:53 The recording is not lying.
41:54 The recording is not out of date, right?
41:56 What library are you using for this?
41:58 Is this a Python library?
41:59 Yes.
42:00 It's VCRPy that we're using now.
42:02 VCRPy.
42:03 That sounds very cool.
42:04 Or PyVCR.
42:04 Yeah.
42:05 If you go to our GitHub repository, github.com/Azure slash Azure dash SDK dash for dash Python.
42:15 Short name, I know.
42:15 But you can see that.
42:17 It rolls off the tongue, doesn't it?
42:18 Yes, it does.
42:19 Yeah, that's cool.
42:20 You can see how you guys are using it there.
42:22 Yeah, this is really cool.
42:23 I hadn't heard about this, but I'm sure that'll be useful for people out there in general, right?
42:27 Not just for people working with Azure tests.
42:29 Yes.
42:30 Yes.
42:30 And this is not the library that we have produced.
42:33 This is an open source component.
42:35 Yeah, absolutely.
42:37 So how do you deal with...
42:40 It's still hard for me to get my mind around how many challenges you have for trying to keep all this stuff consistent.
42:45 So you've got all these different endpoints.
42:47 You've got these different languages.
42:49 How do you collaborate at the documentation level for, okay, we're going to create this new thing that helps people use this.
42:56 The JavaScript team's got to do it.
42:58 Other teams got to do it.
43:00 And they got to be consistent.
43:01 What guidelines and advice do you have for that?
43:03 For the documentation specifically or for the designs?
43:06 More for the documentation, I guess, is what I was asking.
43:09 Although both are interesting.
43:10 But there's a lot of people or companies out there that have multiple technology stacks talking to their same thing.
43:16 What advice do you have for them to help all the different consumers evenly?
43:21 One thing that we are trying to do is to, first of all, better understand in terms of these different technology stacks, what are the most kind of often used patterns?
43:30 And that kind of informs how we write the quick starts and the tutorials.
43:34 On our side, the process, the way it looks is that our developers writing the libraries provide the reference documentation and then include some sort of basic guidance of what they see fit for the readmes that we have on our GitHub.
43:49 And then for each service, we have a content development team that kind of tries to take it and write a more, some sort of a tutorial that's more digestible by all sorts of different audiences.
44:04 And they have a better understanding of what exactly they need to provide for these particular use cases in these different scenarios.
44:10 Okay.
44:11 So there's like a canonical example.
44:13 And this is what we wanted for here.
44:15 And then make a JavaScript version, a Python version, and so on.
44:18 Right.
44:19 Exactly.
44:20 Okay.
44:20 That's quite cool.
44:21 So this whole user testing of people consuming libraries, it's really interesting.
44:26 And like I said, with the PyPI examples and those popular libraries, no, I suspect very few people are doing that.
44:33 A few exclude like the Azure libraries and other big companies.
44:38 But Opus Horse teams, they have these sprints and they have other feedback mechanisms.
44:43 All the stuff we talked about.
44:44 Do you have any advice where people might be able to recreate a little bit of this?
44:49 You know, maybe at the end of PyCon, have a sprint where instead of trying to add new features, it's like, let's do some user testing for people who are attending the conference and are very passionate about our library.
45:01 Or maybe try to grab somebody who's used the other library that's, you know, like Django versus Flask or something and have them come over and, you know, work in our world for a little bit.
45:10 You know, without having one of these labs and whatnot, but maybe using things like LiveShare or something.
45:18 What advice do you have for bringing some of these cool practices that you guys are doing and applying to your libraries to, you know, maybe much smaller groups or open source projects?
45:29 I can start and Johan can nod in.
45:31 But in terms of the conferences, I think that's an amazing idea.
45:34 And that's exactly what we're doing.
45:36 In November, we're attending a Microsoft conference, Microsoft Ignite.
45:40 And that's exactly what we're doing.
45:42 We had a couple, you know, stands set up, allowing people to kind of try things out on their own, as well as take surveys based on the feedback that they would have gotten by completing some basic tasks and kind of taking that down and kind of informing for their designs.
45:57 So, yeah, I think it's absolutely a great way to easily be able to work with people directly if you don't usually have that in person.
46:06 Yeah, it seems like it would be a cool thing for conferences to set up, right?
46:09 I mean, at Ignite, you guys set that up and that's really cool because you're doing that sort of stuff.
46:13 But I'm thinking at PyCon or some other, you know, major JavaScript conference or something, right?
46:20 The conference organizers could have a section where, you know, you could sign up.
46:24 Hey, I'm this library developer.
46:26 I'm on this team.
46:27 I want to have people go through my stuff.
46:29 And I just think it would be great.
46:30 I mean, I have no ability to set that up or anything.
46:32 I can throw ideas out there and maybe people run with it.
46:35 But it seems like it'd be really cool.
46:36 Yeah, that seems like a really good idea.
46:39 One challenge I think that we've seen with that is that very often people are happy to provide some feedback if this is something they really care about.
46:45 But if that's not the case, then usually they would expect some sort of swag or something in return, which is fair.
46:54 It's fair.
46:54 And as companies, it's fine, right?
46:56 It's easy for Microsoft to put together a certain amount.
46:59 But as a solo developer on an open source project, it's much harder.
47:03 But, you know, maybe like a cool little badge that's got the picture, like a little logo and like, you know, I helped improve this at PyCon or something.
47:12 Or I helped improve Flask or whatever.
47:14 I think people can make it work without too much involved.
47:17 Yeah, that sounds like a good idea.
47:19 Yeah.
47:19 Johan, what do you think?
47:20 You can look at it as a contribution.
47:21 It may not be in the form of source code as well.
47:24 All right.
47:24 So having the contribution somewhere in and who contributed to the last release, you know, use relative feedback from people.
47:31 That's a pretty good piece of currency to work with that they do have something that they could hand out easily, right?
47:36 You don't have, you know, little robots that you can hand out for everyone who helps as an open source developer.
47:41 But you do have, like you said, credit.
47:43 That's cool.
47:44 Yeah.
47:44 And it's important as well to be clear on the fact that we have seen a little bit of bias that people want to be nice to you when they give feedback.
47:53 But what you really need is the harsh feedback as well.
47:57 So that's one thing that I think we're getting better at making clear that the participants that we want to hear the good and the bad and not just the good.
48:06 Right.
48:07 Tell us the truth, not just what you think we want to hear, even though it won't make your day not as good, I guess.
48:13 Right.
48:13 Yeah.
48:14 So nice.
48:15 All right, you guys, we're getting short on time here.
48:18 So let me just ask you the final two questions and wrap it up.
48:23 So Kate, we'll go with you first.
48:25 If you're going to write some Python code, what editor do you use?
48:28 I recently started using VS Code and have been enjoying it.
48:32 Yeah.
48:32 Yeah.
48:32 You guys are doing a lot of cool stuff with VS Code.
48:34 It's definitely growing quickly.
48:36 Johan?
48:36 Yeah.
48:36 VS Code is probably my go-to editor right now, but I try to switch between them just to get a feel for what the experience is for using our libraries and the different editors.
48:47 Yeah, sure.
48:48 You definitely want to know, like, well, what's the experience in Sublime or what's it like in VI or whatever, right?
48:53 Right.
48:53 Sure.
48:53 I switch between them basically every week.
48:55 I use a new editor.
48:57 That's got to be a little hard.
48:59 I mean, I know you get used to it and all, but.
49:00 Plenty of mistakes.
49:03 Just, yes.
49:04 Yeah, I bet.
49:06 And then I'll throw this out to both of you.
49:08 Either you can throw something in a notable PyPI package.
49:12 Some library out there that you're like, oh, this thing is super cool.
49:15 We'll use this.
49:15 Maybe people haven't heard of it.
49:17 And if you want, you're welcome to use VCRPy.
49:20 Have you got another one?
49:22 Throw it out there.
49:23 That's a good question.
49:24 That one I would have to think about a little bit.
49:26 All right.
49:27 How about VCRPy?
49:28 That's a good one.
49:29 Is it PyVCR or VCRPy?
49:31 Yeah.
49:31 There is a VCRPy and it sounds like it's the right one, but I don't want to recommend swapping it to the wrong one to people.
49:40 Yeah.
49:41 Yes.
49:41 VCRPy.
49:42 VCRPy.
49:42 Okay, cool.
49:43 Yeah.
49:43 That's if you have things that are really hard to mock out and there's a complicated exchange, that seems like that's a pretty cool way to do it.
49:51 Yep.
49:51 Nice.
49:51 All right.
49:52 Final call to action.
49:53 People are out there.
49:54 Maybe they work at a big company.
49:56 They've got a set of APIs they've got to document across languages or they work at some other company that exposes their API to the world and they've got a document.
50:05 What advice do you have for them?
50:07 Take some of these things you've learned and apply it in their world.
50:10 It's not free to provide a good developer experience for people.
50:14 So understand that you need to, if you want to have a good developer experience, you need to invest time and effort into doing it.
50:22 So if you think that you can get away with having the same documentation for every language, auto-generate things, you will get part of the way, but it will not be great.
50:33 Yeah, absolutely.
50:34 I'll definitely show.
50:35 Kate, what do you think?
50:36 I think another important aspect is how, you know, ensuring that whatever you provide is really well documented.
50:43 It doesn't matter, you know, how much work we put into making sure that things are consistent and idiomatic.
50:49 In the end, people rely heavily on using documentation.
50:53 So making sure you create an easy and kind of consistent story around that is super important.
50:57 Awesome.
50:58 Good advice.
50:59 And thank you both for being here and sharing what you're up to.
51:02 It's really cool.
51:03 Thanks for having us.
51:04 Thank you.
51:04 Yep.
51:05 Bye.
51:05 This has been another episode of Talk Python to Me.
51:09 Our guests on this episode were Kate Olasueska and Johan Stenberg, and it's been brought to you by Springboard and Clubhouse.
51:16 Become a machine learning professional with Springboard's online bootcamp and get a job, guaranteed.
51:22 Visit talkpython.fm/springboard to apply today.
51:26 Clubhouse is a fast and enjoyable project management platform that breaks down silos and brings teams together to ship value, not features.
51:34 Fall in love with project planning.
51:35 Visit talkpython.fm/clubhouse.
51:39 Want to level up your Python?
51:41 If you're just getting started, try my Python Jumpstart by Building 10 Apps course.
51:46 Or if you're looking for something more advanced, check out our new async course that digs into all the different types of async programming you can do in Python.
51:54 And of course, if you're interested in more than one of these, be sure to check out our Everything Bundle.
51:59 It's like a subscription that never expires.
52:01 Be sure to subscribe to the show.
52:03 Open your favorite podcatcher and search for Python.
52:05 We should be right at the top.
52:06 You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm.
52:15 This is your host, Michael Kennedy.
52:17 Thanks so much for listening.
52:19 I really appreciate it.
52:20 Now get out there and write some Python code.
52:22 I'll see you next time.
52:24 Bye.
52:24 Bye.
52:24 Bye.
52:24 Transcription by CastingWords