WEBVTT 00:00:00.001 --> 00:00:03.960 Do you struggle to know where to start with the wide range of Python's data visualization 00:00:03.960 --> 00:00:10.340 frameworks? Not sure when to use Plotly versus Matplotlib versus Altair? Then this episode is 00:00:10.340 --> 00:00:15.380 for you. We have Chris Moffitt, a Talk Python course author and founder of Practical Business Python, 00:00:15.380 --> 00:00:20.120 back on the show to discuss getting started with Python's data visualization frameworks. 00:00:20.120 --> 00:00:26.300 This is Talk Python To Me, episode 384, recorded September 29th, 2022. 00:00:26.300 --> 00:00:44.200 Welcome to Talk Python To Me, a weekly podcast on Python. This is your host, Michael Kennedy. 00:00:44.200 --> 00:00:48.820 Follow me on Twitter where I'm @mkennedy and keep up with the show and listen to past episodes 00:00:48.820 --> 00:00:55.520 at talkpython.fm and follow the show on Twitter via at talkpython. We've started streaming most of our 00:00:55.520 --> 00:01:01.360 episodes live on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube to get 00:01:01.360 --> 00:01:07.680 notified about upcoming shows and be part of that episode. This episode is sponsored by Microsoft for 00:01:07.680 --> 00:01:14.120 Startups Founders Hub. Check them out at talkpython.fm/founders hub to get early support for your 00:01:14.120 --> 00:01:20.100 startup. And it's brought to you by us over at Talk Python Training. Did you know we have one of the 00:01:20.100 --> 00:01:26.060 largest course libraries for Python courses? They're all available without a subscription. So check it 00:01:26.060 --> 00:01:32.180 out over at talkpython.fm. Just click on courses. Transcripts for this episode are sponsored by 00:01:32.180 --> 00:01:39.180 Assembly AI, the API platform for state-of-the-art AI models that automatically transcribe and understand 00:01:39.180 --> 00:01:47.100 audio data at a large scale. To learn more, visit talkpython.fm/assembly AI. Chris, welcome back to 00:01:47.100 --> 00:01:52.260 Talk Python To Me. Thank you. Glad to be here again. I'm glad to have you back. We originally 00:01:52.260 --> 00:01:59.420 had you on to talk about the work that you're doing at Practical Business Python. And looking at the page 00:01:59.420 --> 00:02:05.720 now, your last article, pandas group by warning on the 26th, which as of recording, two days ago, 00:02:05.720 --> 00:02:11.240 looks like you're still really active on Practical Business Python. I am. It's been a while. To be honest, 00:02:11.380 --> 00:02:15.440 I spent a lot of time working on the course that we'll talk about in a moment. And so some of this 00:02:15.440 --> 00:02:20.920 stuff fell by the wayside. And I think like everybody in the COVID times have been a weird 00:02:20.920 --> 00:02:26.600 time warp for us all. So I haven't spent as much time on it as I would like, but I am getting back 00:02:26.600 --> 00:02:33.120 into it. And as you mentioned, just put a short article up there that in some ways kind of encapsulates 00:02:33.120 --> 00:02:38.920 a lot of what I want to do with Practical Business Python is I'm writing in this specific article in general 00:02:38.920 --> 00:02:45.320 on the blog about problems that I encounter that I think can help other people. And this was a short 00:02:45.320 --> 00:02:51.200 article about some kind of gotcha behavior with group by that I've been bitten by a couple times. 00:02:51.200 --> 00:02:57.220 And this most recent time, I decided, you know what, I need to write about this and share it with people 00:02:57.220 --> 00:03:00.320 so that hopefully they're not going to fall into the same traps that I had. 00:03:00.320 --> 00:03:05.900 I love when you write something as kind of a note for yourself or a roadmap for yourself. And then 00:03:05.900 --> 00:03:10.400 later you go back and you search for it. And you're like, I got to remember how this went. 00:03:10.400 --> 00:03:15.100 And like hit number one as your result. You're like, okay, I guess I don't remember this, 00:03:15.100 --> 00:03:19.040 but I'm going back to it. And I, you know, my future self thanks my old self for it, right? 00:03:19.040 --> 00:03:24.600 Yes, exactly. And sometimes I even remember I wrote an article about it and will refer to it. 00:03:24.600 --> 00:03:27.680 And then a lot of times you're right. I'll do a Google search. I'm like, oh yeah, 00:03:27.680 --> 00:03:29.400 I did solve that before. 00:03:30.000 --> 00:03:34.220 How interesting. Yeah, cool. What a great resource. People should be definitely checking 00:03:34.220 --> 00:03:38.440 this out. If you know, you want to do data science, Python data science intersected with 00:03:38.440 --> 00:03:44.660 Python. Of course you were on the show a couple of times ago, way back on 2019 and the before times 00:03:44.660 --> 00:03:49.480 escaping Excel hell with Python and Pandas. Basically you were making the case for 00:03:49.480 --> 00:03:53.160 using the Python data science stack instead of Excel, right? 00:03:53.160 --> 00:03:58.140 Yes, absolutely. Yeah. And you know, the blog and a lot of my experience has been 00:03:58.140 --> 00:04:05.480 doing data analysis, data manipulation, data science, and trying to leverage the power of Python. 00:04:05.480 --> 00:04:12.600 And most people in a business setting, their go-to data science, data science tool or data analysis 00:04:12.600 --> 00:04:18.420 tool is Excel. And it has its place. I'm not advocating we get rid of Excel, but I think there 00:04:18.420 --> 00:04:24.700 are a lot of things that we can do with Python that are much quicker, much less error prone and 00:04:24.700 --> 00:04:30.440 much more efficient than trying to do it in Excel. And Excel is one of those tools where there's such a 00:04:30.440 --> 00:04:36.740 wide range of usage. There's some people that are experts and can do really complex, very efficient 00:04:36.740 --> 00:04:42.180 things in Excel, but there's a lot of people that treat Excel like the proverbial hammer. Everything's a 00:04:42.180 --> 00:04:47.720 nail. So you try and use Excel to do everything from data cleaning to building out your financial 00:04:47.720 --> 00:04:53.380 statements to, I don't know, machine learning. And it's probably not really the best tool for all of 00:04:53.380 --> 00:04:59.580 those things. I think Python really fills a nice niche and had the good fortune of using it for a 00:04:59.580 --> 00:05:06.240 lot of different types of activities in my business career and wanted to talk about that in the podcast. 00:05:06.240 --> 00:05:10.980 You know, one thing that occurred to me, just thinking about the code that I've seen for like 00:05:10.980 --> 00:05:18.320 visualizing data and so on with Python compared to writing algorithms or web apps or something, 00:05:18.320 --> 00:05:24.580 it seems to me that the amount of Python that you have to know and able to maybe import pandas, 00:05:24.580 --> 00:05:30.120 load a CSV and then graph it, you almost hardly need to know Python at all. And it's more about 00:05:30.120 --> 00:05:35.300 knowing the APIs of the various visualization frameworks like Mapplotlib or Seaborn, right? 00:05:35.300 --> 00:05:39.720 No, absolutely. I agree completely. I mean, you basically need to know, like you said, how to 00:05:39.720 --> 00:05:45.240 import and even backing up probably the most, the biggest challenge is getting Python set up on your 00:05:45.240 --> 00:05:49.340 system, depending on the system you have and everything, getting your environments all squared 00:05:49.340 --> 00:05:54.600 away. But once that's done, you're right. The data visualization libraries, no matter which one you 00:05:54.600 --> 00:05:57.820 choose, essentially are knowing how to call functions. 00:05:57.820 --> 00:06:03.420 Yeah, exactly. And they all often seem to have their own little DSL domain specific language for 00:06:03.420 --> 00:06:05.400 what they've decided they're going to do, right? 00:06:05.400 --> 00:06:10.520 Yes, exactly. And I think that's, you know, part of the challenge is everybody thinks a different way. 00:06:10.520 --> 00:06:16.340 And sometimes like a library might make a lot of sense to you, but other people, it doesn't. And so 00:06:16.340 --> 00:06:20.160 that's a lot of where I think some of the challenges in the visualization landscape are, 00:06:20.160 --> 00:06:24.880 are trying to find that right API that makes sense for the actual business problems or 00:06:24.880 --> 00:06:28.560 visualization problems that you have and how, how it fits in your brain. 00:06:28.560 --> 00:06:34.820 Indeed. So if people are coming from a business perspective and maybe Excel is where they or their 00:06:34.820 --> 00:06:39.120 colleagues have been working, I definitely recommend people go check out episode 200. And then there was 00:06:39.120 --> 00:06:43.620 also 10 tips to move from Excel to Python, a lot of common themes here. 00:06:43.620 --> 00:06:44.180 Yes. 00:06:44.180 --> 00:06:48.660 So normally at the beginning of the show, I ask people how they got into programming in Python. 00:06:48.660 --> 00:06:54.120 I've got you to answer that at least once, maybe twice, but probably a third time is, 00:06:54.120 --> 00:06:57.560 is not required. So just give us an update on what you've been up to. 00:06:57.560 --> 00:07:04.340 Sure. I am still working in the medical device industry. My job doesn't require Python, but my job 00:07:04.340 --> 00:07:10.940 does involve a lot of data analysis and working with not necessarily large sets of data, but sometimes 00:07:10.940 --> 00:07:16.700 data sets that are big enough that it's a little bit painful to work in Excel. And I just continue to 00:07:16.700 --> 00:07:23.800 find Python as the tool that I reach for when I need to do data analysis. And I continue to use it to 00:07:23.800 --> 00:07:31.720 build repeatable data analysis pipelines. I use it to clean data, maybe take external data that we buy 00:07:31.720 --> 00:07:38.180 from a third party and integrate it with internal data sets, or as we'll talk about doing data 00:07:38.180 --> 00:07:42.920 visualization. I think there are a lot of things that Python can do from a data visualization perspective 00:07:42.920 --> 00:07:49.740 that are easier than trying to use Excel or some of the other tools that are available in a traditional 00:07:49.740 --> 00:07:56.000 environment. So I continue to live and breathe Python. And I would say, you know, every week, 00:07:56.000 --> 00:08:02.180 I use it a little bit, some weeks, a lot more than others, but continue to enjoy kind of that blend of 00:08:02.180 --> 00:08:07.000 Python, real world problems, and not just for software development's sake. 00:08:07.280 --> 00:08:12.340 Do the people you work with, since program is not officially like your title, do they look at you 00:08:12.340 --> 00:08:13.300 as kind of like a wizard? 00:08:13.300 --> 00:08:14.320 A little bit. Yeah. 00:08:14.320 --> 00:08:14.880 Yeah. 00:08:14.880 --> 00:08:22.560 They do. And I mean, I've done one of the things that I've worked on was a forecasting tool to help 00:08:22.560 --> 00:08:27.080 us forecast our business performance, you know, anticipate what that's going to look like, which I 00:08:27.080 --> 00:08:32.800 think a lot of people did over the COVID times trying to forecast was extremely challenging. And they just 00:08:32.800 --> 00:08:38.260 call it, well, what does the Python tool say? So they don't really, you know, get what the underlying 00:08:38.260 --> 00:08:44.620 libraries are, what's going on, but they do associate my name with Python. And I don't know if they really 00:08:44.620 --> 00:08:48.940 understand what it all means. I certainly try and explain it, but at the end of the day, you know, 00:08:48.940 --> 00:08:52.740 happy with the results. And like you said, they do kind of think there's a little bit of a superpower, 00:08:52.740 --> 00:08:53.900 as you frequently say. 00:08:54.220 --> 00:08:54.840 That's fantastic. 00:08:54.840 --> 00:08:55.880 Or knowing that. 00:08:55.880 --> 00:09:00.400 Well, when you go and look at the basic things that a lot of the tools we're going to talk 00:09:00.400 --> 00:09:04.320 about, the frameworks that we're going to talk about result in, you could easily look over at 00:09:04.320 --> 00:09:09.300 something like Excel and go, well, six, one, half dozen of the other, that kind of the same, 00:09:09.300 --> 00:09:14.800 right? But then the amount of customization and specialization that if you go a little bit 00:09:14.800 --> 00:09:20.360 beyond just the, give me a histogram of this data, but you know, you dig into it a bit, 00:09:20.360 --> 00:09:23.420 it goes far beyond what things like Excel are able to do. 00:09:23.420 --> 00:09:29.240 Yes, absolutely. And it's funny you mentioned histogram, like even a few years ago, there wasn't 00:09:29.240 --> 00:09:35.560 even really a histogram function in Excel, you know, kind of one of the basic functions I use pretty 00:09:35.560 --> 00:09:40.740 much for any new data set is the histogram and Excel didn't have one out of the box. You could build one, 00:09:40.740 --> 00:09:44.880 but it wasn't there. And I think that just kind of speaks to Excel is approaching, 00:09:44.880 --> 00:09:49.760 approaching visualization from a very different perspective, I think, than the Python tools. 00:09:49.960 --> 00:09:56.040 Excel is much more of a, how can I quickly create something and kind of guide the user through it, 00:09:56.040 --> 00:10:01.440 and then give them almost infinite options to customize the visualization. So you can go in and 00:10:01.440 --> 00:10:09.100 tweak any individual data point or axes or colors, which, you know, is useful for getting started, 00:10:09.100 --> 00:10:14.280 but I don't think it scales very well. And it also doesn't have some of the more sophisticated, 00:10:14.280 --> 00:10:18.340 complex visualizations that you can do with the Python libraries that are out there. 00:10:18.600 --> 00:10:24.680 Like Sunburst and other amazing things. Exactly. Yeah, cool. All right. Well, let's mention your 00:10:24.680 --> 00:10:30.160 course real quick, because what we're going to cover today is inspired by the course. It's not the same 00:10:30.160 --> 00:10:34.280 thing as the course, but you recently published a course over in Talk Python, Python data visualization. 00:10:34.280 --> 00:10:42.160 This is really nice because the visualization landscape is so diverse and varied, and it's hard to pick, 00:10:42.160 --> 00:10:48.340 I think, what should I choose? How do I find something that, as you say, fits your brain? How do you find 00:10:48.340 --> 00:10:53.240 something that's modern and maybe is interactive or is good for publications? And so in this course, 00:10:53.240 --> 00:10:58.480 you know, you kind of just do a survey of many of the popular options. You want to give a quick 00:10:58.480 --> 00:11:01.120 elevator pitch on this and then we'll dive into the topics. 00:11:01.120 --> 00:11:08.400 Sure. Like you said, I think the Python landscape is, or Python visualization landscape is so rich. There 00:11:08.400 --> 00:11:14.760 are so many options and many people are discouraged and don't even know where to start. And I think when 00:11:14.760 --> 00:11:20.200 you try and marry up that landscape with the different types of problems you can solve with visualization, 00:11:20.200 --> 00:11:26.560 it really requires you to spend a little bit of time with each one of the main libraries or several of the 00:11:26.560 --> 00:11:32.800 main libraries to think about how they're going to solve your problem. And so the course steps through 00:11:32.800 --> 00:11:38.480 many of the common libraries gives you, like we were talking about, the amount of Python that you need 00:11:38.480 --> 00:11:43.980 to know to do visualization is fairly minimal. So we don't spend a lot of time on Python. It's more about 00:11:43.980 --> 00:11:51.000 the API and how to interact with each visualization library to use it in the way that it's intended. 00:11:51.000 --> 00:11:55.400 And then I also think it's important when people are thinking about visualization, 00:11:55.400 --> 00:12:01.800 it's not just about the library. It's also about thinking about visualization. And as I point out in 00:12:01.800 --> 00:12:05.960 the beginning of the course, visualization is a really broad topic. I mean, there are computer science 00:12:05.960 --> 00:12:11.600 classes that you can take a whole semester on. There are many, many books that are really strong. 00:12:11.600 --> 00:12:17.880 Edward, tough day comes to mind or tough. Yeah, exactly. Exactly. And when you start thinking 00:12:17.880 --> 00:12:23.680 about visualization that way, it changes the way you approach visualization from the Excel approach of 00:12:23.680 --> 00:12:29.580 how do I just build a bar chart to what is the information I have and how am I trying to convey it 00:12:29.580 --> 00:12:37.920 to the end audience? And so I spent a lot of time talking about that. And then I also think that there 00:12:37.920 --> 00:12:44.520 is a data manipulation component to this. Once you start to understand how to structure your data 00:12:44.520 --> 00:12:50.480 correctly using maybe not correctly, but most efficiently for data visualization. Once you start 00:12:50.480 --> 00:12:56.220 structuring your data in that tidy format, then it's very easy to iterate on your visualizations 00:12:56.220 --> 00:13:02.340 and zero in on what's going to work best for you and your end users. So that's what the course talks 00:13:02.340 --> 00:13:09.440 through. It talks to the concepts, real world examples of developing visualizations using many of the libraries 00:13:09.440 --> 00:13:14.940 we'll talk about and then how to customize those visualizations from very basics all the way up to 00:13:14.940 --> 00:13:21.120 building custom dashboards that can be highly interactive and potentially deployed for others to use. 00:13:21.120 --> 00:13:25.900 Cool. Yeah. I definitely learned a ton going through your course. It's over at talkpython.fm 00:13:25.900 --> 00:13:33.060 slash data viz. People can check that out. Let's get maybe a high level landscape. So view of the landscape 00:13:33.060 --> 00:13:39.840 before we dive into these topics, because there's different branches of this, I guess you would say. 00:13:39.840 --> 00:13:46.900 So there's a GitHub repo that you point out by Nicholas Rogier. I'm not sure how to say his name. Sorry, Nicholas. 00:13:46.900 --> 00:13:53.720 This is an adaptation of Jake VanderPlas' graphic about the landscape here. And so, you know, 00:13:53.720 --> 00:13:58.540 how's this picture fit for you? Do you think this is pretty accurate? Let me see if I can. 00:13:58.540 --> 00:14:03.740 I do, because I think it points out a couple different things when you start thinking about visualization. 00:14:03.740 --> 00:14:07.560 So one of the key things that you can 00:14:07.560 --> 00:14:10.700 glean from looking at this and for the people that are listening, 00:14:10.700 --> 00:14:13.900 it's a kind of starburst plot and 00:14:13.900 --> 00:14:18.760 you've got a whole bunch of linkages between these different visualization libraries. 00:14:18.760 --> 00:14:20.820 And something that jumps out is 00:14:20.820 --> 00:14:24.720 Matplotlib is at the center of many of these libraries 00:14:24.720 --> 00:14:29.040 and it's the foundational tool that's used to build other libraries. 00:14:29.040 --> 00:14:30.940 And so I think that's a key concept. 00:14:30.940 --> 00:14:32.780 You can use Matplotlib on its own 00:14:32.780 --> 00:14:38.960 or knowing Matplotlib makes it easier to use some of these other libraries that are built on top of it. 00:14:38.960 --> 00:14:42.380 And the other thing is not shown on this, but, you know, 00:14:42.380 --> 00:14:46.880 from a history perspective, Matplotlib is sort of the grandfather of all 00:14:46.880 --> 00:14:48.700 Python visualization libraries. 00:14:48.800 --> 00:14:54.640 It's been around a long time and what you see on the, for people that can see the visualization, 00:14:54.640 --> 00:14:56.520 on the left-hand side, 00:14:56.520 --> 00:15:01.160 JavaScript visualization is a little bit more modern, like you said, 00:15:01.160 --> 00:15:05.380 and what people expect when you think about an interactive visualization tool. 00:15:05.380 --> 00:15:09.560 And so that's a different approach for visualization that has some pluses and minuses. 00:15:09.640 --> 00:15:12.180 So you've got that Matplotlib and JavaScript, 00:15:12.180 --> 00:15:17.920 I think are the key distinctions for how libraries are constructed for visualization in Python. 00:15:17.920 --> 00:15:20.800 And there are some other ones, there are some other libraries, 00:15:20.800 --> 00:15:26.040 but the two that I really focus on are either Matplotlib based or JavaScript based. 00:15:26.040 --> 00:15:26.360 Sure. 00:15:26.360 --> 00:15:29.660 And the Matplotlib side, we have things like Matplotlib itself, 00:15:29.660 --> 00:15:34.840 but also Pandas and Seaborn and Scikitplot, GGplot, 00:15:34.840 --> 00:15:37.140 stuff that people may be familiar from there. 00:15:37.560 --> 00:15:42.080 And then on the JavaScript side, we've got Bokeh and Plotlib, 00:15:42.080 --> 00:15:44.580 some of the more, as you say, interactive ones. 00:15:44.580 --> 00:15:48.740 And D3JS is in there as sort of a foundational item as well. 00:15:48.740 --> 00:15:49.340 Exactly. 00:15:49.340 --> 00:15:49.980 Altair. 00:15:49.980 --> 00:15:51.460 I don't even see Altair in this list. 00:15:51.460 --> 00:15:53.320 Maybe it's in there somewhere, but it is. 00:15:53.320 --> 00:15:53.680 Yeah. 00:15:53.680 --> 00:15:56.180 It's over there hanging out close to Matplotlib. 00:15:56.180 --> 00:15:57.120 Exactly. 00:15:57.120 --> 00:15:57.700 Yes. 00:15:57.700 --> 00:15:59.220 More strongly related to JavaScript. 00:15:59.220 --> 00:15:59.680 Got it. 00:15:59.680 --> 00:16:04.880 The third one in the three main branches here is OpenGL. 00:16:04.880 --> 00:16:06.800 What do you know about the OpenGL ones? 00:16:07.040 --> 00:16:08.060 Yeah, it's a good question. 00:16:08.060 --> 00:16:09.680 I don't use them a whole lot. 00:16:09.680 --> 00:16:12.440 I don't have a whole lot of experience with them. 00:16:12.440 --> 00:16:20.040 I think there are certainly maybe certain like very high volume data analysis that might be 00:16:20.040 --> 00:16:23.640 where performance is really important, where some of those OpenGL libraries, 00:16:23.640 --> 00:16:25.540 I think were originally founded. 00:16:25.540 --> 00:16:34.020 But I get the sense that most people are gravitating towards either those Matplotlib or the JavaScript ones that we've talked about. 00:16:34.020 --> 00:16:38.700 I don't have as much experience nor see as much development there with those libraries. 00:16:38.700 --> 00:16:48.500 Yeah, maybe they're about visualizing changing data that is flowing in real time and you can actually see a change because, you know, OpenGL is basically a graphics library for animation, right? 00:16:48.500 --> 00:16:48.800 Sure. 00:16:48.800 --> 00:16:49.200 Yeah. 00:16:49.200 --> 00:16:51.740 I think that real time component is a good distinction. 00:16:51.740 --> 00:16:52.480 Yeah, probably. 00:16:53.120 --> 00:17:02.440 Now, before people run fleeing to the hills, just because some of these projects are grouped under JavaScript doesn't mean you have to write JavaScript to use them, right? 00:17:02.440 --> 00:17:02.900 Correct. 00:17:02.900 --> 00:17:03.500 Absolutely. 00:17:03.640 --> 00:17:03.760 Yeah. 00:17:03.760 --> 00:17:10.120 And all the ones that I cover and the ones on here actually have a really nice API on top of it. 00:17:10.120 --> 00:17:12.740 The JavaScript is abstracted away. 00:17:12.740 --> 00:17:19.960 And in some ways, there's some benefits, like I'll call it Altair because it leverages VegaLite. 00:17:19.960 --> 00:17:27.540 And anytime that underlying JavaScript library is updated, you kind of get all of those benefits for free through Altair. 00:17:27.540 --> 00:17:31.980 So it's, you know, in the spirit of open source, building on the shoulders of others. 00:17:32.360 --> 00:17:35.920 And there's a lot of benefits to having that JavaScript foundation. 00:17:35.920 --> 00:17:40.640 And you don't need to understand JavaScript to use any of them. 00:17:40.640 --> 00:17:40.840 Yeah. 00:17:40.840 --> 00:17:48.780 And Altair really is like a transformation layer into a VegaLite definition, which is then processed by JavaScript to be rendered. 00:17:48.780 --> 00:17:51.260 So there's even this sort of separation layer. 00:17:51.260 --> 00:17:55.160 So it's not like you necessarily have to change your code to pick up the changes there. 00:17:55.160 --> 00:17:55.380 No. 00:17:55.380 --> 00:18:00.920 And I think at some point, depending on how deep down the rabbit hole you go, there could be points where, okay, it is really, 00:18:01.160 --> 00:18:07.240 if you're doing something highly custom or something really unique, understanding what's going on under the hood could be useful. 00:18:07.240 --> 00:18:10.700 But you can get pretty far without having to know that. 00:18:11.240 --> 00:18:21.240 Maybe you're trying to make a book or an article and you need something just so you're like, you know what, I'm just going to dump out the VegaLite definition and just add two things to it, but create it through Altair. 00:18:21.240 --> 00:18:21.880 Exactly. 00:18:25.080 --> 00:18:29.540 This portion of Talk Python To Me is brought to you by Microsoft for Startups Founders Hub. 00:18:29.540 --> 00:18:31.620 Starting a business is hard. 00:18:31.620 --> 00:18:36.560 By some estimates, over 90% of startups will go out of business in just their first year. 00:18:36.560 --> 00:18:46.040 With that in mind, Microsoft for Startups set out to understand what startups need to be successful and to create a digital platform to help them overcome those challenges. 00:18:46.040 --> 00:18:48.740 Microsoft for Startups Founders Hub was born. 00:18:48.740 --> 00:18:55.380 Founders Hub provides all founders at any stage with free resources to solve their startup challenges. 00:18:55.380 --> 00:19:04.000 The platform provides technology benefits, access to expert guidance and skilled resources, mentorship and networking connections, and much more. 00:19:04.560 --> 00:19:13.580 Unlike others in the industry, Microsoft for Startups Founders Hub doesn't require startups to be investor-backed or third-party validated to participate. 00:19:13.580 --> 00:19:16.260 Founders Hub is truly open to all. 00:19:16.260 --> 00:19:17.760 So what do you get if you join them? 00:19:17.760 --> 00:19:26.020 You speed up your development with free access to GitHub and Microsoft Cloud computing resources and the ability to unlock more credits over time. 00:19:26.020 --> 00:19:35.760 To help your startup innovate, Founders Hub is partnering with innovative companies like OpenAI, a global leader in AI research and development, to provide exclusive benefits and discounts. 00:19:35.760 --> 00:19:41.180 Through Microsoft for Startups Founders Hub, becoming a founder is no longer about who you know. 00:19:41.180 --> 00:19:54.720 You'll have access to their mentorship network, giving you a pool of hundreds of mentors across a range of disciplines and areas like idea validation, fundraising, management and coaching, sales and marketing, as well as specific technical stress points. 00:19:54.720 --> 00:19:59.960 You'll be able to book a one-on-one meeting with the mentors, many of whom are former founders themselves. 00:19:59.960 --> 00:20:05.080 Make your idea a reality today with the critical support you'll get from Founders Hub. 00:20:05.080 --> 00:20:11.040 To join the program, just visit talkpython.fm/foundershub, all one word, no links in your show notes. 00:20:11.040 --> 00:20:13.340 Thank you to Microsoft for supporting the show. 00:20:13.340 --> 00:20:20.520 Well, let's start with the granddaddy, as you put it, Matt Plotlib. 00:20:20.520 --> 00:20:29.060 So, there's so many shiny new things, but you make the case that knowing Matt Plotlib is still really worthwhile and really important, right? 00:20:29.060 --> 00:20:31.320 Because it is the foundation of so many things. 00:20:31.320 --> 00:20:31.960 Exactly. 00:20:31.960 --> 00:20:35.400 It is the foundation, and it's so important to know it. 00:20:35.400 --> 00:20:40.040 And also, it is extremely highly customizable. 00:20:40.300 --> 00:21:06.960 And I was thinking about this, and in some ways, I think if Matt Plotlib, if you stripped out a lot of the old and like everything that they've done in the last three to five years and just focused on that new, and if you could erase all the old tutorials and all the maybe ancient answers on Stack Overflow and just focused on the new stuff, people would have a much different perspective on Matt Plotlib. 00:21:06.960 --> 00:21:19.460 I think it's hard when you have something that's been around so long and has evolved that you do get some cases out there where, okay, an API or the way they approached a visualization in the past was clunky. 00:21:19.460 --> 00:21:23.240 But in the five or 10 years since, it's improved. 00:21:23.240 --> 00:21:26.640 And if you use the new and improved, it's really streamlined and really easy. 00:21:26.640 --> 00:21:32.260 So I think it's important that people not get turned away right off the bat from Matt Plotlib. 00:21:32.260 --> 00:21:47.120 And there are certain types of visualizations where if you need a high degree of customization, if you want to print it out or include in a manuscript or a book, Matt Plotlib is really useful and powerful for it. 00:21:47.120 --> 00:22:00.680 In that distinction about the new and the old, originally when Matt Plotlib came out, correct me if I'm wrong, it's my limited understanding, is it was somewhat modeled on the Matlab way of programming and it had this imperative API. 00:22:00.680 --> 00:22:03.360 The new one is more object oriented, isn't it? 00:22:03.460 --> 00:22:04.860 Yes, that's exactly right. 00:22:04.860 --> 00:22:14.160 So there's this state based interface that was based on Matlab and for people making that transition from Matlab to Matplotlib, it was seamless, right? 00:22:14.160 --> 00:22:15.980 And they really understood it and made sense. 00:22:15.980 --> 00:22:20.280 But that way of doing things doesn't, it's not really Pythonic. 00:22:20.280 --> 00:22:30.720 And so the object oriented interface is newer and is clearly the direction that Matplotlib documentation wants to steer you down that path. 00:22:30.720 --> 00:22:38.940 And if you stay on that path, then it makes more sense, I think, from a Python perspective, and you do have a tremendous amount of power. 00:22:38.940 --> 00:22:47.360 And I would say the other thing that I think turns people off with Matplotlib in the past is the visualizations are relatively unstyled. 00:22:47.360 --> 00:22:54.420 I mean, they're kind of plain, whereas some of these newer libraries just out of the box make something that look really nice. 00:22:54.420 --> 00:22:58.540 Matplotlib allows you to customize it, but that's extra work. 00:22:58.540 --> 00:23:04.780 One of the things that Matplotlib has done is they have a new theming or a relatively new theming API. 00:23:04.780 --> 00:23:12.400 And so if you use that, then you do get visualizations that look a little nicer out of the box and are more visually appealing. 00:23:12.400 --> 00:23:17.000 Yeah, it does have that kind of, it looks fine, but it kind of just looks a little bland, right? 00:23:17.000 --> 00:23:19.300 And it doesn't have that D3JS feel. 00:23:19.300 --> 00:23:19.760 Absolutely. 00:23:19.760 --> 00:23:23.280 So I got to give them some pretty mad props on the XKCD. 00:23:23.280 --> 00:23:24.780 Exactly. 00:23:24.780 --> 00:23:28.500 Have you, you've seen this, it sounds like. 00:23:28.500 --> 00:23:28.800 Yes. 00:23:28.800 --> 00:23:29.840 Yes, I have. 00:23:30.000 --> 00:23:34.880 So if people, I'm sure most people out there listening know the XKCD comic, right? 00:23:34.880 --> 00:23:41.400 If you don't go to your terminal or command prompt and run Python 3 and then just type import anti-gravity, then you'll know. 00:23:41.400 --> 00:23:47.160 But they have this, it's been around forever and it has this sort of style of like handwritten, but not handwritten. 00:23:47.160 --> 00:23:52.580 And one of the themes you can get is you can get the XKCD theme. 00:23:52.580 --> 00:23:53.340 Exactly. 00:23:53.340 --> 00:23:54.000 Yes. 00:23:54.000 --> 00:23:55.280 And it's really cool. 00:23:55.280 --> 00:24:02.260 And every once in a while, you'll find an article that someone put together where they show this beautiful visualization. 00:24:02.260 --> 00:24:13.000 And then you'd be surprised that it's matplotlib and they show all the steps and you can configure it and you can make something as nice as any of the JavaScript frameworks that are out there. 00:24:13.000 --> 00:24:14.640 But it does take some time. 00:24:14.640 --> 00:24:16.380 There are more lines of code to get there. 00:24:16.640 --> 00:24:17.160 Yeah, that's true. 00:24:17.160 --> 00:24:20.280 On this XKCD thing, it might sound like it's, well, that's funny. 00:24:20.280 --> 00:24:21.620 Like everybody loves XKCD. 00:24:21.620 --> 00:24:34.220 I do think there is some value to presenting results, whether that be a user interface or a visualization of analysis where you want to give it this preliminary look, this unfinished look. 00:24:34.220 --> 00:24:34.860 Right. 00:24:34.860 --> 00:24:39.580 And so if you're going to come into a meeting and you want to say, this is what the early data says. 00:24:39.580 --> 00:24:42.400 This is what our first pass analysis says. 00:24:42.400 --> 00:24:43.860 You know, put it in the XKCD. 00:24:44.160 --> 00:24:47.640 They might set that tone versus if it's like perfect and beautiful. 00:24:47.640 --> 00:24:48.500 Like, well, you're done. 00:24:48.500 --> 00:24:49.440 Like, no, no, we're not done. 00:24:49.440 --> 00:24:49.820 We're not. 00:24:49.820 --> 00:24:51.000 We're really far from done. 00:24:51.000 --> 00:24:52.480 This is just the beginning. 00:24:52.480 --> 00:24:55.040 I just want to give you a hint of what we're finding out. 00:24:55.040 --> 00:24:55.300 Right. 00:24:55.360 --> 00:24:58.640 I think there's a way that you could actually use this that would be practical. 00:24:58.640 --> 00:24:59.200 I agree. 00:24:59.200 --> 00:25:00.420 I mean, it's a good point. 00:25:00.420 --> 00:25:07.500 And, you know, one of the things that I think when you go into a business setting, everybody's used to standard Excel plots. 00:25:07.500 --> 00:25:21.460 And when you bring in something else like this, like an XKCD plot or some other plot that people aren't used to, it does get them to focus and look at it a little bit different and can steer the discussion in a little bit different way. 00:25:21.460 --> 00:25:21.780 Yeah. 00:25:21.780 --> 00:25:38.200 So one of the little areas that I thought was just really nice and really simple would be things like if you go and plot something with matplotlib and you have a lot of ticks along the bottom, it's very common that the words start to overlap each other. 00:25:38.200 --> 00:25:39.680 And you're like, well, this isn't working. 00:25:40.680 --> 00:25:48.460 And just little simple things like putting an angle on the values and the X axis can make it so much nicer. 00:25:48.460 --> 00:25:49.120 Right. 00:25:49.120 --> 00:25:49.480 Yes. 00:25:49.480 --> 00:25:49.840 Yeah. 00:25:49.840 --> 00:25:50.340 So. 00:25:50.340 --> 00:26:06.420 And one of the things that I do have a soft spot in my heart from matplotlib on the official documentation, I think under the tutorials, they took one of the blog posts that I wrote on matplotlib and it has been incorporated into the official tutorial. 00:26:06.420 --> 00:26:06.920 Oh, that's cool. 00:26:06.920 --> 00:26:07.640 About how to get started. 00:26:07.640 --> 00:26:09.280 So I think that's kind of cool. 00:26:09.280 --> 00:26:09.940 You know, I. 00:26:09.940 --> 00:26:10.300 Yeah. 00:26:10.460 --> 00:26:10.960 Proud of that. 00:26:10.960 --> 00:26:11.480 Absolutely. 00:26:11.480 --> 00:26:22.720 Another one that I thought was nice to know about that's not at all obvious is formatters using like F string formatter type things for when the data gets put up there. 00:26:22.720 --> 00:26:32.300 So maybe instead of having, you know, two, zero, one, zero dot, you know, some huge long decimal, you could format that as a number with no decimal point. 00:26:32.300 --> 00:26:32.740 No sense. 00:26:32.740 --> 00:26:33.280 Right. 00:26:33.280 --> 00:26:35.600 I use that all the time for currency. 00:26:35.600 --> 00:26:39.280 So if you're showing like millions, you don't want to have all the zeros. 00:26:39.440 --> 00:26:43.440 Maybe you just format as a dollar sign to M or whatever. 00:26:43.440 --> 00:26:43.620 Yeah. 00:26:43.620 --> 00:26:46.640 And matplotlib makes that very easy to do. 00:26:46.640 --> 00:26:50.620 And dates here, as you're showing are always one of those challenges with it. 00:26:50.620 --> 00:26:57.940 With any time you're plotting something, trying to figure out the right level of dates that convey the information, but don't crowd out the visualization. 00:26:57.940 --> 00:26:59.040 Yeah, absolutely. 00:26:59.380 --> 00:27:00.800 So there's a lot to learn there. 00:27:00.800 --> 00:27:06.520 But I still think a lot of people will be doing matplotlib, but also a lot of people will be choosing the newer ones. 00:27:06.520 --> 00:27:06.740 Yeah. 00:27:06.820 --> 00:27:10.460 Now for each of these, you came up with some pros and cons. 00:27:10.820 --> 00:27:18.520 So for the pros category of matplotlib, you have robust options that can do almost anything and lots of documentation examples. 00:27:18.520 --> 00:27:18.960 Yeah. 00:27:18.960 --> 00:27:19.660 Yeah. 00:27:19.660 --> 00:27:22.740 I mean, that is anything you want to do. 00:27:22.740 --> 00:27:24.100 You can do it in matplotlib. 00:27:24.100 --> 00:27:24.540 Yeah. 00:27:24.540 --> 00:27:28.420 The challenge is going to be, and you can find documentation examples. 00:27:28.420 --> 00:27:36.580 What does get challenging is because the API has evolved and changed over time, looking at the official docs are great. 00:27:36.580 --> 00:27:42.680 But when you can't find it there and you go searching on the web, you will find a tutorial from seven years ago. 00:27:42.680 --> 00:27:50.200 And the way they're telling you to do it may work, but it's not the most efficient way or may not work well with the way the rest of your code is structured. 00:27:50.200 --> 00:27:50.640 Right. 00:27:50.640 --> 00:27:55.220 Maybe it's not taking advantage of the themes or maybe it's using the stateful API instead. 00:27:55.220 --> 00:27:55.780 Exactly. 00:27:55.780 --> 00:27:56.940 There's a lot of reasons it could be. 00:27:56.940 --> 00:27:57.340 Exactly. 00:27:57.340 --> 00:28:07.340 The final, you know, challenge with matplotlib is there is some degree of interactivity, but it's not at the high level that some of the other libraries have. 00:28:07.340 --> 00:28:16.720 So if you have a scatter plot and you want to zoom in on an individual plot, look at the data, it's not as easy to do in matplotlib as it is in some of the other libraries. 00:28:17.140 --> 00:28:24.160 The other pro that I'd say with matplotlib is if you are trying to get your visualization in another format. 00:28:24.160 --> 00:28:30.280 So you're trying to put it in a document, you're trying to put it in a PDF, SVG, you know, any kind of graphic format. 00:28:30.280 --> 00:28:33.020 Matplotlib supports that out of the box. 00:28:33.020 --> 00:28:39.200 It's very easy to save it in any format and move it into whatever other document you have. 00:28:39.200 --> 00:28:46.360 Whereas some of the other ones are a little more challenging to do that or maybe don't have as high quality output as matplotlib does. 00:28:46.480 --> 00:28:50.780 Especially if it's an SVG, you can scale that almost infinitely, right? 00:28:50.780 --> 00:28:51.880 Yes, exactly. 00:28:51.880 --> 00:28:52.280 Yeah. 00:28:52.280 --> 00:29:08.700 Another thing that is going to be a repeated theme throughout many of these frameworks, but it's also here, is the ability, you know, not to just make one chart or one picture or one plot of some variation, but to build multiple plots. 00:29:08.700 --> 00:29:09.180 Right. 00:29:09.180 --> 00:29:15.360 So for example, a matplotlib, you can put, there's a, on the examples here, they have a MRI with EEG. 00:29:15.360 --> 00:29:25.560 And so they've got the brain, a picture of the brain and then a MRI, and then also some other measurement, which I don't know enough brain science to know what it's for, but. 00:29:25.560 --> 00:29:26.260 Neither do I. 00:29:26.260 --> 00:29:26.840 It looks cool. 00:29:26.840 --> 00:29:27.960 It looks cool. 00:29:27.960 --> 00:29:28.140 Right. 00:29:28.160 --> 00:29:33.140 But you can, you can create a single picture that puts like two graphs and then a graph below it. 00:29:33.140 --> 00:29:33.300 Right. 00:29:33.300 --> 00:29:36.620 There's a way to compose these beyond just making one picture. 00:29:36.620 --> 00:29:37.220 Exactly. 00:29:37.220 --> 00:29:41.960 And if you think, well, I could do this in Excel, I could put two or three graphs next to each other. 00:29:42.100 --> 00:29:44.680 But here's where the power of matplotlib comes into play. 00:29:44.680 --> 00:29:49.520 Let's say you were running experience experiments in a lab and you're doing hundreds of these. 00:29:49.520 --> 00:29:52.880 You wouldn't go into Excel and individually position all this. 00:29:52.880 --> 00:29:54.380 No, you write your Python script. 00:29:54.380 --> 00:29:59.580 You develop that layout for those different visualizations, and then you're done. 00:29:59.580 --> 00:30:02.840 You know, once you get the data, you just kind of run through it and you can iterate through it. 00:30:02.900 --> 00:30:12.060 And that's really the power using a real programming language, do visualization versus just trying to do one-off visualizations in Excel or some of the other tools out there. 00:30:12.060 --> 00:30:12.320 Sure. 00:30:12.320 --> 00:30:17.200 And instead of necessarily putting those pictures into a notebook output, you could write a loop. 00:30:17.200 --> 00:30:22.840 It says, go get me this experiment's data and save, generate this graph and save it to a file. 00:30:22.840 --> 00:30:23.760 Get the next one. 00:30:23.760 --> 00:30:30.960 You know, find out even what's in the folder, pull them all in, loop over them, generate one picture, then the next, then the next, then the next with the right name. 00:30:30.960 --> 00:30:36.400 And so it could just be all automatic, either in a script or just in a notebook that doesn't have as much output. 00:30:36.400 --> 00:30:36.640 Right. 00:30:36.640 --> 00:30:37.260 Exactly. 00:30:37.260 --> 00:30:37.800 Yep. 00:30:37.800 --> 00:30:39.960 The full power of Python is at your fingertips. 00:30:39.960 --> 00:30:40.720 Yeah, absolutely. 00:30:40.720 --> 00:30:42.180 All right. 00:30:42.180 --> 00:30:46.200 So that's the kernel of one half of this branch. 00:30:46.200 --> 00:30:47.620 Maybe the next one. 00:30:47.620 --> 00:30:51.240 This one's a little bit surprising to me because I just don't do enough pandas. 00:30:51.240 --> 00:30:59.800 But pandas, when I think about pandas, it's about manipulating data and reading data and transforming data and doing that tidy data preparation that you spoke about. 00:30:59.880 --> 00:31:00.880 All those different things. 00:31:00.880 --> 00:31:04.280 But there's also graphing built into pandas itself. 00:31:04.280 --> 00:31:04.800 Yes. 00:31:04.800 --> 00:31:07.820 And that graphing is built on top of Matplotlib. 00:31:07.820 --> 00:31:23.340 So that's why from a course perspective and thinking about visualization, I think having that Matplotlib foundation sets you up so that when you're in pandas, where, like you said, you'll be doing the majority of your data input manipulation and analysis. 00:31:23.540 --> 00:31:28.300 It's important to understand what visualization tools you have there. 00:31:28.360 --> 00:31:38.560 And a lot of the standard visualizations that you want to do with a line chart, scatter plots, bar charts, box plots, histograms, you can do with pandas. 00:31:38.680 --> 00:31:43.440 And it's mostly a very thin wrapper around Matplotlib. 00:31:43.440 --> 00:31:49.620 So that's why it helps if you get this output and you look at it and say, well, I just want to customize it a little bit. 00:31:49.620 --> 00:31:59.260 Typically, if you know that Matplotlib API, then you could customize it in pandas or go into Matplotlib and customize the pandas output. 00:31:59.620 --> 00:32:02.340 So I think that is really useful. 00:32:02.340 --> 00:32:09.220 The other thing that's interesting about pandas is it will allow you to plug in other backends. 00:32:09.220 --> 00:32:16.220 So Matplotlib is the default, but you can enable backends for Plotly and a few of the other visualizations. 00:32:16.220 --> 00:32:20.700 So it can be kind of this universal interface to visualization. 00:32:20.940 --> 00:32:33.840 In my experience, I don't use the pandas visualization a whole lot, but I do think it's important for people to understand it's out there because there are times where that's the quickest way to get something out there. 00:32:33.840 --> 00:32:37.860 And it's sufficient for the quick and dirty needs. 00:32:37.860 --> 00:32:43.860 Well, yeah, sometimes you open up a panda, you read something like a CSV or whatever, and you just want to know, well, what is this? 00:32:43.860 --> 00:32:47.220 And you would type DF for data frame, DF dot head or tail. 00:32:47.220 --> 00:32:50.500 And it gives you just a little brief view into it. 00:32:50.700 --> 00:32:53.360 There's the picture's worth a thousand words sort of thing. 00:32:53.360 --> 00:32:58.520 And if all you have to say is just dot plot and that's it or dot hist. 00:32:58.520 --> 00:33:02.600 And now you have a picture instead of that, that tail or head equivalent. 00:33:02.600 --> 00:33:06.620 That's a really cool way to just sort of quickly explore the data. 00:33:06.620 --> 00:33:07.180 Exactly. 00:33:07.180 --> 00:33:12.020 And there are some other more unique plots in pandas. 00:33:12.020 --> 00:33:20.580 So there are some plots called the Andrews curves and parallel coordinates that are more advanced. 00:33:20.580 --> 00:33:26.900 And to be honest, you know, it's more of a data science kind of machine learning plot. 00:33:26.900 --> 00:33:35.020 It's not something you use that often, but it is important to understand if you need it, that it is out there in pandas for you. 00:33:35.020 --> 00:33:35.240 Yeah. 00:33:35.240 --> 00:33:40.520 The Andrews curves reminds me a lot of like a Lorentz generator, a tractor. 00:33:40.520 --> 00:33:42.940 Something from chaos theory. 00:33:42.940 --> 00:33:45.940 A bunch of lines like looping over and over and over and over. 00:33:46.060 --> 00:33:46.400 Exactly. 00:33:46.400 --> 00:33:47.360 It looks pretty cool. 00:33:47.360 --> 00:33:48.000 It does. 00:33:48.000 --> 00:33:50.100 You'd probably blow some people's minds if you put that up. 00:33:50.100 --> 00:33:51.560 But, you know, it's specialized. 00:33:51.560 --> 00:33:59.340 But it does speak to there when you start getting into this visualization world, there are specialized libraries. 00:33:59.340 --> 00:34:05.920 And you might find something in your niche that you're working that, you know, this is really useful and it's really powerful. 00:34:05.920 --> 00:34:07.420 And it's maybe hard to do in other tools. 00:34:07.420 --> 00:34:10.840 And boom, it's easy in pandas or some of the other tools. 00:34:10.840 --> 00:34:11.220 Yeah. 00:34:11.360 --> 00:34:27.120 So let me tell, I normally try to avoid saying code on here because it's audio, but given a data frame, I can say plot.figure and then just call Andrews curves, give it the data frame and a name and boom, you get this amazing visualization of your data. 00:34:27.120 --> 00:34:29.280 Like on three incredibly simple lines. 00:34:29.280 --> 00:34:29.760 Yes. 00:34:29.760 --> 00:34:34.480 And this is the kind of stuff that I was referring to at the beginning when I said like, you almost don't need to know Python. 00:34:34.600 --> 00:34:37.860 I mean, technically, there's a little bit of Python, but barely, right? 00:34:37.860 --> 00:34:38.240 Right. 00:34:38.240 --> 00:34:55.940 And it's just starting to understand how to think about how you could use these tools and getting familiar with it so that the first time you're trying to build something is not you're not also trying to learn in the API and all these other visualization topics on top of trying to solve whatever problem you're trying to solve with visualization. 00:34:55.940 --> 00:35:02.480 Well, yeah, there's definitely some neat stuff to do visually with pandas and people should certainly be using it. 00:35:02.780 --> 00:35:07.460 Also, based on the Matplotlib kernel, we have Seaborn. 00:35:07.460 --> 00:35:08.620 Where does this fit into the world? 00:35:08.620 --> 00:35:27.720 So Seaborn builds on top of Matplotlib, like you said, but it really focuses more on doing statistical analysis of your data and taking that data and frequently transforming it in some ways and developing a visualization. 00:35:28.200 --> 00:35:32.880 So I use it a lot for histograms and box plots. 00:35:32.880 --> 00:35:37.560 So I use it a lot for the time to see it a lot of people who have seen the data and see it a lot of people who have seen the data and see it a lot of people who have seen it in some ways. 00:35:37.560 --> 00:35:43.240 So that's where you take a lot of people who have seen it in some ways. 00:35:43.240 --> 00:35:47.920 And that's where you take a lot of people who have seen it in some ways. 00:35:47.920 --> 00:35:52.600 So you have seen it in some ways. 00:35:52.600 --> 00:35:55.820 So you've got nine, 12, you know, however many plots you need. 00:35:55.820 --> 00:36:06.980 And it gives you an opportunity to spot data anomalies, trends very easily and present a wealth of information in a very compact frame. 00:36:06.980 --> 00:36:11.140 And what I like about Seaborn is it is very easy to do this. 00:36:11.140 --> 00:36:16.340 So the code that's required to do this is typically, you know, one or two lines of code. 00:36:16.340 --> 00:36:20.360 And you get this really nice plot that has different colors. 00:36:20.360 --> 00:36:23.800 It has data varying across the rows and columns. 00:36:23.800 --> 00:36:36.160 You can change the shape, the size, everything to, you know, with those visualization concepts that we cover in the beginning to develop visualization that really give you a lot of insight very quickly. 00:36:36.160 --> 00:36:37.020 Yeah, absolutely. 00:36:37.020 --> 00:36:39.460 It's very statistical focused, isn't it? 00:36:39.620 --> 00:36:40.040 It is. 00:36:40.040 --> 00:36:40.780 It is. 00:36:40.780 --> 00:36:43.100 You know, it is very statistical focused. 00:36:43.100 --> 00:36:49.420 And some of the plots are more complex and having a statistical background will help you understand them. 00:36:49.420 --> 00:37:02.620 But simple things like just doing bar plots or histograms or account plot or a heat map, you know, are relatively straightforward to explain to others and easy to create with Seaborn. 00:37:02.620 --> 00:37:03.660 And then you're right. 00:37:03.660 --> 00:37:08.720 There are some of the plots that are definitely much more kind of deep in the statistical toolbox. 00:37:08.720 --> 00:37:12.120 And you have to know how you would explain them to others. 00:37:12.120 --> 00:37:15.060 So this faceting thing is pretty interesting. 00:37:15.060 --> 00:37:24.200 If you've got, if you go to the Seaborn examples, they've got one for looks like five different variables or sorry, four variables. 00:37:24.200 --> 00:37:26.820 And they're looking at two different groups on one of them. 00:37:26.820 --> 00:37:38.040 And you can just say, show me, you know, basically instead of give me a picture of this data, take any two pieces of information and generate a graph to show me how those things correlate. 00:37:38.320 --> 00:37:45.200 One of the pieces of data that you did a lot of work with was the automobile gasoline efficiency data from the EPA, right? 00:37:45.660 --> 00:37:46.220 Yes. 00:37:46.220 --> 00:38:01.800 What I find really fascinating about this is when you get the data set up properly and you want to look at all these different relationships, it's very easy to slice and dice and mix up those relationships to see where the trends are. 00:38:02.000 --> 00:38:08.640 So if we're looking at fuel efficiency, we could look at the year the cars were manufactured. 00:38:08.640 --> 00:38:12.700 We could look at it by are they front wheel drive, all wheel drive cars? 00:38:12.700 --> 00:38:14.640 How many cylinders do they have? 00:38:14.640 --> 00:38:15.240 Are they electric? 00:38:15.240 --> 00:38:16.940 SUV versus SUV? 00:38:16.940 --> 00:38:17.920 All these things. 00:38:17.920 --> 00:38:18.140 Yeah. 00:38:18.220 --> 00:38:18.440 Yeah. 00:38:18.440 --> 00:38:20.860 You know, there could be a price component. 00:38:20.860 --> 00:38:24.680 There could be just a whole bunch of different ways to look at it. 00:38:24.680 --> 00:38:31.080 And when you have a big data set, those are the types of things that are very difficult to do just by looking at the numbers. 00:38:31.080 --> 00:38:33.500 And that's where visualization really shines. 00:38:33.500 --> 00:38:40.940 And where Seaborn makes it tremendously easy to just quickly iterate through and say, okay, I want to look at these two variables together. 00:38:41.160 --> 00:38:42.600 Now let's layer in a third variable. 00:38:42.600 --> 00:38:44.060 Now there's a fourth variable. 00:38:44.060 --> 00:38:45.080 Well, I don't like this. 00:38:45.080 --> 00:38:46.820 Let's switch them around a different way. 00:38:46.820 --> 00:39:01.640 And I use Seaborn a lot because of that flexibility of just exploring the data, quickly figuring out what those trends are, what those insights are, and rapidly iterating through it for the exploratory analysis. 00:39:01.640 --> 00:39:07.100 Another thing that Seaborn has is it looks really nice and it has this idea of themes. 00:39:07.100 --> 00:39:07.700 Yes. 00:39:07.700 --> 00:39:09.720 So it's easy to make it look good, right? 00:39:09.720 --> 00:39:10.220 It is. 00:39:10.340 --> 00:39:26.800 I mean, Seaborn out of the box applies some themes and also does things behind the scenes with the visualization to make it cleaner and to try and format the data so that, you know, dates line up appropriately and colors look nice. 00:39:26.800 --> 00:39:30.540 And there's appropriate spacing and things like that. 00:39:30.540 --> 00:39:38.700 And there are some other things that are pretty easy to control to turn on and off or change the color palettes with Seaborn. 00:39:38.860 --> 00:39:43.260 But generally out of the box, it strives to look good and it looks nice. 00:39:43.260 --> 00:39:47.760 And what's also beneficial about it is it is just matplotlib under the hood. 00:39:47.760 --> 00:39:57.640 And so if you get to the point where you've done your analysis and things look pretty good, but you want to do some tweaks, there are some convenience functions in Seaborn to do that. 00:39:57.900 --> 00:40:00.940 But there's also, it's just matplotlib under the hood. 00:40:00.940 --> 00:40:04.820 So if you know matplotlib and you want to tweak some things, you can do that as well. 00:40:04.820 --> 00:40:05.540 Yeah, for sure. 00:40:05.540 --> 00:40:14.640 One thing I want to maybe take a step back on here with matplotlib on a lot of these examples, not this one I got up here, but lots of them. 00:40:14.760 --> 00:40:20.380 I'll see, and you talked about this, that we'll have like semicolons some of the time. 00:40:20.380 --> 00:40:25.480 And, you know, Python's famous for not requiring semicolons at the end of lines. 00:40:25.480 --> 00:40:26.840 What's the story there? 00:40:27.100 --> 00:40:32.880 Yeah, the semicolons are just an artifact of when you're developing in a Jupyter notebook. 00:40:32.880 --> 00:40:39.800 And when you show that plot, that matplotlib will show additional information about the plot. 00:40:39.800 --> 00:40:44.260 So it will have kind of like a string descriptor, and then it will show the plot. 00:40:44.260 --> 00:40:48.320 And if you use the semicolon, it will suppress that extra information. 00:40:48.320 --> 00:40:49.860 So all you see is the plot. 00:40:49.860 --> 00:40:53.800 So it's certainly, it's not required by any means. 00:40:53.800 --> 00:40:55.400 And you're right, it does. 00:40:55.640 --> 00:40:59.520 For people that have played with Python for a while, you kind of wonder why there's semicolon there. 00:40:59.520 --> 00:41:04.580 But it's just to suppress some of that extra information that gets shown. 00:41:04.580 --> 00:41:05.780 And it's not on all the lines. 00:41:05.780 --> 00:41:07.540 It's just on certain plotting lines. 00:41:07.540 --> 00:41:08.880 The other lines don't need it, right? 00:41:08.880 --> 00:41:11.640 So it's a little unclear if you're not sure, which is why. 00:41:11.640 --> 00:41:12.580 All right. 00:41:12.580 --> 00:41:22.280 Moving on from Seaborn, we start to bridge our way over into the JavaScript D3JS side of things with Altair. 00:41:22.280 --> 00:41:25.340 And Altair, I think, is certainly very well known. 00:41:25.500 --> 00:41:26.300 Very well respected. 00:41:26.300 --> 00:41:27.540 It's one of the newer ones, isn't it? 00:41:27.540 --> 00:41:28.180 It is. 00:41:28.180 --> 00:41:28.720 Yes. 00:41:28.720 --> 00:41:29.560 I have to look. 00:41:29.560 --> 00:41:30.980 I don't remember off the top of my head. 00:41:30.980 --> 00:41:39.680 But if Matplotlib was started in like 2012, Altair is probably in the last five years or so. 00:41:39.680 --> 00:41:40.020 Yeah. 00:41:40.020 --> 00:41:40.600 Yeah. 00:41:40.600 --> 00:41:41.400 I would imagine. 00:41:41.660 --> 00:41:47.880 So definitely much newer, but has been tremendous amount of updates. 00:41:47.880 --> 00:41:48.640 What was it? 00:41:48.640 --> 00:41:49.300 2015. 00:41:49.300 --> 00:41:50.060 Okay. 00:41:50.060 --> 00:41:50.420 Yeah. 00:41:50.500 --> 00:42:01.940 And Jake, who started it and maintains it, did an awesome job of leveraging a lot of the best practices from Python libraries as well as R to build Altair. 00:42:02.180 --> 00:42:04.460 And like we said, it's built on top of Vega. 00:42:04.460 --> 00:42:15.600 So there's that, you know, anytime that new work is done in that JavaScript library, it's easier to port it to Python so that you can leverage that as well. 00:42:15.980 --> 00:42:18.220 Quite popular, almost 8,000 GitHub stars. 00:42:18.220 --> 00:42:25.900 And Jake Vander Plaas added or changed 353,000 lines and removed 240,000 lines. 00:42:25.900 --> 00:42:29.620 And Allison BG as well, something on a similar scale. 00:42:29.620 --> 00:42:30.780 That's a ton of work. 00:42:30.780 --> 00:42:31.220 Yeah. 00:42:31.220 --> 00:42:31.700 Yeah. 00:42:31.700 --> 00:42:33.100 It's a fabulous library. 00:42:33.100 --> 00:42:37.360 I mean, there's one of the things I really like about the library is the gallery. 00:42:37.360 --> 00:42:38.520 Yeah, that's where we're going now. 00:42:38.520 --> 00:42:42.380 And the documentation is really great for Altair. 00:42:42.380 --> 00:42:48.680 And once you start to get into it, you need to spend a little bit of time to just make sure you understand how the library works. 00:42:48.680 --> 00:42:53.100 But then probably 90% of the time, you're going to go to the gallery and try and find something. 00:42:53.100 --> 00:42:55.460 And you look at the code and you're like, oh, okay, that's how I do it. 00:42:55.460 --> 00:42:55.640 Yeah. 00:42:55.640 --> 00:42:56.600 Like, this is the one I want. 00:42:56.600 --> 00:42:58.520 Yes, exactly. 00:42:58.800 --> 00:43:01.460 There's a lot of interesting aspects about creating graphs here. 00:43:01.460 --> 00:43:07.640 So one of the common ones is to create a scatter plot, which is called mark circle in this world, right? 00:43:07.640 --> 00:43:11.620 And hopefully those are similar enough to be put together. 00:43:11.620 --> 00:43:12.260 It's the same thing. 00:43:12.260 --> 00:43:17.200 And those create little dots that show like, where's all the data from these different categories, say. 00:43:17.200 --> 00:43:21.560 And one thing that's interesting is you can say, I'd like to color. 00:43:21.560 --> 00:43:25.040 I want the X to be this value and the Y to be that value. 00:43:25.040 --> 00:43:32.620 But then I want the color of the dot to be based on another column and maybe the size to be on a third one. 00:43:32.620 --> 00:43:41.640 So in this EPA car data, you could say, well, I want the color of the dot to be the type of vehicle, like an SUV or a car or whatever. 00:43:41.640 --> 00:43:44.960 And then I want the size to be the number of cylinders in the engine. 00:43:45.120 --> 00:43:47.120 And that's just incredibly easy. 00:43:47.120 --> 00:43:49.120 But it really draws out the data. 00:43:49.120 --> 00:43:49.940 It does. 00:43:49.940 --> 00:43:54.560 And Altair makes it easy to combine this in different ways. 00:43:54.560 --> 00:44:01.600 So if you want to have a scatter plot and bar chart or a histogram, you can combine these together. 00:44:01.840 --> 00:44:04.840 You can also do the faceting that we talked about with Seaborn. 00:44:04.840 --> 00:44:11.180 You can do with Altair, where you change the variable across the columns and rows to get different plots. 00:44:11.180 --> 00:44:18.720 And the other thing that Altair, one of the other things Altair introduces is interactivity out of the box. 00:44:19.500 --> 00:44:25.000 So you, because it is JavaScript based, you then have that ability to go in. 00:44:25.000 --> 00:44:34.480 And as you're doing here, for the people who can see it, you can hover over a spot and then control what information is shown for that hover. 00:44:34.480 --> 00:44:37.960 So you can see, oh, what's going on with this individual dot? 00:44:38.080 --> 00:44:45.960 Well, here's the, it's a Volkswagen Rabbit from Europe and it has a 71 horsepower engine and gets 31.9 miles per gallon. 00:44:45.960 --> 00:44:55.620 And so it's really cool and really useful for doing that exploratory analysis where you kind of want to see the individual data points and maybe drill into it a little bit more. 00:44:55.620 --> 00:44:57.260 Yeah, the interactivity is great. 00:44:57.260 --> 00:45:04.180 The ability to add custom tooltips and then have those tooltips, have like F-string style formatting as well. 00:45:04.180 --> 00:45:05.580 It's pretty excellent. 00:45:05.800 --> 00:45:09.840 So if the data doesn't show up just the way you would like, you know, you can have something. 00:45:09.840 --> 00:45:13.480 You can say small, medium, large versus like this number or that number. 00:45:13.480 --> 00:45:16.400 So you can kind of think about it separately and differently, right? 00:45:16.400 --> 00:45:16.920 Exactly. 00:45:16.920 --> 00:45:26.080 And one of the other things that's interesting about Altair is it does try to infer different aspects about your data. 00:45:26.080 --> 00:45:30.240 So it tries to understand, well, is this data continuous data? 00:45:30.240 --> 00:45:31.440 Is it date data? 00:45:31.440 --> 00:45:35.240 And you can specify the different types of data. 00:45:35.640 --> 00:45:41.020 So you can say that it's quantitative or it's an ordinal value or nominal value. 00:45:41.020 --> 00:45:46.520 And the actual visualization will change a little bit depending on that data type. 00:45:46.520 --> 00:45:48.640 That's a unique thing that Altair does. 00:45:48.640 --> 00:45:57.860 And I think that's one of those things that as you start going down this visualization path, you start to think about your data a little bit differently and think about how you want to present it. 00:45:58.260 --> 00:46:03.640 And Altair gives you that window into all the flexibility you have with presenting data. 00:46:03.640 --> 00:46:08.240 One of the really nice aspects of interactivity of these is the legend. 00:46:08.240 --> 00:46:11.700 So the legend will show, you know, it really looks nice. 00:46:11.700 --> 00:46:15.660 It matches like the color and the name and it's in a font that is pretty readable. 00:46:15.920 --> 00:46:23.760 But if you set it up right, you can go and actually click on these and either just highlight one or you can have them sort of be toggle buttons. 00:46:23.760 --> 00:46:31.100 And so if you want to just focus on, you know, let me pull out just in this case, it's got like agricultural finance, government type of spending or something. 00:46:31.100 --> 00:46:37.400 You can just say, I want to just see the educational and health and click that and it highlights that sort of separate from the rest of them. 00:46:37.400 --> 00:46:37.800 Yes. 00:46:38.120 --> 00:46:39.180 And it's really easy. 00:46:39.180 --> 00:46:42.900 There's like one line of code that you need to do to set that all up. 00:46:42.900 --> 00:46:45.160 Add selection or something simple like that, right? 00:46:45.160 --> 00:46:45.760 Exactly. 00:46:45.760 --> 00:46:46.260 Yep. 00:46:46.260 --> 00:46:54.080 One other thing I want to touch on with Altair here, and I also want to talk about an example, but you talked about, was it data? 00:46:54.080 --> 00:46:56.460 Was it transforms or something? 00:46:56.460 --> 00:47:01.880 You've got to either use the file or a little server or something to process the data. 00:47:01.880 --> 00:47:03.280 What's the story of that? 00:47:03.280 --> 00:47:03.560 Yeah. 00:47:03.560 --> 00:47:13.560 So one of the things that can be a little tricky with Altair is behind the scenes, it's translating whatever data you have into a JSON file. 00:47:13.560 --> 00:47:18.960 And so what that means is when you have 10 or 20 data elements, it's not that big a deal. 00:47:18.960 --> 00:47:24.620 But when you have thousands of elements, you can suddenly get to a point where that file is really huge. 00:47:24.620 --> 00:47:32.020 And so rightly so, Altair makes sure that you don't inadvertently embed that in your notebook. 00:47:32.300 --> 00:47:40.540 So you could end up with your Jupyter notebook suddenly being, you know, 50 megs because you've got all these Altair visualizations in there. 00:47:40.540 --> 00:47:49.460 So there are some options for how you can manage that data so that it's not necessarily stored directly in the notebook file. 00:47:49.460 --> 00:48:03.780 Maybe you have that data stored separately, kind of like on a cache file in a directory or potentially more of like a real-time streaming option where you have a backend service that's running behind the scenes that streams up the data to you. 00:48:03.880 --> 00:48:08.740 So there is a little bit more complexity sometimes with Altair to get it running. 00:48:08.740 --> 00:48:14.360 And because of that visualization, because of the JSON approach that it uses. 00:48:14.600 --> 00:48:19.120 So that's certainly one of the watchouts and things to keep in mind. 00:48:19.120 --> 00:48:24.380 The other thing that sometimes has been a challenge for me with Altair is saving visualizations. 00:48:24.380 --> 00:48:30.840 So if you want to create something as an SVG in Matplotlib or Seaborn, it's very straightforward. 00:48:30.840 --> 00:48:32.240 You just save it. 00:48:32.240 --> 00:48:41.180 With Altair, sometimes it can be a little challenging because of the way it's trying to render those visualizations and save them to a PNG or SVG file. 00:48:41.180 --> 00:48:44.200 Yeah, so you've got to set that up and select the right one. 00:48:44.200 --> 00:48:46.940 It'll work if you don't have too much data without doing that, right? 00:48:46.940 --> 00:48:50.480 But then there's some limit where it's like, you know, this is too much. 00:48:50.480 --> 00:48:52.660 You've got to have to push it outside of the notebook. 00:48:52.660 --> 00:48:53.160 Yes. 00:48:53.160 --> 00:48:56.860 You'll get an error message and it'll kind of tell you what's going on. 00:48:56.860 --> 00:49:01.420 But I do mention it in the course, you know, some of the options for getting around it. 00:49:01.420 --> 00:49:04.280 And the documentation is good about what those options are as well. 00:49:04.280 --> 00:49:04.560 Sure. 00:49:04.560 --> 00:49:05.400 All right. 00:49:05.400 --> 00:49:09.560 So to wrap up Altair, I want to talk through a little example here. 00:49:09.560 --> 00:49:14.060 And I'll put this example in the show notes so people can check it out. 00:49:14.060 --> 00:49:16.520 Let's try to describe this picture here. 00:49:16.520 --> 00:49:20.280 And like I said, I'll put it in the show notes so people can see it. 00:49:20.280 --> 00:49:25.060 You have this Amazon author reviews for the top 20 most reviewed books, 00:49:25.060 --> 00:49:29.240 most reviewed authors over the last 10 years or something like that, right? 00:49:29.240 --> 00:49:29.600 Yes. 00:49:29.600 --> 00:49:30.960 Tell us what's going on in this picture. 00:49:30.960 --> 00:49:34.160 And we can maybe talk through just the API components that make it happen. 00:49:34.440 --> 00:49:34.700 Sure. 00:49:34.700 --> 00:49:39.340 So behind the scenes, the data is, I can't even remember, well, I guess this one's through 00:49:39.340 --> 00:49:44.720 2016 or so, 10 years or so of Amazon reviews. 00:49:44.720 --> 00:49:50.400 So on the X axis, it's 2009 through 2020. 00:49:50.400 --> 00:49:52.640 And then on the Y axis, we have authors. 00:49:52.860 --> 00:49:55.800 So a lot of famous authors in this timeframe. 00:49:55.800 --> 00:49:59.300 And some of these authors will have one book a year. 00:49:59.300 --> 00:50:00.760 They'll have multiple books a year. 00:50:00.760 --> 00:50:06.960 And so it will have a circle where for each author for the year they published the book 00:50:06.960 --> 00:50:11.140 and the size of the circle represents how many reviews they had. 00:50:11.320 --> 00:50:17.540 So this is a really quick way to see how consistent some of these top authors are over years. 00:50:17.540 --> 00:50:21.560 You know, it's sort of interesting, like Dale Carnegie is at the top. 00:50:21.560 --> 00:50:27.140 I mean, he wrote this book, I don't know, probably 50 plus years ago, but it's still a bestseller 00:50:27.140 --> 00:50:28.020 on Amazon. 00:50:28.020 --> 00:50:35.100 And then you have other people that are maybe a little more sporadic, but it's a very easy 00:50:35.100 --> 00:50:40.940 way to see the consistency and then the number of reviews that each author gets for a year. 00:50:40.940 --> 00:50:45.860 And I would say, you know, if an author has more than one book, obviously they'll have more 00:50:45.860 --> 00:50:46.280 reviews. 00:50:46.280 --> 00:50:47.520 So it's not broken out by book. 00:50:47.520 --> 00:50:48.640 It's just purely by author. 00:50:48.640 --> 00:50:48.920 Right. 00:50:48.920 --> 00:50:52.560 And when you build up this picture, some of the things that happen is there's a legend that 00:50:52.560 --> 00:50:52.940 shows up. 00:50:52.940 --> 00:50:56.700 The legend is basically just a copy slightly offset of what's on the left. 00:50:56.700 --> 00:50:59.820 But what you really want to know is what's the size of the circle means. 00:50:59.820 --> 00:51:04.760 So you can add like an alternative legend and you can put a grid behind it and make it 00:51:04.760 --> 00:51:06.120 really easy to follow the timeline. 00:51:06.120 --> 00:51:08.840 There's just a really cool bunch of features. 00:51:08.840 --> 00:51:13.800 And this is the kind of picture I was thinking about when we were thinking, when I said, you 00:51:13.800 --> 00:51:19.560 know, like if you just go and call plot or circle or whatever, you know, mark circle, you 00:51:19.560 --> 00:51:21.160 end up with something that's not all that impressive. 00:51:21.160 --> 00:51:23.960 But if you layer a few of these ideas on, then it's great. 00:51:23.960 --> 00:51:29.640 We said we've got the author in the year and then the size, we configure the size based 00:51:29.640 --> 00:51:30.800 on the number of reviews. 00:51:30.800 --> 00:51:34.420 Then we configure the color based on the author so that it's a little easy to follow the 00:51:34.420 --> 00:51:37.900 little easier visually to look at this and see the information. 00:51:37.900 --> 00:51:45.560 And then we also do this thing that is interesting with Altair is when you think about Seaborn 00:51:45.560 --> 00:51:51.160 and Matplotlib and Pandas and some of the other libraries we'll talk about, you typically do 00:51:51.160 --> 00:51:53.440 the data manipulation in Pandas. 00:51:53.440 --> 00:51:58.760 Altair has its own ability to do manipulation and transformation of data. 00:51:59.120 --> 00:52:04.700 And so there is this option, you know, I could have filtered it down to the top 20 customers 00:52:04.700 --> 00:52:06.080 or authors, excuse me. 00:52:06.080 --> 00:52:12.180 But I use this transform filter to select only the top authors. 00:52:12.180 --> 00:52:16.700 And that's all in Altair, not using any Pandas. 00:52:16.700 --> 00:52:21.360 And then the final thing that we do is configure the width and the height and the title. 00:52:21.780 --> 00:52:28.120 So that's all, you know, kind of one long piece of code that looks intimidating as you, you know, 00:52:28.120 --> 00:52:29.740 maybe if you haven't worked with Altair. 00:52:29.740 --> 00:52:34.620 But when you take a step back and break it down and think about what it is you're trying 00:52:34.620 --> 00:52:35.600 to do with your visualization. 00:52:35.600 --> 00:52:41.960 And then with the basic understanding of the Altair API, it's pretty straightforward and extremely 00:52:41.960 --> 00:52:42.400 powerful. 00:52:42.400 --> 00:52:42.720 Sure. 00:52:42.720 --> 00:52:44.380 You kind of got to do it in steps. 00:52:44.380 --> 00:52:51.400 It's a very fluent API, you know, Alt.chart.markcircle.encode.configure. 00:52:51.400 --> 00:52:57.500 But if you take each one of those relatively simple function calls, then you try to understand 00:52:57.500 --> 00:52:58.280 that and see what you're doing. 00:52:58.280 --> 00:52:59.580 And then it turns out to be not too bad. 00:52:59.580 --> 00:53:00.420 Yeah, exactly. 00:53:00.420 --> 00:53:06.240 And what people need to realize when they're thinking about this course is anytime I develop 00:53:06.240 --> 00:53:08.820 code and it looks like this and it has this many lines, you're right. 00:53:08.820 --> 00:53:10.220 It wasn't, I didn't start off. 00:53:10.220 --> 00:53:12.660 I did first, let's just do author versus year. 00:53:12.660 --> 00:53:13.660 I don't like this. 00:53:13.660 --> 00:53:14.960 I need to tweak one more thing. 00:53:14.960 --> 00:53:16.440 Then I need to tweak one more thing. 00:53:16.440 --> 00:53:18.960 And you iterate over it to get there. 00:53:18.960 --> 00:53:24.100 And once you kind of understand how all these libraries work, it's not too much work, but 00:53:24.100 --> 00:53:25.580 it does take a little bit of time. 00:53:25.580 --> 00:53:30.600 And that's why it's important to, you know, dive into the data and play with it and experiment 00:53:30.600 --> 00:53:32.900 with it and see what works for you. 00:53:32.900 --> 00:53:34.900 Yeah, it almost is its own little mini language. 00:53:34.900 --> 00:53:36.180 It is. 00:53:36.180 --> 00:53:36.860 It is. 00:53:36.860 --> 00:53:37.840 All right. 00:53:37.840 --> 00:53:39.980 Before we move on to the next one, question from the audience. 00:53:40.100 --> 00:53:40.620 I'm all out there. 00:53:40.620 --> 00:53:44.160 Can we do responsive and animated workflow diagrams with matplotlib? 00:53:44.160 --> 00:53:49.140 For example, continuous builds development on different server or deployment on different 00:53:49.140 --> 00:53:49.500 servers? 00:53:49.500 --> 00:53:55.620 Not entirely sure exactly what you're asking, but certainly you can automate these things, 00:53:55.620 --> 00:53:55.780 right? 00:53:55.780 --> 00:53:56.920 It doesn't have to be in a notebook. 00:53:56.920 --> 00:53:59.080 This could all be put into a script, right? 00:53:59.260 --> 00:53:59.480 Yes. 00:53:59.480 --> 00:54:03.540 And there are matplotlib does support some animation. 00:54:03.540 --> 00:54:08.820 And I can't remember how much of this is out of the box matplotlib versus third party libraries. 00:54:09.040 --> 00:54:13.760 But I've certainly seen visualizations that people have done with matplotlib where it's, you know, 00:54:13.760 --> 00:54:16.960 something changing over time or steps in a process. 00:54:16.960 --> 00:54:19.440 You can do that with matplotlib. 00:54:19.440 --> 00:54:22.160 So you could create one of those language battles. 00:54:22.160 --> 00:54:27.040 Have you ever seen those where like over 20 or 30 years, it's either a browser or the language 00:54:27.040 --> 00:54:29.020 is when the book is popular, then it goes up. 00:54:29.020 --> 00:54:29.440 And yeah. 00:54:29.440 --> 00:54:29.700 Yes. 00:54:29.700 --> 00:54:30.160 Yeah. 00:54:30.640 --> 00:54:30.840 Yeah. 00:54:30.840 --> 00:54:31.920 You could do that. 00:54:31.920 --> 00:54:36.480 I'd have to look and see what would be the best approach, but those sorts of options are 00:54:36.480 --> 00:54:36.780 out there. 00:54:36.780 --> 00:54:37.040 Fun. 00:54:37.040 --> 00:54:37.920 All right. 00:54:37.920 --> 00:54:42.460 Sticking in the JavaScript side of things, the other really popular one over there is Plotly. 00:54:42.460 --> 00:54:44.620 What's special and unique about Plotly? 00:54:44.620 --> 00:54:52.520 I think Plotly is special and unique because it is a newer plotting library, kind of on the 00:54:52.520 --> 00:54:53.960 order of Altair. 00:54:54.320 --> 00:55:04.180 It is supported by a company out of Canada, but the Plotly visualization library is completely 00:55:04.180 --> 00:55:04.920 open source. 00:55:04.920 --> 00:55:06.720 It is based on JavaScript. 00:55:06.720 --> 00:55:12.080 What I like about it is everything is interactive out of the box. 00:55:12.080 --> 00:55:17.660 So any plot you make, as soon as it renders, you can take your mouse and you can hover over 00:55:17.660 --> 00:55:17.900 it. 00:55:17.900 --> 00:55:19.020 You can zoom in. 00:55:19.020 --> 00:55:21.640 You can limit the range of data. 00:55:22.340 --> 00:55:24.640 And so that is really powerful. 00:55:24.640 --> 00:55:29.460 And then the second thing that I really like about it is the history of Plotly. 00:55:29.460 --> 00:55:32.100 There was a separate Plotly visualization. 00:55:32.100 --> 00:55:36.920 And then there was something called Plotly Express, which was streamlined. 00:55:36.920 --> 00:55:42.440 And the Plotly Express API, in my opinion, is very similar to Seaborn. 00:55:42.440 --> 00:55:45.720 And it is very, I think it's Pythonic. 00:55:45.720 --> 00:55:47.860 It's very easy to understand and pick up. 00:55:47.860 --> 00:55:53.260 And over time, they've expanded it to where now that's kind of the default visualization. 00:55:53.260 --> 00:55:56.760 So it's very easy to get started with Plotly. 00:55:56.760 --> 00:56:00.540 It makes those interactive plots out of the gate. 00:56:00.540 --> 00:56:03.180 And then it does have some unique plots. 00:56:03.180 --> 00:56:07.600 So some of the kind of custom tree map plots, scatter matrix. 00:56:07.980 --> 00:56:12.880 You can do plotting on maps to show geographic plots. 00:56:12.880 --> 00:56:20.020 Those are all like out of the box, work pretty well and are fairly simple to create. 00:56:20.020 --> 00:56:20.220 Yeah. 00:56:20.220 --> 00:56:23.800 The tree map and the sunburst, those come from Plotly, right? 00:56:23.800 --> 00:56:24.400 Yes. 00:56:24.400 --> 00:56:27.100 At least are in Plotly is one of those things. 00:56:27.280 --> 00:56:33.260 And one of the other things that is interesting about Plotly, like any of these visualization 00:56:33.260 --> 00:56:39.700 tools, you have to be able to go in and configure and customize and tweak things. 00:56:39.700 --> 00:56:45.680 And Plotly gives you that ability to generate a plot, but then you can update it over time. 00:56:45.680 --> 00:56:47.240 So you can change colors. 00:56:47.240 --> 00:56:49.440 You can change the way data is presented. 00:56:49.440 --> 00:56:55.260 You can change pretty much anything with the plot using a fairly simple API as well. 00:56:55.260 --> 00:56:55.560 Yeah. 00:56:55.560 --> 00:56:59.100 People should definitely go and check out the tree maps example. 00:56:59.100 --> 00:57:04.280 One of the really cool interactions is, for example, the one that I'm looking at here has, 00:57:04.280 --> 00:57:06.780 here's the world and then here's the different continents. 00:57:06.780 --> 00:57:08.800 So like Asia, Africa, the Americas. 00:57:08.800 --> 00:57:14.420 Then if you want to, within each one of those, they've got a little box that says, well, here's 00:57:14.420 --> 00:57:20.660 how large of an impact, like for example, Nigeria and Egypt are, you know, have more people 00:57:20.660 --> 00:57:22.460 in it, I guess, than the other countries. 00:57:22.460 --> 00:57:25.360 And then they're colored by what their actual values are. 00:57:25.360 --> 00:57:28.820 But if you want to just focus on say Africa, you can just click on that section of this 00:57:28.820 --> 00:57:32.920 thing and it just zooms in to show you just that information. 00:57:32.920 --> 00:57:35.280 And this, you're not going to get this with Matt Plotlet. 00:57:35.280 --> 00:57:35.820 No. 00:57:35.820 --> 00:57:36.140 Right? 00:57:36.140 --> 00:57:36.420 No. 00:57:36.420 --> 00:57:37.420 No, you're not. 00:57:37.420 --> 00:57:38.220 And it's... 00:57:38.220 --> 00:57:39.720 The ability to just dive in and out of the data. 00:57:39.720 --> 00:57:40.680 Yeah, absolutely. 00:57:40.900 --> 00:57:43.180 And it is very simple. 00:57:43.180 --> 00:57:47.300 I use Plotly quite a bit, especially when I want to explore the data. 00:57:47.300 --> 00:57:53.760 So you want to have a scatterplot or this tree map or any of the other visualizations and you 00:57:53.760 --> 00:57:57.300 can easily filter or zoom in, zoom out. 00:57:57.300 --> 00:57:58.640 And yeah. 00:57:58.640 --> 00:58:02.940 So you're showing some of the other cool kind of unique visualizations that are out there 00:58:02.940 --> 00:58:06.400 to Plotly that maybe aren't as available in some of the other libraries. 00:58:06.580 --> 00:58:06.780 Yeah. 00:58:06.780 --> 00:58:07.240 Yeah. 00:58:07.240 --> 00:58:09.240 The Sunburst has a real similar... 00:58:09.240 --> 00:58:13.220 The Sunburst is like a pie chart, but as you interact with it, it like zooms into those 00:58:13.220 --> 00:58:15.640 sections in pretty amazing ways. 00:58:15.640 --> 00:58:16.000 Yeah. 00:58:16.000 --> 00:58:17.080 Well, that is pretty cool. 00:58:17.080 --> 00:58:18.260 I haven't actually seen that one. 00:58:18.260 --> 00:58:18.740 That's neat. 00:58:18.740 --> 00:58:20.420 It's just like... 00:58:20.420 --> 00:58:20.560 Yeah. 00:58:20.560 --> 00:58:21.840 It just draws... 00:58:21.840 --> 00:58:24.040 It just says, I want to explore this data, right? 00:58:24.040 --> 00:58:26.140 Not only can I, but like, I'm going to. 00:58:26.140 --> 00:58:27.380 Yes, exactly. 00:58:27.380 --> 00:58:27.860 Yeah. 00:58:27.860 --> 00:58:28.280 Cool. 00:58:28.280 --> 00:58:31.920 Does Plotly have this idea of like a backend server like Altair? 00:58:31.920 --> 00:58:34.600 Plotly, out of the box, no. 00:58:34.960 --> 00:58:39.600 It's more, I guess, you know, I have to think behind the scenes, the actual architecture. 00:58:39.600 --> 00:58:45.480 I don't know, but I do know that you don't have to necessarily worry about your data as 00:58:45.480 --> 00:58:50.280 much and suddenly having a huge file that shows up in your notebook that you have to deal with 00:58:50.280 --> 00:58:50.960 Altair. 00:58:50.960 --> 00:58:51.320 Yeah. 00:58:51.320 --> 00:58:51.840 Amazing. 00:58:51.840 --> 00:58:52.940 This is good looking stuff. 00:58:52.940 --> 00:58:53.380 Yes. 00:58:53.380 --> 00:58:58.780 And really top marks on the animation and interactivity, sort of diving into the data, 00:58:58.780 --> 00:58:58.980 right? 00:58:58.980 --> 00:58:59.400 Yes. 00:58:59.400 --> 00:58:59.900 Yep. 00:58:59.900 --> 00:59:00.180 Cool. 00:59:00.180 --> 00:59:00.580 All right. 00:59:00.620 --> 00:59:04.500 So that's the building block libraries for the different options. 00:59:04.500 --> 00:59:09.620 I know there's many other plotting libraries and, you know, we saw in our original graph 00:59:09.620 --> 00:59:12.860 that there's a bunch here that we didn't touch on, but, you know, these are the ones that 00:59:12.860 --> 00:59:14.780 you felt are most important right now. 00:59:14.780 --> 00:59:14.940 Yeah. 00:59:14.940 --> 00:59:15.340 Yeah. 00:59:15.340 --> 00:59:16.540 You know, it's interesting. 00:59:16.540 --> 00:59:19.320 I struggled a little bit with where to draw the line. 00:59:19.320 --> 00:59:23.960 You know, what else, some of the other things I might want to bring in after I posted about 00:59:23.960 --> 00:59:28.900 the course, I did get some feedback, you know, how come we didn't talk about a bouquet and 00:59:28.900 --> 00:59:30.700 panel and hollow viz. 00:59:30.700 --> 00:59:36.220 And, you know, I think the short and sweet answer was I had to draw the line somewhere. 00:59:36.220 --> 00:59:40.780 I didn't have as much experience with those libraries, so I didn't dive into them. 00:59:40.780 --> 00:59:47.540 They are also libraries that are kind of undergoing some, they've undergone changes in the past and 00:59:47.540 --> 00:59:52.400 they, they are working to clean up their documentation, get the examples cleaned up. 00:59:52.560 --> 00:59:56.200 So I think it's certainly worth considering those as well. 00:59:56.200 --> 01:00:01.160 The other one that I would point out that is really interesting to me, but I haven't used 01:00:01.160 --> 01:00:07.100 it, but I think a lot of your listeners might be interested in is a library called Plot9 that 01:00:07.100 --> 01:00:11.300 is meant for people that have used ggplot from an R perspective. 01:00:11.300 --> 01:00:15.420 And it's essentially like a direct port of that to Python. 01:00:15.420 --> 01:00:22.540 So if you really like ggplot and miss it from R, then you can use Plot9 to replicate 01:00:22.540 --> 01:00:27.020 that in Python and it uses a matplotlib behind the scenes. 01:00:27.020 --> 01:00:32.960 And it's, I mean, it looks fascinating to me, especially for people that come from that 01:00:32.960 --> 01:00:33.520 R background. 01:00:33.520 --> 01:00:33.940 Sure. 01:00:33.940 --> 01:00:38.660 That's something we haven't even touched on is like the influence of R and like the parallels 01:00:38.660 --> 01:00:39.100 there. 01:00:39.100 --> 01:00:40.260 And, and... 01:00:40.260 --> 01:00:40.360 Yes. 01:00:40.360 --> 01:00:40.880 Yes. 01:00:40.880 --> 01:00:42.200 We probably want too much. 01:00:42.200 --> 01:00:45.140 So we've got two more things to cover and these are... 01:00:45.140 --> 01:00:46.800 How do I maybe run my code? 01:00:46.800 --> 01:00:50.600 If I want to put it online and make it interactive and let other people interact with it. 01:00:50.600 --> 01:00:50.800 Right? 01:00:50.800 --> 01:00:54.580 So two of them, Streamlit and Plotly Dash. 01:00:54.580 --> 01:00:55.500 Tell us about these. 01:00:55.500 --> 01:00:55.840 Yes. 01:00:55.840 --> 01:01:01.360 So, you know, everything we've talked about now there, especially with Plotly and Altair, 01:01:01.360 --> 01:01:03.440 there is some degree of interactivity. 01:01:03.620 --> 01:01:11.440 But when you want to build a dashboard or want to build more of like an application where 01:01:11.440 --> 01:01:17.800 you can select and filter data, maybe have different visualizations, maybe have complex visualizations 01:01:17.800 --> 01:01:19.700 like, like maps. 01:01:19.700 --> 01:01:24.880 You need something more than just the out of the box Altair or Plotly. 01:01:24.880 --> 01:01:31.940 And Streamlit is a very simple way to wrap a little bit of extra Python code around your 01:01:31.940 --> 01:01:36.020 visualization and you get this interactive application for free. 01:01:36.020 --> 01:01:40.760 And so like the demo you're showing right now is a great, you know, really powerful example 01:01:40.760 --> 01:01:44.120 that shows Uber ride sharing in New York City. 01:01:44.120 --> 01:01:48.060 It has sliders for you to choose what time the pickup happens. 01:01:48.060 --> 01:01:53.840 And then it has these real time visualizations for different parts of the city about how 01:01:53.840 --> 01:01:55.020 many pickups are happening. 01:01:55.020 --> 01:02:02.180 And what's so cool about Streamlit is there is very little additional Python code you need 01:02:02.180 --> 01:02:03.020 to do that. 01:02:03.020 --> 01:02:10.000 So the workflow that I will typically do is I'll do my visualizations in Seaborn or Plotly 01:02:10.000 --> 01:02:10.640 or Altair. 01:02:10.640 --> 01:02:16.040 And then once I realize I need that next level, I can then just easily plop them into a separate 01:02:16.040 --> 01:02:18.540 file with a couple lines of Streamlit code. 01:02:18.540 --> 01:02:21.540 And boom, I've got an interactive application that I can run. 01:02:21.540 --> 01:02:21.960 Yeah. 01:02:21.960 --> 01:02:25.040 It's a really interesting way of programming. 01:02:25.040 --> 01:02:30.960 You basically write a top to bottom procedural script that says, if I, in this case, what are 01:02:30.960 --> 01:02:35.760 we, we're putting, we're putting the hour of a pickup, the hour of data you want to slice 01:02:35.760 --> 01:02:36.320 and visualize. 01:02:36.820 --> 01:02:41.000 And you said, well, if I could write a function that would make a graph given the hour, then 01:02:41.000 --> 01:02:42.800 you just say, and make the web app. 01:02:42.800 --> 01:02:43.820 You know what I mean? 01:02:43.820 --> 01:02:47.860 And it gives you the interactive sliders for all the variables that go in. 01:02:47.860 --> 01:02:51.140 And then you just, as the slider changes, it just recalls your functions. 01:02:51.140 --> 01:02:56.380 And you don't have to know anything about web programming or AJAX or front end code. 01:02:56.380 --> 01:02:57.920 And all of that just happens. 01:02:57.920 --> 01:02:59.520 It's, it's pretty fantastic. 01:02:59.520 --> 01:03:04.000 Streamlit was recently acquired, I think so recently, but within a year or two. 01:03:04.000 --> 01:03:07.780 Yeah, they're acquired by Snowflake for a really large amount of money. 01:03:07.780 --> 01:03:12.800 I know a lot of people are kind of scratching their head at that valuation, not to knock on 01:03:12.800 --> 01:03:13.160 Streamlit. 01:03:13.160 --> 01:03:15.640 I mean, congrats to them, but it's really interesting tool. 01:03:15.640 --> 01:03:19.300 And it will be interesting to see what Snowflake does with them. 01:03:19.300 --> 01:03:22.460 But you know, this tool right now is open source. 01:03:22.460 --> 01:03:29.860 And I do think it is a very powerful, easy way to get a real web native interactive app with 01:03:29.860 --> 01:03:30.820 very little code. 01:03:30.820 --> 01:03:31.740 Yeah, absolutely. 01:03:31.740 --> 01:03:32.480 Let's see. 01:03:32.920 --> 01:03:36.840 Demetrius out there has a question, says, there are all these ways to make graphs quickly, 01:03:36.840 --> 01:03:42.860 but I can't find anywhere on how, any information on how to make an interactive calendar with 01:03:42.860 --> 01:03:43.620 events quickly. 01:03:43.620 --> 01:03:45.020 Yeah, that's a good question. 01:03:45.020 --> 01:03:46.240 I don't know. 01:03:46.240 --> 01:03:47.000 Yeah, I don't know either. 01:03:47.000 --> 01:03:50.360 Of any of these plots that have like a calendar function. 01:03:50.360 --> 01:03:52.200 You don't possibly Streamlit. 01:03:52.200 --> 01:03:53.280 It might. 01:03:53.280 --> 01:03:53.780 Maybe. 01:03:53.780 --> 01:03:57.300 It's interesting that you bring up Streamlit because Streamlit does have like third 01:03:57.300 --> 01:04:00.620 party apps or plugins that you can incorporate. 01:04:01.220 --> 01:04:02.860 that the individual opposed to that. 01:04:02.860 --> 01:04:06.000 I definitely encourage them to take a look at Streamlit and see if there's something out 01:04:06.000 --> 01:04:06.220 there. 01:04:06.220 --> 01:04:10.240 So I interviewed, I believe it was Adrian. 01:04:10.240 --> 01:04:11.720 Let me double check. 01:04:11.720 --> 01:04:11.940 Yeah. 01:04:11.940 --> 01:04:16.420 Adrian Truel back early days, early days of Streamlit. 01:04:16.420 --> 01:04:19.680 I mean, we're talking two years ago before it was acquired about that. 01:04:19.720 --> 01:04:21.080 So people can check that out. 01:04:21.080 --> 01:04:22.500 And I'm somewhat familiar with that. 01:04:22.500 --> 01:04:27.340 The other one, though, that's very fascinating that I don't know about is Dash. 01:04:27.340 --> 01:04:28.920 How's that compared to Streamlit? 01:04:28.920 --> 01:04:31.620 What's the, this comes from the Plotly company as well. 01:04:31.620 --> 01:04:32.100 Yes. 01:04:32.100 --> 01:04:35.260 So Plotly, like we mentioned, is a company. 01:04:35.260 --> 01:04:45.640 They have Dash, which is a much more sophisticated and in-depth platform for developing interactive 01:04:45.640 --> 01:04:47.500 applications or dashboards. 01:04:47.500 --> 01:04:55.900 So whereas Streamlit is a little bit of code, Dash is much more of you kind of are embracing 01:04:55.900 --> 01:05:04.000 HTML and CSS and you're doing callbacks and you have just a ton of flexibility in how you 01:05:04.000 --> 01:05:05.940 structure your application. 01:05:05.940 --> 01:05:10.880 And like this, this demo you're having here, you've got wind speed histograms and you've 01:05:10.880 --> 01:05:16.380 got a line chart that's fully interactive and interactivity between the charts. 01:05:16.380 --> 01:05:19.340 As you choose one, it influences another. 01:05:19.340 --> 01:05:25.700 You can, Dash gives you flexibility to kind of manage the backend as well. 01:05:25.700 --> 01:05:27.040 So you can run it. 01:05:27.040 --> 01:05:31.300 I think it's, it's a Flask server, like on your system, you could do that. 01:05:31.300 --> 01:05:36.360 But if you wanted to do an enterprise grade deployment, you could do that as well. 01:05:36.360 --> 01:05:40.900 It's really, if you can think of it, Dash will probably let you do it. 01:05:40.900 --> 01:05:45.620 And if you're a big enough company and it's mission critical, Dash does have that enterprise 01:05:45.620 --> 01:05:50.080 support where you can pay a company to host it and support it for you. 01:05:50.080 --> 01:05:54.040 You can build, if you, people should go look at the gallery for Plotly Dash. 01:05:54.040 --> 01:05:55.580 There's a bunch of interesting things. 01:05:55.580 --> 01:06:01.220 And one of the areas that stands out, I mean, you've got many, many different types of visualizations 01:06:01.220 --> 01:06:01.740 and whatnot. 01:06:01.740 --> 01:06:05.880 But one of the things that stands out for me is the streaming, streaming data aspect, 01:06:05.880 --> 01:06:12.680 you know, create a dashboard where you hook it up to stock market data or IOT data and 01:06:12.680 --> 01:06:13.940 it just goes, right? 01:06:13.940 --> 01:06:14.400 Right. 01:06:14.400 --> 01:06:15.180 Exactly. 01:06:15.420 --> 01:06:22.240 It's designed to, you know, support a lot of data and low latency and all those kinds 01:06:22.240 --> 01:06:22.660 of things. 01:06:22.660 --> 01:06:28.900 So it's, it's really powerful, but this is one of those areas we talked about everything up 01:06:28.900 --> 01:06:29.460 until now. 01:06:29.460 --> 01:06:31.880 You don't really have to know a whole lot of Python. 01:06:31.880 --> 01:06:36.280 Once you start getting into building Dash, it gets a little more complicated. 01:06:36.280 --> 01:06:41.380 And I think that's where you want to make sure you've got a good solid Python foundation 01:06:41.380 --> 01:06:45.440 before you go and build a dashboard to run your company. 01:06:45.440 --> 01:06:45.740 Yeah. 01:06:45.740 --> 01:06:47.260 But it's very powerful. 01:06:47.260 --> 01:06:47.880 Very powerful. 01:06:47.880 --> 01:06:48.200 Yeah. 01:06:48.200 --> 01:06:51.360 The interactivity between the different elements is also quite interesting. 01:06:51.360 --> 01:06:52.020 It is. 01:06:52.020 --> 01:06:52.500 Yes. 01:06:52.500 --> 01:06:58.140 And that's one of the, I mean, you can do that to some degree with Streamlit, but Dash just 01:06:58.140 --> 01:07:04.160 makes, yeah, you can have a ton of interactivity between the different widgets and the different 01:07:04.160 --> 01:07:05.020 visualizations. 01:07:05.020 --> 01:07:05.420 Cool. 01:07:05.420 --> 01:07:06.300 Well, it looks great. 01:07:06.300 --> 01:07:09.700 If I had a dashboard that looked like this, I'd be proud of it. 01:07:09.780 --> 01:07:13.180 It's not one of those things that's just like, oh, I guess I guess it works. 01:07:13.180 --> 01:07:14.460 You know, no, it looks great. 01:07:14.460 --> 01:07:14.940 Yeah. 01:07:14.940 --> 01:07:15.700 I would be too. 01:07:15.700 --> 01:07:16.040 Yeah. 01:07:16.040 --> 01:07:16.280 Cool. 01:07:16.280 --> 01:07:19.760 All right, Chris, we spent a lot of time on, on this. 01:07:19.760 --> 01:07:21.760 I think we should probably button it up, but. 01:07:21.760 --> 01:07:22.040 Yeah. 01:07:22.040 --> 01:07:22.720 Bring it home. 01:07:22.720 --> 01:07:23.760 A lot of great stuff. 01:07:23.760 --> 01:07:24.200 Yeah. 01:07:24.200 --> 01:07:27.240 A lot of great stuff in the visualization library space. 01:07:27.240 --> 01:07:32.700 I think that's one of the really very powerful aspects of Python is it's just all of these 01:07:32.700 --> 01:07:33.020 tools. 01:07:33.020 --> 01:07:34.140 It's not the language. 01:07:34.140 --> 01:07:37.440 It's not the standard library, which all those, they are important. 01:07:37.440 --> 01:07:41.440 Like it's, people need to think this is sort of what people are talking about when they 01:07:41.440 --> 01:07:42.400 say Python's awesome. 01:07:42.400 --> 01:07:43.580 It's great to use. 01:07:43.580 --> 01:07:45.060 It's not how it does a for loop. 01:07:45.060 --> 01:07:47.760 It's that I can say, you know, dot Andrew's curve. 01:07:47.760 --> 01:07:48.700 Yes. 01:07:48.700 --> 01:07:49.220 Yeah. 01:07:49.220 --> 01:07:51.120 And it all builds, right? 01:07:51.120 --> 01:07:55.640 So if you're, you know, just starting on Python, then you start to build a little pandas 01:07:55.640 --> 01:07:56.380 knowledge. 01:07:56.380 --> 01:07:58.980 You don't have to throw that away and then focus on visualization. 01:07:58.980 --> 01:08:04.400 It all builds on top of it and you can leverage all that knowledge and then all the other wonderful 01:08:04.400 --> 01:08:05.500 libraries that are out there. 01:08:05.740 --> 01:08:05.980 Absolutely. 01:08:05.980 --> 01:08:06.560 All right. 01:08:06.560 --> 01:08:08.460 Before you get out of here though, final two questions. 01:08:08.460 --> 01:08:12.580 If you're going to write some Python code, what editor or editors do you use? 01:08:12.580 --> 01:08:15.160 I'm pretty much a hundred percent VS Code now. 01:08:15.160 --> 01:08:15.600 Right on. 01:08:15.600 --> 01:08:16.640 Even over notebooks. 01:08:16.640 --> 01:08:17.200 Yes. 01:08:17.200 --> 01:08:17.860 Yes. 01:08:17.860 --> 01:08:25.140 I've gotten to where I use the native VS Code notebooks and I really like that. 01:08:25.140 --> 01:08:27.180 Like the comment divider type of style. 01:08:27.180 --> 01:08:34.000 It's just, they, they have continued to update it so much that it just seems like it's a superior 01:08:34.000 --> 01:08:38.100 approach for what I do and how I manage my environments right now. 01:08:38.100 --> 01:08:38.400 Nice. 01:08:38.400 --> 01:08:41.980 And then notable PyPI package or as. 01:08:41.980 --> 01:08:42.180 Yeah. 01:08:42.280 --> 01:08:43.460 I put two in here. 01:08:43.460 --> 01:08:47.840 I don't have a ton of experience with them, but I wanted to call them out because I do 01:08:47.840 --> 01:08:49.120 want to spend some time with it. 01:08:49.120 --> 01:08:50.840 So the first one is Splink. 01:08:50.840 --> 01:08:56.580 I wrote an article a couple of years back about doing data linkage or data duplication. 01:08:56.800 --> 01:09:01.240 And so for, for those of you that aren't familiar, it could be a situation where let's say you 01:09:01.240 --> 01:09:05.960 have your customer database and it's got Chris Moffitt lives at one, two, three main street 01:09:05.960 --> 01:09:08.440 and you have a third party data set. 01:09:08.440 --> 01:09:14.960 And it says, Mr. Moffitt lives at one, two, five main street and street is spelled a street. 01:09:14.960 --> 01:09:17.780 How do you merge all that data together? 01:09:17.780 --> 01:09:19.200 How do you do fuzzy matching? 01:09:19.200 --> 01:09:22.240 And I played around with different options. 01:09:22.820 --> 01:09:29.240 And this Splink is one that's actually came out of the UK from an individual that works 01:09:29.240 --> 01:09:32.480 at the Ministry of Justice, which I think is just a cool name. 01:09:32.480 --> 01:09:38.260 And he talks about using this to, to merge, you know, millions of records together. 01:09:38.260 --> 01:09:44.120 And I think it's a kind of really interesting tool that is something that you can't really 01:09:44.120 --> 01:09:46.540 do in Excel and it's a challenging problem. 01:09:46.540 --> 01:09:51.120 And anytime someone's spent some time on that kind of problem, I think it's really interesting 01:09:51.120 --> 01:09:52.860 and I want to spend some more time looking at that. 01:09:52.860 --> 01:09:53.120 Yeah. 01:09:53.120 --> 01:09:56.960 And they have on that GitHub repo, they got a couple of videos introducing it, which is great. 01:09:56.960 --> 01:09:57.300 Yes. 01:09:57.300 --> 01:09:58.040 And the other one. 01:09:58.040 --> 01:09:59.040 Red frames. 01:09:59.040 --> 01:10:04.280 So this is another one that I've seen come across my Twitter feed a couple of times. 01:10:04.280 --> 01:10:12.100 I have not used it directly, but it is another library for manipulating data that's interoperable 01:10:12.100 --> 01:10:18.420 with pandas, but gives a little bit more of that fluent API where you can kind of string 01:10:18.420 --> 01:10:24.780 all these commands together to modify your data in ways that pandas supports a lot of this, 01:10:24.780 --> 01:10:27.940 but there are certainly some things in pandas that are a little bit clunky. 01:10:27.940 --> 01:10:33.840 And this looks like it's an attempt to try and bring some of that R goodness to Python. 01:10:33.840 --> 01:10:34.160 Yeah. 01:10:34.240 --> 01:10:36.920 It also looks a little bit like bringing SQL to it. 01:10:36.920 --> 01:10:37.020 Yes. 01:10:37.020 --> 01:10:37.480 Yeah. 01:10:37.480 --> 01:10:37.620 Yeah. 01:10:37.620 --> 01:10:37.920 Yeah. 01:10:37.920 --> 01:10:38.600 Certainly there's. 01:10:38.600 --> 01:10:42.260 A filter, a group, a sort, you know, change the names a little bit. 01:10:42.260 --> 01:10:42.640 Yeah. 01:10:42.640 --> 01:10:45.500 Filter away or sort order by, you know, it looks a little bit like SQL. 01:10:45.500 --> 01:10:47.140 It just looks really interesting. 01:10:47.580 --> 01:10:52.700 And like I said, I wanted to get some visibility to it and I haven't used it extensively, but 01:10:52.700 --> 01:10:54.380 certainly want to play around with it a little bit more. 01:10:54.380 --> 01:10:56.540 I thought your listeners might be interested. 01:10:56.540 --> 01:10:56.820 Yeah. 01:10:56.820 --> 01:10:57.620 It looks very cool. 01:10:57.620 --> 01:10:58.940 Thanks for sharing that. 01:10:58.940 --> 01:10:59.380 All right. 01:10:59.380 --> 01:11:02.000 People are interested in Python data visualization. 01:11:02.000 --> 01:11:03.220 They want to know more. 01:11:03.220 --> 01:11:04.160 What do you tell them? 01:11:04.160 --> 01:11:05.540 Hey, check out the course. 01:11:05.540 --> 01:11:08.220 So really excited about the course. 01:11:08.220 --> 01:11:14.880 If you've liked what you've listened to here, the course on Talk Python training has the examples, 01:11:15.340 --> 01:11:18.760 the notebooks for you to go through and learn and play with this on your own. 01:11:18.760 --> 01:11:22.280 And by the end of it, you should be at the point where you can start to apply it to your 01:11:22.280 --> 01:11:22.660 own data. 01:11:22.660 --> 01:11:24.320 So encourage you to check it out. 01:11:24.320 --> 01:11:25.660 And if you do check it out, let me know. 01:11:25.660 --> 01:11:26.900 Be interested to see what you think. 01:11:26.900 --> 01:11:27.120 Yeah. 01:11:27.120 --> 01:11:31.360 It definitely covers all of this in hands-on detail, not just conceptually. 01:11:31.360 --> 01:11:32.440 So yeah. 01:11:32.440 --> 01:11:33.740 Thanks for being here, Chris. 01:11:33.740 --> 01:11:35.260 Thanks for sharing your experience. 01:11:35.260 --> 01:11:36.560 And yeah. 01:11:36.560 --> 01:11:37.600 Happy to have you back on the show. 01:11:37.600 --> 01:11:38.000 Thank you. 01:11:38.000 --> 01:11:38.900 Great discussion. 01:11:38.900 --> 01:11:39.480 Yeah, you bet. 01:11:39.480 --> 01:11:39.780 Bye-bye. 01:11:39.780 --> 01:11:40.000 Bye. 01:11:40.000 --> 01:11:43.880 This has been another episode of Talk Python To Me. 01:11:44.480 --> 01:11:45.700 Thank you to our sponsors. 01:11:45.700 --> 01:11:47.320 Be sure to check out what they're offering. 01:11:47.320 --> 01:11:48.740 It really helps support the show. 01:11:48.740 --> 01:11:50.960 Starting a business is hard. 01:11:50.960 --> 01:11:57.060 Microsoft for Startups, Founders Hub provides all founders at any stage with free resources 01:11:57.060 --> 01:11:59.560 and connections to solve startup challenges. 01:11:59.560 --> 01:12:04.100 Apply for free today at talkpython.fm/founders hub. 01:12:04.900 --> 01:12:06.220 Want to level up your Python? 01:12:06.220 --> 01:12:10.340 We have one of the largest catalogs of Python video courses over at Talk Python. 01:12:10.340 --> 01:12:15.440 Our content ranges from true beginners to deeply advanced topics like memory and async. 01:12:15.440 --> 01:12:18.100 And best of all, there's not a subscription in sight. 01:12:18.100 --> 01:12:21.020 Check it out for yourself at training.talkpython.fm. 01:12:21.100 --> 01:12:22.920 Be sure to subscribe to the show. 01:12:22.920 --> 01:12:25.700 Open your favorite podcast app and search for Python. 01:12:25.700 --> 01:12:27.020 We should be right at the top. 01:12:27.020 --> 01:12:32.160 You can also find the iTunes feed at /itunes, the Google Play feed at /play, 01:12:32.160 --> 01:12:36.360 and the direct RSS feed at /rss on talkpython.fm. 01:12:36.360 --> 01:12:39.800 We're live streaming most of our recordings these days. 01:12:39.800 --> 01:12:43.200 If you want to be part of the show and have your comments featured on the air, 01:12:43.200 --> 01:12:47.640 be sure to subscribe to our YouTube channel at talkpython.fm/youtube. 01:12:48.040 --> 01:12:49.480 This is your host, Michael Kennedy. 01:12:49.480 --> 01:12:50.760 Thanks so much for listening. 01:12:50.760 --> 01:12:51.940 I really appreciate it. 01:12:51.940 --> 01:12:53.840 Now get out there and write some Python code. 01:12:53.840 --> 01:13:14.440 I'll see you next time. 01:13:14.440 --> 01:13:44.420 Thank you.