#308: Docker for Python Developers (2021 Edition) Transcript
00:00 Docker is one of the core elements of developing Python applications in consistent ways as well as running them across different hardware universally. On this episode, you'll meet Peter McKee from Docker. He's here to catch us up on what's happening in the Docker universe for Python developers. This is talk Python to me, Episode 308, recorded March 9 2021.
00:34 Welcome to talk Python, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm @mkennedy, and keep up with the show and listen to past episodes at talk python.fm. And follow the show on Twitter via @talkpython. This episode is brought to you by 'Datadog and Us' over at Talk Python Training. Please check out what we're offering during our segments. It really helps support the show a little bit of news, we just released not one, but two new courses and more are just around the corner. You might remember my interview with Sebastian foskey over on episode 279, where we discussed the 'Modern Python developers toolkit'. Since then, he and I have teamed up to bring you this whole idea of modern tools applied to well structured Python projects as a course. And we just released it. It's called 'Modern Python projects'. You can find it over at
00:34 talk python.fm/modern-python-projects, or you'll find the link in the show notes. If you like the idea of a total toolchain makeover for how you're building your Python projects and applications, be sure to drop by the course page and watch the overview video to see if it's right for you. I'm sure everyone will learn something interesting in this course, there's so many cool things in here, no matter your experience level. Another new course we just launched is 'Full web apps with FastAPI'. 'FastAPI' is definitely the framework with the most energy lately. But if you build your API's with 'FastAPI', will you need another framework for your HTML user interaction side? In other words, can you replace 'Flask' or 'Django' with 'FastAPI'? The answer is 'Yes'. And this course is your guide to doing so check it out at talk python.fm/FastAPI-web, link in the show notes. I'm really excited about these two courses. And I hope you all love them. We'll be giving away five tickets to attend 'Pycon us 2021'. This conference is one of the primary sources of funding for the PSF. And it's going to be held may 14 or 15th. Online. And because it's online this year, it's open to anyone around the world. So we decided to run a contest to help people especially those who have never been part of 'Pycon' before attended this year, just visit a 'talk python.fm/pycon2021'. And enter your email address. And you'll be in the running for an individual Pycon ticket complements of talk Python. These normally sell for about $100 each. And if you're certainly want to go I encourage you to visit the 'Pycon' website, get a ticket and that money will go to support the PSF and the Python community. Congratulations to Tammy Doe, Tammy won, the second of five tickets we're giving away and there's still more chances to win. If you want to be in this drawing, just visit talk python.fm/pycon2021. Enter your email address. You'll be in the running to win a ticket. Now let's get on to that interview. Peter, welcome to talk Python to me.
03:13 Yeah. Thanks for having me. Great to be here.
03:15 Hey, it's really great to have you here as well. It's been far too long since we've talked 'Containers' and 'Docker' and all the things I was looking back at something like show nine or something like that. It was I believe 2015. Has anything changed with Docker since 2015? I suspect something has
03:29 Yeah, there's been it's Yeah, 'containers' are interesting. They're moving. moving very fast. Right, Very fast. Yeah, on just about all friends. I think the whole developer 'DevOps' , 'Devsec Ops', you don't just have embraced it the past five years.
03:42 Yeah, I was thinking what you might might respond to from that question. And it to me, it seems like not so much how containers work and run, but just how people have embraced them and how many places they show up and how many people are saying, Hey, we have a cluster, you just run your thing over here. Just give us your container and we'll help you run it as part of our 'Platform as a Service' or whatever it is there. Yeah, about hosting. Yeah,
04:04 yes. Very interesting. And then there's all these little micro maybe micros not the correct term, but all these a lot of 'SASS'es and startups and trying to help in containers at the at the core of a lot of that you get into that container we can run it and do some really interesting 'SASS'es there starting up little niche platforms as a service type things. Yeah. 100%
04:23 Yeah, absolutely. It's it's super interesting. Before we dive too deep into that though, let's get what started your story. What's How did you get into programming
04:30 as a kid? My dad's an accountant, but a nerd at heart, you know, a techie, let's say and he started out this is interesting, at least to me, he started out with calculators before. PCs were a big thing. He had calculators and he programmed these calculators and stuff. And anyways, long story short, he ended up getting I think Apple two was the first kind of computer we had and I was fascinated about it, started programming on it just learning messing around. Fast forward after after high school. I was in the Coast Guard and had gotten out and was going back to school. And, you know, I started back as an accountant, my dad said, Hey, accounting is a good foundation like math and science and those type of things, and I hated it, hated it with debits and credits. And I was just like, micro macro. Now it's very, yeah, now it's a lot more interesting back then I couldn't stand it. But but then I lucked out. My father was running 2 steel companies. And he had a consulting team that that was writing software for him, you know, like, very exciting stuff like inventory management, working process tracking, right.
05:29 It's not exciting, but it's the kind of stuff that if you look at what people were doing before was dreadful, right. It's like transformative, even though it seems so boring and basic. Yeah,
05:38 yeah. And he said, Why don't you much come work for me and this consulting firm, the main the owner had some life issues, whatever, and had to move on and, and I said, Well, I don't you know, I don't know what I'm doing. He said, Oh, we'll figure it out. There's a big Barnes and Noble at the top of the hill there you can we got a whole section on it. Yeah, it literally Yeah. And and I did, and I did and you know, it was I had a passion I had a burning desire, right to learn. It was fascinating to me. Yeah. And so I switched over a computer science degree 'MIS' and but by the time I got farther along, in that I was getting paid the program and I was learning a ton more on the job. So that kind of remember
06:15 your first experience of getting paid to write code. Was it just like, I can't believe actually paying me to do this? Absolutely. Absolutely. It was.
06:23 Yeah. I mean, it's literally how they say, you know, find something you love and are passionate in and the money will come. And yeah, it did. It certainly did. I was I really enjoyed it. And yeah, it was getting paid to learn and write application. So yeah, it was it was awesome. It was awesome. Yeah,
06:37 it's fantastic. I remember my first experience like that as well. I'm like, I had better figure this out before they realize I can't actually do this stuff. Yeah, I could do the things they want. If I like, oh, any moment, they're just gonna say no, if you actually don't get to do this anymore, but it was it was great.
06:50 Yeah, yeah, exactly. By that time, I said, You know, I have to get on a bigger team and learn. I know, I got to the point where, okay, I can write programs, but I don't know how to be an engineer. And I'm not sure I could put it in those type of words. But but luckily, after that, I got on a team and had some great mentors. So yeah, that's, that's how I got to kind of get started.
07:07 Yeah, fantastic. I think that's good advice. You know, it's, it's really cool to be on small teams, because you really get your hand in so many areas you're not pigeon holed into Well, I'm the one who optimizes a stored procedure. So I do that, right. You get like a really broad experience. But at some point, when it's like, I, there's no one I can talk to, or learn from to go farther in this than it. It gets to be a little bit limiting,
07:30 I think, yes, exactly. Like, well, that's how I got from it. I was like, I don't know where to go next. And I don't know who you know, I didn't have anybody there with me. Yeah, I think having a mentor, either, you know, well structured or kind of loosely structured, or whether you look back on it five years ago. Oh, that person was kind of mentor for me is very powerful. Right is very powerful.
07:48 Yeah. Yeah, absolutely. It totally is. I remember making that transition to a really great group of people who were every one of them. I felt like was way smarter than me. And it was intimidating. But absolutely was a good outcome.
08:00 Yeah, that's the place to be right. Unless, you know, of course, I think a lot of us too, in tech deal with the ego issue. Sometimes. I know I did, right? Like, Hey, I'm pretty smart. I know what I'm doing then you when you're young and ambitious, right? And you run into other really smart people that like are smart. And you go, Okay, I'm not that smart. Yeah, it's a little humbling. But it can be humbling. Yeah,
08:20 those are probably good life lessons, but they're hard to
08:22 remember. Yeah, absolutely. Yeah.
08:25 Fantastic. So How about now? What are you doing day to day?
08:28 Yeah, so I head up the Developer Relations team at Docker. And I get to do what I love. You know, I still get to play around with tech and learn new things and build stuff. But then I also get to transfer that knowledge to others, or at least try. So teaching. And mentoring is what I do mostly day to day write content and workshops and those type of things. I help folks learn to use 'Containers' and 'Docker' to help build applications, right. So yeah, it's it's fantastic. I love my job I truly,
08:54 I can imagine, you know, I really thought for a long time that this developer evangelists sort of role just seems so neat, because you get to work with a bunch of people, you're not really there just closing boring JIRA tickets every day, right? You get to just work at like the fun level of software and really interact with a lot of different people. You get to go to conferences, maybe be on a podcast, all those kinds of things. I just it's a really cool space to be after you've learned enough in the industry to play that role.
09:23 Yeah, 100%. I remember seeing I had thought about the role before, but I think it was at Google's, what does it 'Google Next'? I think it's their conference. And that was when I kind of I known about 'Dev advocates' and 'Dev rel', but when I saw a couple speakers like Oh, wait, 'Dev alleys', oh, they get paid to do this. This isn't just, you know, Peter, who's a software engineer then speaks right? This is their job. And I was like, Well, yeah, that's pretty interesting. Yeah. Yeah.
09:46 Yeah. It's I think, you got to have the right personality for it. But I think it's really fantastic. I did actually a whole Episode Episode 189 on this role, and I had four or five people who were at different companies doing this. So yeah, it was it was really really neat. So, you know, I do want to catch up on what is new, and Docker and containers in that whole space. But there are a lot of people with lots of different backgrounds. Let's, you know, listen to show who I'm sure everyone's heard of Docker and containers, right? It's like hearing of 'Linux' or something. But that doesn't mean you're super familiar with it, or you really understand the advantages or maybe the history. So let's actually maybe start with just like the history of containers a little bit. I know, when I first saw the talk by the original, I think was the original CEO of Docker gave a really interesting sort of tie back to shipping containers. Yes. So was this whole idea of containers? Where's it come from? What what's the deal?
10:39 Yeah, so when I, when? Yeah, that was him. When I first saw his his I think it was at a 'PY conference'. Now that I think about it actually in Europe. Yeah, he gave a quick brief kind of like, what he called lightning talk, right? And it kind of went over at the end. And people like went kind of crazy. And I remember kind of seeing and I was like, if you ever go watch it, I recommend people go watch it. And kind of look for the aha moment. At least it took me a bit right. And you know, he did ran these commands. And then Oh, yeah. And then there's this process running inside of a container and super easy. And I forget what he demoed. But it took me a while right, to be honest. Right? He stepped back alley, whoa, okay. We have VMs What is this container thing? And it took me a little bit to figure out like, oh, wow, this is super powerful. But you know, 'containers', the constructs to build 'containers' have been around for a while since the 70s. Right, and but in Unix, different 'Unix', 'Unix flavors' would change Root and those type of things. And then I think, around 2000, I believe was when the 'V server' came out. And then and then about eight years, so we had 'Linux v serve', and FreeBSD had some, you know, container like functionalities to isolate processes into those type of things. So 'v server' was kind of like a virtual, you know, sitting on top of the core kernel and having some virtualization there. Output virtualization. When
11:53 I first heard about Docker, I thought, Oh, this is this amazing new thing. And then as I researched it more I'm like, actually, this, the pieces had always, not always, but they had been there for a while, like these pieces you're talking about. They just hadn't really been put into a nice little package the way Docker has, right?
12:09 Yeah, yeah. And I think that's the beauty of what Docker did. And Solomon and his team, right. There's, I think there was three founders. But yeah, two is was it 2013? There it is on the screen. Yeah. So 2013, Docker, kind of took off, right? It kind of exploded, it was just that and, you know, they simplify containers, these constructs were all there, but they made it easy. The ability to deep you know, drop back down in into the details or stay very high level and, and use, you know, 'Docker run', 'Docker push', you know, those 'Docker builds', right, those three main commands, were super powerful. And I think that was the beauty of Docker. They just made it easier for the average developer or, you know, for if you had to be a 'Linux' 'Kernel Developer' to kind of really understand containers before that. Yeah, they simplify. Yeah,
12:52 'Linux Admin' or something like that. Right? Yeah.
12:54 100%. And it's kind of like, I think Microsoft did a great job at this. I think 'Apple's done a great job of this, you know, Microsoft with, you know, VB, right. And I'm gonna get beat up on Internet's for this visual, 'Visual Basic', the Visual Basic 6, I mean, kind of weird, but it was a game changer, right? You had 'C++', 'MFC', you know, writing these 'C++' to write desktop apps and business apps, right? And 'VB 6' came along, said, Oh, no, no, it's you know, if you need a textbox, and some labels and a button, just drag it on there, right?
13:23 Look, I won't be jailed for that. 'VB' was transformative. It was magical. Yeah, the problem with VB was you could go so far, and then you're like, well, VB doesn't do that. So now you now you get a grown up. Now it's time to put your big boy pants on and get a grown up language and suffer the, you know, 'C++' huge gap. But for what a lot of people bill with that. It was incredibly, to be honest, there's nothing like that today that I can think of that is so powerful for building a little, distributable application I can give to you. Yeah, 'Electron' feels super complicated and hard. 'Python', you can build stuff, but it's always kind of like, but this is a challenge or that, you know, even though the windows stuff with 'WPF' is it's way more complicated than it was magic, right? It's just It's too bad that there's not sort of a semi modern version of something that simple for people.
14:13 Yeah. If, who knows if 'Visual Basic' had curly braces might have a different story. I mean, perhaps? Like, truly Right. I mean, I remember having debates with it. Yeah. What's not a real language is you know, it's basic. It's the basic programming language, right? Well, yeah, yeah.
14:28 Yeah. Yeah. And john sheen out there in the live stream says rad concepts seem to be coming back as these low code, no code platforms. That's a really interesting tie in there. I do agree with that. I think that they've gone even lower than or simpler than what Visual Basic was to like, really, really simple. So yeah, as much as exactly equivalent, but it's definitely You're right. Those things are definitely coming back.
14:48 Yeah, I think we were real complicated. Now. We're going real too simplifying right
14:52 as the pendulum swings hard. Yeah,
14:53 I think we'll find it. I do believe though that with technology right, abstractions get higher and higher and Well, this will kind of roll in a little bit into where the container history went next and, you know, to 20 1718 ish Kubernetes kind of dominated the orchestration, right? And still does, but it is, I was gonna say horribly, horribly complex, and maybe it is maybe I'll get beat up for that too. But it is extremely complex, right? extremely complex.
15:21 And you're mostly talking about like running a 'Kubernetes cluster' is complex, not necessarily like somebody has already set up for you and you give them Yes, have a definition and say run that, like that part might not be so hard, but like if you're gonna do the whole stack, right? Yeah.
15:34 The whole stack from the ground up, right? Yeah. It's it's a bunch of moving parts, getting install getting everything working together. Security, networking. Yeah, it's it's, it's not not simple, right? It's not simple at all.
15:46 This portion of talk Python, to me is brought to you by 'Datadog'. Are you having trouble visualizing 'latency' and CPU or memory bottlenecks in your app, not sure where the issues coming from or how to solve it. 'Data dog' seamlessly correlates logs and traces at the level of individual requests, allowing you to quickly troubleshoot your Python application. Plus, their continuous profiler allows you to find the most resource consuming parts of your production code all the time, at any scale with minimal overhead. Be the hero that got that app back on track at your company. Get started today with a free trial at talk python.fm/datadog , or just click the link in your podcast player shownotes. Get the insight you've been missing with data dog.
16:27 I think it'll get easier and easier and easier. And orchestrator runtimes. You know, I see a lot in tech, like the 'App Engine' from 'Google' with 'Python', right? We're basically serverless now, but when it first came out, you're like, like, you're gonna run my whole app? No, no, no, I need to control all that. Yeah, so we did away with it. But now I'm coming back, you know, serverless, we're kind of coming back to it. Same with Kubernetes. Right? Oh, you're ECS in ACI on, you know, hey, give me a container. I'll just run it and manage it. You know, we said no, no, no, I need to control all that. And then you get this massive Kubernetes. And now I think we're going back, right? Yeah, I just want to give you my containers and run them for me, right? I don't have time to deal with all this in Google's and Google announces 'Autopilot', those type of things. Right. So I think that's the way we're going to go. And I think 'Docker' as a company. About a year ago, when we sold off the enterprise a little over a year ago sold the enterprise business off got of kind of packaged software, enterprise software moved more to 'SAAS' back to development. Yeah,
17:21 well, let's dig into that for a minute. Because I remember seeing Docker coming along, and it was like, Oh, this is a really cool, really cool dev tool. But how are they going to make any money out of this? And I feel like the first big step was, well, what we're going to do is we have special offerings for enterprises and help them, you know, tame their zillions of different types of software they need to run, it sounds like that. That's not the case anymore. What's that transition was what happened with Docker around that,
17:45 I think, first the business model was we shifted to an enterprise business model. Prior to that Docker was just exploding, right, and building tools. And you know, every Docker con, there was a new release and a new tool. And you know, a lot of them would fizzle out and stuff like that. And the people with the purse strings, you know, the money said, Hey, hold on, you know, this is great. You're doing some wonderful things, but we want our money back eventually. So can you go sell something? And they probably Yeah,
18:09 open source in Colombia. We all love it. But like, we're investors. Where's our TEDx? Yeah,
18:14 yeah. So we shifted a little bit to enterprise. And you know, it was just an old business model, I think, on prem long licenses, three year licenses, at least a year, if not three years, and then consulting on top of that, so you know, paying a couple $100,000 for the software, and then and then twice that for consulting services is just a model that's a little bit outdated. And you know, it's still there, of course, but you know, 'Salesforce' was really the first that kind of was starting to drive the nail into the coffin, so to speak,
18:42 right? Well, it seems to me where a lot of the those stories are moving are, are, you know, are you moving to Azure? Are you moving to 'AWS'? Are you moving to 'Linode'? Like, where instead of trying to solidify and really modernize the data center inside companies, a lot of them are going How can we just get out of the data center business? In which case, that's a totally different story, right? In terms of like how they work with you guys,
19:03 we did have an advantage, you know, 'Cross Cloud', one single pane of glass for all your containers across clouds, internal external, but it was just it was a day late and $1, short with funding, when we kind of tried to do that. And yeah, so that sold off the enterprise business and focus back on what Docker does really well is build development tools, development experiences, making container development first, really easy, right? And that's kind of what we're focused on. We call it shift left, right, focusing back on the developer. We're still very interested in 'Ops' and 'DevOps' and 'Dev Sec Ops', but more from a developer set perspective and not that day to long term running your production environments and stuff. Yeah. And so we got got small again, got focused again, you know, we have something like 10 million developers attempted desktop installs, you know, billions of images being pulled out of out of hub, and kind of looked at and said, hey, there's a business here, right? Just official images on hub is a huge, huge asset for us, right? You know, with Python and packages and all the languages and runtimes right? To get really good high quality scanned packages is hard. Right? And and how does that so you know where you're getting your Mongo database from who's produced it, Docker kind of puts their stamp on it.
20:17 Okay, so you guys have like, officially verified certified either direct images or Docker files that build the images out there.
20:25 Yeah, yeah. So we have official images, which we kind of say, Hey, this is following best practices. This is, you know, been scanned for security vulnerabilities. That doesn't mean there's not security vulnerabilities in there. If anybody works in the in the security space, right? That's, like impossible, but yeah, lowers your attack surface.
20:42 This is we're recording this the week that we're basically hearing your 60,000 direct companies have been compromised by the Microsoft Exchange vulnerability, and then they're probably providing products out there, right. Like, it's, it's tough. Yeah, yeah, it is. But at the same time, there's also a lot of news around supply chain vulnerability in the solar winds thing was a supply chain vulnerability. I don't believe the don't know, but I don't believe the exchange one was, but you know, Docker is certainly part of that supply chain, either strengthening or weakness there, right. Like we hear about how 'PyPI', and 'NPM' and 'Ruby Gems', and all of those have people have been, you know, putting renewed interest in trying to get bad things into there. And I always have the same thought about Docker, if I just grab an arbitrary Docker file off the internet, because Oh, look, this configures the thing that I need and I Docker build Docker run, while it might be configuring it with extra goodies for someone else right in there, right. And so that sounds like a really good thing to at least have verified to the degree that you guys can.
21:40 Yeah, yeah. You used to be with security. Right? It's, I remember sitting at a large computer manufacturing company here in central Austin, Central Texas. I won't mention their name. But you know, getting Oh, nice. Thank you. Yeah. So we're, you know, launching a big release, and we're all sitting in the go, No, go meeting. And it's, you know, a couple days before release. And of course, then that's when the security team walks in. Right. And, oh, you know, like, literally like dropping down. Hey, you didn't fill out your your TPS reports. And
22:11 exactly, well, it's such the wrong time to do it. Because, you know, a lot of those ideas have to be built in early and you gotta Yeah, it's really tricky to layer that on at the very, very end
22:20 and 100%. And now we containers, developers a lot more are closer to the operating system and worrying about it, you know, before you had your VM and you had, you know, you had 'OP's folks taking care of those badges. And keeping,
22:33 here's my code, it's going to need engine x and micro whiskey. You figure that out. Go
22:36 right. Yeah, now we're throwing it in containers and devs are doing a lot of that. So shifting that security even farther left, hey, scan your images make, you should know what's going in your images, you know, what base images you're using, those type of things are super important. Yeah. Okay. And controlling that whole life, like you said that that whole lifecycle, right of when you're building your images, and how that through 'CI/CD', right, doing the consistency consistently, making sure you're doing scans, right. So you can predict that a whole pipeline? 100%? Yeah, we
23:03 definitely need to be thinking about that. It's an industry. It's serious.
23:06 Yeah, there's so many smart people out there, and they move fast, they move really fast. And keeping up with the hackers is hard.
23:14 That's for sure. That's for sure. So one of the things I thought it might be fun to touch on is just some of the areas that containers are useful. Maybe some of their their benefits, things like that. Right. I think there's a lot of different use cases that are not well, it makes DevOps more consistent with development, even though that may be one of the benefits.
23:33 Yeah. You know, the biggest problem that Docker solves is that packaging, it runs on my machine, right. And that's been part of our marketing since the beginning. It works for me, right?
23:42 There's a fantastic thing that people should google maybe if I find I put it on the show before, but there's a thing called 'It works. 'It works on my machine' certification, you can get the official certification. This is done by Jeff Atwood. And some folks, oh, that's fine. Another guy over there. So you get this big banner says 'It works on my machine'. And the way you get it as you compile your code and getting the latest from any version control. And the reason for that is totally optional. And up to you, you launch the application that has been compiled, you cause at least one code path to be the to that needs to be checked to be executed. The preferred way to do this is 'ad hoc' manual testing. You may omit this step if the code change was less than five lines or in the developers professional in the end. And then you check it into source control. Right. And so yeah, I mean, that's, that's a joke. And the reason I bring it up is it's just, you know, so common in the industry that you run into this, this problem, right that, well, we test it over here and under this environment works fine, but in a different environment. It doesn't. And, you know, we try to backfill that with things like 'CI/CD' and whatnot, but it's, you know, it's still it's not a perfect fit. So with the Docker, you can get much closer to running on exactly the same system.
24:55 Yeah, yeah. The idea is not only package your app, but package the operating system. all of its dependencies, your app, its configuration, all the application dependencies, all in one package right into an image, and then pass it share that around, right makes it so much easier at that large computer manufacturer, right, we have two huge data centers, and I talked about this in my Getting Started video, you know, 50 in each center, just for the Browse layer, and dub dub dub. And if you go through and patching those machines, those 50 and you miss one, and of course, we had scripts and all that, but, you know, we still had to manually you list of what servers are in and out of the out of the data center, who's in DNS who's not right, you miss one, and then you get this these these random errors where I hit three good servers, the fourth one breaks, and then you hit 5. And you go back to server two, and you're fine,
25:39 right? Yeah, that's really tricky. When it's going to, you know, could be wrong round robin type of exactly balancing, and you're like, Well, sometimes it's wrong, but I don't know when, right, yeah.
25:49 So we used to just go and you know, restart all the servers. One by one. Some of the some of the troubleshooting Yeah, good. troubleshooting get into is like, we have no idea. Yeah, it's I think Docker really, really attacks that really well. But then when you're looking at developers, what I'm really excited is, you know, moving into microservices, and kind of more modern cloud first kind of development, right, again, getting everything running on your machine, you know, 10,15,100, microservices, that you're not going to use, you're not going to touch you don't work on but they need to be there. So you know, you have this mesh of microservices, or even just three or four of them, right that work together, you have to have your sign off, you know, your single sign on blah, blah, blah, all these things, and getting databases and queuing and everything running locally. It's hard, right? If I gotta install 'Mongo' and 'SQL' Server locally, and then maybe projects are using 2 different versions, right. So now if everything's in containers, and then if you're using 'compose', which I'll use 'Orchestration' loosely here, but it 'Orchestrates' your containers locally,
26:46 right? Because Docker is all well, and simple and easy. When the goal is I need to fire up this 'Docker Container'. And I put the thing I want to run at the end, you know, I say, 'Docker Run', you know, container, something. But if I want to run multiple things, and they've got to discover each other on the network, they've got to have DNS names. And this one has to start before that one, right. All of a sudden, it's a whole, a whole another level of like, manual challenges to do that by hand. So 'Docker Compose' is basically a way to write a file that takes a bunch of Docker containers, and starts them the right way with the right ports and everything. Yeah,
27:21 exactly, exactly. While it gives you kind of built in discoverability. So DNS, you can easily plug in volumes and share volumes across your containers. Yeah, super powerful. And it's also a way to, like he said, writing out all these 'Docker Run' commands, and then your you know, your run command is huge. You can put that all into a 'Compose file', and then 'Control version' control like that. Yeah, yeah. And then once I exactly my goal I could set up and then you know, setting up a new 'Dev', hey, you know, pull latest from GitHub and 'Docker Compose' up. And you know, it brings your application,
27:53 I've got a couple ideas here that I thought might be fun to bounce off you. And the very first one on the list is onboarding new developers. Yeah. I mean, it's one thing to say, well, we're going to create a project, I want to work on it, but almost no one starts jobs with brand new projects, they start jobs with, here's what we've been working on for four years. It's kind of clunky, but it works. Go right.
28:13 100% 100% and I wrote I wrote a blog post about this, but and then my mentees right, a lot of them come out of boot camps. And you know, they're used to right click New Project, right. And like, that's not where you're gonna go, right, you're gonna jump into a code base. That is not the latest and greatest react, you know, functional components, or you're gonna have old stuff, and you got to be productive. It's a huge, I mean, I'm sure the audience will know, be great to hear the last time you joined the 'Dev team', how long it took you to just get your computer set up and get your environment set up and get familiar, right? It? It's a week, at least a week, if not two weeks.
28:45 Yeah. Or your your machine dies. Yeah. And you're like, well, now I got to get it all set up. Again. I haven't done that for a year and a half. So I forgot how to do that. Right?
28:52 Yeah, I'm trying to get everything into 'Containers', as much as possible. Run everything in 'Containers'. When you get into the GUI apps, it gets a little fun. But yeah, if you're running everything 'Containers', it makes it super simple to be able to move those onboard new devs Yeah, 100% 100%.
29:06 So another one of those that I think we kind of already touched on more or less 'Reliable Development', like everybody's working literally on the same system, and you don't have the works on my machine certification thing.
29:17 Yeah. 100%. Another one, though, is,
29:18 you know, I'm here on my Mac. But maybe I want to do things that are as close as possible to where I deploy my various stuff, things, which is a 'UBUNTU' right. And Mac is kind of like a 'UBUNTU' I mean, it's got sort of a Unix flavor to it, but it's not a 'UBUNTU' itself. It's not exactly that right. And so having this ability to be closer to the environment that you're going to run at, or you're going to deploy to, even though you happen to be developing it on something that's not that
29:49 Yeah, being able to get correct versions, right of everything across your stack is very hard, right? It seems
29:56 simple, but it's not I mean, I could get 'nginx' I could brew install 'nginx' on my machine, which probably have done already, but maybe I've got like the latest, greatest Where's by default over on 'UBUNTU' if I just say, you know, "apt install nginx" I think it's held back a little ways, for some reason, unless you really like configure it for the latest sources. And so, you know, maybe the configuration file has something new that I could do on my machine that's supported. But if I try to put it out there, it's just going to, you know, service failed to start, you know, find a log file somewhere and figure this out. Right? That's, especially when you're new. It's like, I don't even know where the log files are. Why is this happening to me? It's so frustrating. Right? Right. Yeah, you're pulling down a project and the the readme are getting started is compile, right? Make,
30:36 run make, right? Yeah, and unless you are a C, a 'Linux c developer', and you start getting errors, you know, off to Stack Overflow, you go and start randomly trying things, right. And that, as a senior developer, you know, like, I'm so scared of a 'KUBER machine', right, because I'm just changing settings and stuff. And yeah, but if you were doing that all inside of your container, and really a Docker file is the steps you would do to install an application, right? And so they're all they're repeatable, have the correct versions of, you know, libraries all in the image, right?
31:08 I think is a simple preconception to have, like, Docker saves me from knowing Linux, in terms of the commands, I don't need to know Linux, if I'm going to run on Docker, and maybe off the Kubernetes, or whatever. And so I can just do Docker, and then everything will be fine. But in practice, like those steps you just described, those are 'apt install' this, you know, and the various configuration commands of Linux itself. So there, you do need to know a little bit, there's a very small subset that you need to do. But you know, you don't really have to be an admin of Linux. So I do think it takes down the bar. But I still think it's interesting that it's, you still kind of have to know a little bit about how to set up the system. Yeah,
31:45 yeah. It's nice, because you can start high level like, Hey, I just run my, my Mongo image, and it works. And I'm not exactly sure,
31:51 right, I just here's the connection, string or good.
31:53 Yeah, here's the connection string. And then if you start, okay, I want to build my own and I want to get slim as image as possible, right, you're going to, you're going to start diving more in and really the skill set to build your images is you have to understand Docker, but then you also have to understand Linux commands, right? How to install things on Linux, configure them, for sure. But you don't have to your point, right, you don't have to start there. Like, don't become a super, you know, lab admin, you know,
32:16 get the Linux command line Bible or whatever. And like, right, or tear through it,
32:21 right? Yeah, I think it is. You can go very far with without knowing that just basic Linux commands. Yeah,
32:26 a few examples of like, these are the three things you really got to do.
32:29 Yeah. And the nice thing too, about official images that you can, they're stored in GitHub too. So if you go into hub and see it, you can jump into the GitHub repo and actually read the Docker file. So it's great way to kind of kind of learn write of best practices and those type of things. Yeah, absolutely.
32:45 You talked about building like thin images and whatnot. One of the interesting things is taking these Docker files and layering them, right, like, sort of just saying, I'm gonna, I'm gonna need 'nginx'. And I'm gonna need 'Python', I'm gonna need all this. So let's just build one that has all that. And any minor change you make to the dockerfile means you completely rebuild all of that stuff. And maybe part of that's compiled from source that takes two minutes. And it's like, why is it so slow? Right? You could make that four or five Docker files, that one depends on the other that depends on the other and very subtle, small way. So like, only the very last bit that changes maybe really requires much work, right? You want to speak to that a little?
33:22 Yeah, so you can do two things in there, I think you want to make sure one of the best practices you want to do is only one run one process inside of a container. And I'm gonna I'm gonna I use container image sometimes interchangeably. So I navigate, I get yelled at for the peoples on, I'm probably
33:38 making it wrong. If I'm an image, I believe, yeah,
33:41 no, and you're totally fine, right? Because you can use them interchangeably. But so container is just a running image, right. And if you're computer scientists or object or a programmer, it's your class and your objects, your classes, your image, and then your object is your container. So containers are instance of running. And I use them interchangeably all the time. And I get yelled at and one of my talks but but generally speaking, one process running inside of your container, you know, it's the old Linux idea separation concerns, do one thing do really well, microservices outside of things, but then to be able to create your images as small as possible. Alright, you can do multistage builds is what we call it. So you can have a stage in your Docker file that builds your app that has all your compilers, your libraries, everything you need to build your application, make that one part of your Docker file. And then below that, you can reference that and so like a lot of 'Golang' stuff, and we do this internally, is we'll use Docker to build all our apps multi architectural to architect and then below that just pulled out the compiled binary and put that in my image. So now your image is extremely small. It only has your executable in it and everything else that you compiler tools, GCC, all that type of stuff is left out. Oh, that's
34:49 interesting. Yeah.
34:50 So those are kind of like throw away containers. It'll do everything in container, get your nice Dev build environment, build it, and then pull only what you need out. So even like in a node world, where you're not It's not a compile compiled language or pythons compiled, not a compiled language, right, you can do everything you need an earlier stage, and then pull that out just what you need into the final stage to be able to serve those up in your runtime. That's it. And that's the best way to kind of get the smallest lightest images you want. And the smaller and lightest that, you know, you can get them, then they're they're easier to pass around, they're easier to start. They're faster startup and the attack surface from a security perspective is very small.
35:27 Yeah, less is better. They're less is better. Yeah, absolutely. Alright, some other areas that I think too, more than are certainly related, but not exactly the same. I feel like the data science side has a lot to gain from the whole 'container' stuff, because so often, the environments are multi technology, you know, you might have Oh, we're gonna use this library. And it depends on 'Fortran', we use this library, it depends on 'RUST'. And we're gonna control that from a 'Jupyter Notebook'. And it's, you know, it's different environments make that harder, easier to set up. And so I feel like data science has a lot to benefit from here as well,
36:01 I think so too. And that's not my world, day to day for sure. So I gave a talk at the Toronto 'Machine Learning Group' a while back, and the talks should be online. So if you're interested, check that out. And when I was step back, and really thought about what I wanted to talk about, how is Docker really helpful in these scenarios? And it's kind of all the things we've been talking about. It's, it's how do I share my dataset with other data scientists, right, and you can put those images and you can put those in volumes and share those right? And then how do I get a consistent environment? You know, reproducibility is the biggest thing in science. Right? Right.
36:35 That's the other half of that that I was hinting at. It's right.
36:37 Yeah, yeah. And so having that reproducibility that I can share with other scientists, and you have everything packaged together that I was running, right, and you might swap out the data with your own data, but but the process is all the same. So I think reproducibility is huge. And talking with those folks and machine learning, folks. That's what they were looking for. Right is how do I pass this around? How do I share this? How do I get my environment up quickly, without having to become a full blown computer scientist, right? a software engineer,
37:03 or an admin or something? 'admin' And yeah, one of the challenges in science in general, I believe lately, is there's been a what you call a crisis of reproducibility, but like a real focus on reproducibility. And because of that, I think some folks have realized certain studies are very hard to reproduce. And it used to be what they would publish is, here's a graph I generated with my software. And here's the analysis of the graph like, well, that can mean anything, I can make a graph out of anything. That doesn't mean very much, right? And so then the move to like 'Jupyter, notebooks' and 'IPython', and so on to say, Well, here's the code. And here's the graph. And here's the explanation. And that's fantastic. But what version of the library that maybe had a small interior change that no one sees that might, you know, propagate some change down there. And do you can solidify all that stuff into a Docker container. And then say, this is exactly what we ran, at least, maybe it was wrong. Maybe was right. But this is what we did. Yeah.
37:56 Yeah. Just even a minor tick in a version of a, an algorithm that's processing the data, right. And I'm one tick behind you. And well, that's not the same, right. It's not the same know.
38:06 It might be okay, but we need to know that. That's how the differences Yeah, one thing that's interesting, have you seen this project? Gigantism? No, I haven't. Okay, so this one I had this guy's on the show back on episode 238. And it's really all about 'Docker'. It's super interesting. Oh, cool, too. It's like a collaborative way for building like, 'local environments' in 'Docker', but then sharing those, and then people can work on them. And they can like publish them to the cloud. And there's like, all these cool activity streams. And it's just all about 'Orchestrating' data science with facilitating that by doing Docker, they have interesting things like if you go in and open up like a terminal, and you type some commands, I believe they try to like capture those and make those part of what has to be done to the Docker container, even though you didn't change the files that they're trying to do a lot of stuff to make it like really the same. So I don't know. It's an interesting idea.
38:52 Oh, that's interesting. I'm gonna go check it out. Yeah, jumping back to our earlier conversation, build that 'Dockerfile'. Right. Is is any, especially when you're learning commands, right, that build debug cycle tests, cycles can be painful, a little bit? Yeah. So we're trying to think about that at Docker, right? Like, it's one thing to have an image and just run it and use it. And then it's another to build build an image, right, a very complex image and kind of so what you end up doing is kind of what you say you create a base image and then you're inside of that running commands and they go crap, okay, what did I run there?
39:20 example write that down. You know, it kind of love to reverse sometimes that thing I just did to make this work. Make that part of the Docker file, you know, right. extractor. I did those five things. I typed on the command, the terminal command prompt when I logged in with 'DockerRun', you know, 'z shell' or 'bash'.
39:39 Yeah, I need that too. Yeah. 100% 100%.
39:41 Yeah. Advice. Do you got advice on that? I mean,
39:43 it's basically saving at each point right now. It's very low fidelity, right? You're, you're running the command and trying to remember right, and it's painful. Yeah, yeah, it's very painful. What
39:54 some up arrow action like okay, I did that. Then down arrow, down, arrow down. Okay. Those are the things something like that.
39:59 Yeah. Yeah, we've been playing around with some interactive, lightweight kind of essay editors, but tools right to, to build your images. So the concept of building an image from scratch, it's where you kind of you can do a 'Dockerfile'. Say from, you know, "UBUNTU", from 'Alpine', what you can do from scratch, and that's literally nothing right? And you're moving everything in yourself. And that's tough, but it's if you the most secure and cleanest image, that's the way to do it. And yeah, so we've been thinking about tools around that. And it's exactly that capturing what you're doing and what works, right. So you can play a little bit and take try and figure things out. And then once you figure it out, you're like, Oh, crap, what, what was that sequence? That just worked, right? Because I did five things. But those three things were what actually worked. And that could be tough for sure.
40:44 Yeah, I find that what I do is all file, I'll try to create the 'Docker file', it'll invariably fail in some way that I didn't predict. So then I'll get it as far as I can run it in an interactive mode and try to, you know, get it to go to where I want it and then go, Okay, well, these are the steps. Maybe it errored out during the build, but you couldn't figure out why. So you want to go explore something to see what the log generated for that failure? Right? There's a little bit of back and forth. Yes.
41:08 100% 100%? Yeah.
41:10 What do you think about things like attaching basically using Docker within some of these tools like 'PyCharm' and 'VS Code', both have these ways to say, just run this, but in Docker, when I press 'Debug'?
41:23 Yeah, my old school developer, you know, no fancy editors, or a text editor, you know, it dies a little bit inside. But I've had to get over that. Right? Like, no. Right? Like, I tell my mentees to I'm like, okay, when we start like you're in a text editor, remove all remove all the tools because you got to feel the pain to understand what helps. Yeah, sure. But with that said, right, yeah, I mean, the tools are fantastic, right? The context switching, right? If you can stay right in your ID, your main tool and do everything within there and do it really well. It's super powerful. The 'VS Code' plugin is super powerful, right of managing our 'containers', running 'containers', 'Dev containers' connecting into 'ACI' and 'Azure' and launching your 'Containers'. Right? They're extremely powerful. Yeah,
42:08 yeah, I feel like it's gonna get to the point where you might be running stuff in Docker, but you don't even know it. Yeah, you might conceptually know. But there's nothing about what you do or what you see that makes you feel like oh, it's running in Docker. You just I pressed run it did these things. I saw it in my browser, or I saw the output. Yeah. I don't know where that happened. Right. But it happened in Docker.
42:25 Exactly. Yeah, it just works. Right? Yeah, that's where we're trying to go is like with our tools is like you The more you know, your they get out of the way, right? They're not in your ways. Again, maybe the best way to say it. Yeah, really good tools, you kind of Oh, you kind of go does have really doing a lot for me. And then when you figure out you're like, Oh, yeah, it's doing a ton. I just don't know what to do. It doesn't so well. Right,
42:45 exactly. I think we have time for a few more topics that we can touch on. So we talked about the 'Multi Container stuff'. And we talked about, you know, run this on my cloud. And I feel like a lot of times what that means these days is Kubernetes. Right? Everybody seems to have a hosted Kubernetes offering. Sometimes that means just give us your container. Others that means well, we'll really just run the cluster sort of configure it for you. But you kind of got to still own those VMs that run it. There's some sort of spectrum there. What are your thoughts on people working with 'Kubernetes' versus just directly with 'Docker' itself? Or maybe even a 'Pass'? Like almost a serverless type thing?
43:20 If you spent a lot of time on the internet? Right? We're all on the bleeding edge. Right? You really are. Yeah, yeah. You read a lot of blog posts about 'Kubernetes' and 'Service Meshes' and 'Serverless'. And those are all fantastic and wonderful, right? But a lot of us out there are not we're not the Googles or Netflix or, you know, Disney plus, right, you can see I watch a lot of movies, but anyways, you know, so you have small teams, small three, five person teams that really are writing the majority of the software out there, right. And there's nothing wrong with standing a VM up putting a Docker engine in it and running your containers like that, right. That's the way we started. It's extremely powerful. You know, before I came to Docker had a consulting business, a 'Custom Dev'. And that's how we ran all over our containers, right? We had a Docker engine installed on the VM, sometimes two 'VM's, right, that we managed as the 'Orchestrator'. And it ran fantastic. And we could scale we had load, right? It was in the update it and to maintain it, all the things we kind of been talking about today. were super simple, because they The only prerequisite for the VM was that the engine installed.
44:24 Yeah. And to be honest, that's how my stuff is running. Now I have a 'CI/CD' stuff set up. So I do get pushed to a certain branch, my VM's automatically work with the 'Load Balancer', they automatically update themselves, they restart and read like it's, it's hands off, right. And you know, the small, small set of us like it works.
44:41 Yeah, 100% 100% and serverless. And then also kind of the 'Container' 'PaSS' that are out there. So ECS ACI, so ECS is a 'Elastic Container Service' from Amazon 'ACI as as your Container instances'. And I don't know if Google has one they probably do too. Yeah. Love you 'GOOGLE'. But yeah, anyways, you know, it's being able to take your container and just run it in the cloud and worry about the container running and what it needs to do and how it needs to run right and ECS will take care of the rest underneath the underneath the covers, that's a fantastic option. I mean, it's a, it's kind of, you know, it's like those progressions, right? If you're doing a single VM with the Docker engine, or or two of them, a couple of them. And that becomes painful when you need to scale really look at ECS or ACI, right? Because that will help you scale a lot easier. And you don't need to be a Kubernetes expert. And then if you do the right thing, then go to Kubernetes. But I always tell people, and 'Swarm' is a great option to have 'Swarm' is an incredible 'Orchestrator' it work with
45:38 how does that compare to Kubernetes? You know, I hear 'Docker Compose', I hear 'Swarm' I hear 'Kubernetes'? I'm not sure when I should be thinking about which
45:46 so 'Compose' is fantastic for local development. I would use it primarily there, you can use 'Composer' to deploy into production.
45:52 Would I use compose if I had only a single container I wanted to run? But maybe it had a complicated, yes. And layering, potentially.
46:00 Yeah, so compose works great there to even just even if you're running one container, and all the command line switches and flags and environment variables, and secrets, all those type of things, like put them all in your 'Compose file'. So one, you don't have to remember all of them. And two, you don't write 'Bash', 'Bash scripts' to just run the Docker command. Right? You plug them all in just like that. You can see your 'PasS' imports, they have volumes, all those type of things, right. Yeah, you know, and I tell people start, think about 'Swarm', right? If you're doing one instance, don't jump to, like, have really good reasons to jump into Kubernetes. It's super power, powerful. It's a great tool. It really is. But it comes with a ton of complexity, right? So you don't need to go there's like
46:39 I said before, there's a difference between I want to run this on 'Kubernetes'. And I want to have my own 'Kubernetes Cluster' for which I'm the 'Admin'. Right, right. Yeah.
46:46 And don't get stuck in I also know advice in some engineering, like don't get focused on developing for your resume, right. A lot of times you Oh, we got to get in Kubernetes don't have Kubernetes I've done all this right. And and yeah, you know, you want to learn new technology and stay cutting edge. But it that's a tough one. That's a tough one to go to. Yeah, you can get very far with single engines with 'Compose'. And then with 'Swarm' and 'Swarm' is a great 'Orchestrator'. It's built into the engine, it moved to the enterprise with 'Mirantis' when they bought the Docker enterprise business, so they maintain it we got out of Docker proper has gotten out of the 'Orchestration' business, but it's a fantastic 'Orchestrator'. It just works a prior life at Docker, I ran all the customer success technology. So our knowledge base or search engine, our training, tools and everything, and we ran on 'Compose', 'Compose' in 'Swarm' on 'Swarm' five, nine uptime, never touched, it didn't have a DevOps team, right? It just, it really works. When you need things like side cars, and more complex networking, and the more complex data storage volumes, those type of things, that's where Kubernetes really shines. But other than that, you know, 'Swarm' is fantastic in single engines will get you very far. And then like you said, I would go to 'EKS' or something like that first, right? Have a 'Kubernetes Cluster' and manage at the level that you need to manage that and nothing lower right, until you have access.
48:05 Chris Manzano creator and the creators of 'Pyramid framework' out there is in the live stream says 'Compose' kind of fills the same space as bill that did in Python land for managing one or multiple containers locally. Yeah. Awesome.
48:16 Hello, Chris. Hey, Chris. Yeah,
48:18 very cool. So the last thing that I want to touch on the last topic I think we have time to talk about is the future where things going yeah, with Docker, right, like we've seen this move to focusing more on you said, shift the left more in the developer tools and whatnot.
48:32 I'm glad you brought up the roadmap. Thank you. My bosses will be happy. My my Masters, anyways, love you, Matt. But so get out there for four /Tiger forward /roadmap. That is our public roadmap. That is our roadmap, right? We work on that roadmap, we're actually coming out with a blog post with a feature and that was spawned came out of the roadmap, right? It was an idea from the community. It's integrating in with different terminals with 'Docker Desktop'. And that came out of the community. And people said, Hey, I run 'iTerm' on my Mac and set is 99% of the other, you know, developers on Mac, we want that integrated our integrated desktop, and so that the optics were were really great. Anyways, we didn't implement it. So. So you'll see Ben, if you're looking at the screen on the right, Ben is the best PM I've ever worked with. Seriously. He's fantastic. But he is in there constantly. Yes. So we look at this roadmap, we follow this roadmap, we build off of this roadmap, we don't have a secret shadow internal roadmap,
49:30 right? And we say hey, yeah, different channels for like Windows and Microsoft and the engine and all that stuff, right?
49:36 Yep. And so go through, read through there put in you know, put in an issue if you have one. If you don't find it. It's where you can interact, add comments, those type of things. We do use that constantly. You know, I don't so we have 'Docker con' coming up so I don't want to give any secrets away. But yeah, we have. I would say look for us in this year, the past year we really kind of solidified the business right really did a lot of things that Docker didn't do early in its early days, right? As a business wise, you were shifting around a lot, but focus back on 'hub' and say, Hey, you know, let's tighten up 'hub'. This is a place where massive amount of developers come for value. So we ended up putting good features like law, audit, logging, and more team features and fixed our plans and seats and how the pricing and how that works, and then started moving towards developer tools. So integrating with 'ECS' 'ACI' into the public clouds, you'll see a lot more of that coming, you see a lot more tools around local development, 'Container First Development', a lot of sharing. So be interesting to say I'm working on a feature, I have a branch, and I want to share that with you, Michael, hey, you know, take a look at this real quick. And now you're in that same scenario, right? I have to check out your branch. If I'm doing everything on my machine. Do I have all the versions correct? Do I have everything right? But what if you could just wrap that all up a 'Container', and I just share it with you. And you fire that 'Container' up, and you're in a Dev environment and it not just running the 'Container' but able to see my code and everything and the app running inside the 'Container' and interact with it? Right?
51:00 Yeah, that's cool. That's powerful things a little bit like the VS Code feature where they have that's like, launch this in code off of 'GitHub'. Yep. And it just like sets up that remote environment.
51:09 Exactly. Exactly. Yep. 'Launch' launches for me. And also, too, I can notify you, right? So you get in a little desktop notify, Hey, you got something shared with you in 'Launches' and VS Code, and it brings it all in for you. Yeah, cool. into your earlier point. You go, oh, if you didn't know you're running containers, you might not know it, right? It's like, Oh, I just got this, this PR shared with me. And I want to check it out. Right. And it's just the Run button. And it's good as I push the Run button. Yeah. Yeah. Awesome. A lot of stuff coming around that a lot of more features in for teams and sharing features coming in home, we have scanning in the security scanning in there now.
51:42 Yeah, a lot of interesting things to GitHub added was notifications of security, 'CVES' and whatnot. Like, periodically, I'll log in, and I did this class that interacted in a very minor way with an 'Electron JS app'. And it's always like, your repository has this critical vulnerability, because some random version in a package 'log.JSON' is, you know, like, so for me, I'm like, I don't really need that. But I'll go ahead and just accept it, because I just need it to be quiet to me. But yes, in general, those features are super useful when those come up for your actual code. Yeah, anything like that you guys are thinking about.
52:17 So we partner with 'Snyk', which is a dev security scanning company, security company, just scanning because they do a bunch of stuff. But now you have 'Docker space scan'. So you can scan your images locally, get reports see high or nice low severity. Yeah. And then that'll run in hub. So when you push into hub, your images will be scanned, if you have a turned on. And so you have a central place where you can see all those. And then you can use 'Snyk' to remediate them and and those type of stuff and 'Snyk' has really good times where because
52:45 maybe you you scan even you could scan locally, push your stuff to production, right, and then you haven't touched it for three weeks. Right? In the meantime, something's come out, you need to take action on. Right. And so having an on the hub giving you notifications is constantly running. Right?
52:58 Exactly, exactly. Yeah, being able to get notified of your base images, right. So if you run in, denoted a Python base image, right, when that changes underneath you, right, and there's security vulnerabilities introduced, you want to know, right, should I go to the next version or not? And then I think to to your earlier point of, you know, it's like, I'm not a security professional, right? So I see these reports, and I do a scan and I go, oh, heck, you know, 300, you know, high severity vulnerabilities, right? What do I do? And I think what we're trying to do is say, Hey, here's how you remediate that, right? You're on version 3.2. If you bump up to 3.3, it fixes the so you should do that, right? A little minor version upgrade, or major upgrade takes you to instead of 300. Now you have 20 vulnerabilities right. And they're all medium. So yeah, I think we need that as engineers to be able to get our job done quicker writing code, you know, it's a primary function, right? So I got a bunch of these vulnerabilities into your point, you kind of like just Yes, just clear these red X's off of here and right, we might not know exactly what's going on. So really trying to help you remediate those vulnerabilities. But also understand why and what you're doing right now, that
54:01 sounds really valuable. We got it for our code, we should have it for our basically our infrastructure coming from the live stream to Docker, potentially be able to share really big files across the globe, allowing others to recode easily. How would this work? Honestly, I don't know. I mean, it certainly will allow you to share files as images. And it will definitely allow you to have environments set up like data science environments that could analyze those, but what are your thoughts?
54:24 I think we could do some of that now. So I'm through hub, bam, just 'Docker push'. And 'Docker pull' is very powerful, right? You can get a whole disk image, let's say, right, right.
54:33 Because instead of getting just the 'Docker file', you got to build you get the result, which could have had files copied and locally and all set up in the file system internally, right? Yeah.
54:40 Yeah. Yeah. So two ways to share with with just the Docker file and say, Hey, build this yourself, or I've built it. Here you go, right. Yeah. I would love to hear, you know, reach out to me on Twitter or something. Right. And let me know I'd love to hear more of what you're trying to do. And because that sounds interesting, right? Yeah. 'Bandwidth' is interesting to around the world, right? So you know, I sit here in in Austin, where You know, high speed internet? Well, really the rest of the world doesn't have that. Right. So low bandwidth. And you know, of course, I got a screening desktop laptop, right that I use in fast and not, you know, the whole world doesn't have those advantages. So, you know, and if I'm pulling down a Mongo, some image, right, and I'm picking on Mongo all the time, but you know, if it's if it's over a gig, that's, that's big, right, that It'll take a while on some network. So yeah,
55:23 yeah. All the video and video courses and audio processing and file exchange, I had to buy unlimited data from 'Comcast', because I was going over my 1.2 terabyte per month data cap every month I was predicting Yeah, yeah. So a lot, a lot of data these days. All right. Well, you know, fantastic stuff you guys have coming up there. I guess maybe we should probably wrap it up just for the sake of time. But yeah, I guess we'll we'll hit the the two final questions I always ask. First of all notable PyPi package. This is like something interesting that you ran across, maybe you wanted to share people that find useful some Python library,
55:57 I love testing. So 'test containers.python', that's a little inside joke that no one on here really knows. But I have I have developer friends. So I will. One of my favorites. Things that argue against is testing just as a thought experiment. It's really fun, right? But anyways, but don't listen to me testing is super important, right? And being able to test and containers and pick off these pre built containers and use right in your in your app right in your code is super powerful, extremely powerful. So definitely interested in test containers.
56:25 Yeah, it's super cool. It lets you basically create a width block that says, with MySQL container as 'MySQL' or with my 'MongoDB' in dB container as that and it just fires up a Docker container, you do your tests within that thing, and maybe use a 'PI test fixture' to preload the test data, then run your code against it do instead of mocking out the database and mocking out 'Redis'. You just run a test 'Redis' and then throw it away, which I think that's fantastic. It's like a hybrid between the integration tests and unit tests, isolation type stuff goes back to our earlier conversations around reproducibility, right. Yeah, having same environments are very important, right? Yeah, absolutely. And then final question. If you're gonna write some code, what editor Do you use these days?
57:04 If we were to ask me that about two years ago, it would have been 'Sublime', but plain text editor, you know, back and forth, all tabbing backwards through my terminal. But now I fell in love with 'VS Code'. Yeah, it's extremely powerful. So I feel
57:16 like 'Sublime', like so many of the people who love 'Sublime' have just naturally moved to 'VSCode'. I mean, 'Sublime' was really cool. I just had all these like, weird community stuff, like there was 'Sublime'. And then 'Sublime3', and there was permanently in beta. And like, are these getting updated, which one is like the real thing? It's just it was always kind of a little unclear where it was going. And then, you know, 'VSCode', came along with a similar model and just said, Well, why don't we put 100 people just like constantly adding features to this? Yeah, exactly. I don't know what that number is. But it's high.
57:42 Yeah. And it's interesting. You say that, because I was a longtime supply? supply. I bought it right and purchased it forever. And yeah, it was like 2 or 3. And I have a theme over here, but it doesn't work in three. And how do you get it was just weird in 'VS Code' just kind of snuck in. And I said, Yeah, I'm gonna use it. I'm gonna use it. And now I keep just using, I use it as a text editor back then. Right. And now just keep using more and more features, plugins. I mean, the ecosystem around that is so powerful.
58:06 There's neat stuff, and especially there's a lot of neat Docker things going on in it.
58:09 Yeah, I'm gonna run my 'VS Code' in a 'Docker' and then run, you know, 'Dev Containers', and
58:14 it'll be like Inception VS Code all the way down. All right, I'll let you got a moment for a couple more audience questions before we wrap it up?
58:22 Absolutely. You know, man, I'll chat away for days, but I know your audience might get bored.
58:28 Alright, so Brian sands out there. Hey, Brian says Do you know of any good examples using 'Docker Compose' with 'Postgres' for persistent Local Storage? I have a hard time finding up to date working info to do this properly.
58:39 Yeah, checkout 'Awesome Compose'. So it's a 'GitHub' repo. Docker owns it. So awesome. That 'Awesome-compose'. I can't remember if it lives on the Docker organization or not. But there's a ton of different kind of configurations with different stats. super
58:53 interesting. I had no idea about this question, Brian. Yeah, thank
58:56 you, Brad. I paid him. I'll send you that. I'll then mail you here in a minute. No, but yeah, check that out. If you can't get your specific stack, you're gonna get something very close, right. So you get there's 'Flask' in there. There's, there's 'nginx' like 'Flask' and 'Postgres' different kinds of three tiered stacks. And then within that, you'll have 'Docker files', and then you'll have a 'Compose file' to kind of bring it all together. And you should get some really great examples in there. If not, please do hit me up on Twitter, and I'll get you pointed in the right direction. For sure. Yeah,
59:24 very cool. Awesome. And then, you know, ask, what is the current state of Windows based Docker containers?
59:30 I know there's nanostone gonna tread lightly on this one, you know, it was super excited when Microsoft when we started supporting and then for Windows containers, you know, the focus has been not as much you know, the the amount of people actually using Windows containers is very, very small compared to Unix based containers. So you know, just because of that the focus is not that on that there's still thing they're still being developed on. They're just not as prominent in the industry for whatever reasons, especially with
59:56 oil, you know, where I would say they probably as is standard With Microsoft these days, their focus seems to be if it runs on 'Azure'. Yeah, we love it. Right? This is where we will sacrifice many, many of the things so that it will run on 'Azure'. And a lot of that I believe is check this thing in Windows and like, say Visual Studio code, so I can push deploy to run on 'Docker' on 'Azure'. Yep. I think that's probably where a lot of it is.
01:00:21 Yeah. And 'WSL2' is huge. Right? So talking to the Microsoft folks, right? They want to be the place where you come the Windows platform is the place you come for developing, right? So whether you're windows developer, or Linux developer, right, you're using VS Code, you're running on Windows, you're using WSL2 for Unix development, or using Windows for Windows development. That's their thought, right? So the underlying platforms kind of change. And with containers, they can do some of that. So they're focusing a lot on 'WSL2', which is which is super powerful,
01:00:49 right? Which really just means make the 'WSL2' good. So that it runs on 'UBUNTU' or something like that, right. I mean, just leverage that 'UBUNTU' Docker support, I suspect. Yeah. While we're talking platforms, I've been doing basically nothing but in one stuff these days, since mid December, I thought I would, you know, got my Mac Mini. Yeah, in one and I thought I'd be juggling, this would work here that would work there. And I just literally have just turned off my laptop and just do this. And I love it. What's the story of Docker and in one?
01:01:14 Yeah, so we love 'Apple'. It was nice to see our little. I don't know if you saw on Apple's keynote, you know, we saw the Docker desktop on there, which was super cool. We're getting very, very close to shipping. You know, a release. Okay. A Ga? Yeah, I mean, really close. I don't want to give a date anything, but really, you know, tomorrow.
01:01:32 Yeah, like to me, okay, we'll do that. I'll take it very,
01:01:35 very close to working on it. It's one of our primary feature sets to finish to get done right there. And again, it came from the community, it was our number one issue, right when everyone's job, you know, I
01:01:45 think everybody jumped into our GitHub issue and said, We need some more. All in one stuff has come out way stronger than I expected. I thought, oh, that'd be kind of a neat, who knows what they're doing. But like, as it came out, it's like, wow, this is really informative. In some ways. Yeah.
01:01:57 You can get the tech preview. It's pretty stable right now. You know, it's just gonna be on probably, you know, one off little issues on your environment, how you run containers, and what exactly what container, you know, because they got to be compiled for the different labs got
01:02:11 to basically be an 'ARm' image.
01:02:12 Yeah, yeah. And sometimes you run 'ARM', but then, you know, Mongo doesn't have an 'ARM' version yet. So you're running it in 'Emulation' And you know, so there's those type of things. We're very, very close to releasing GA. Stay tuned. next couple weeks, I would say,
01:02:24 yeah, fantastic. Shout out to the MongoDB. People, where's your M One version? Come on. All right. We're gonna wrap it up. Now, Ricardo says that working on Docker, it feels like it's easier with 'Alexei' one of the other container styles. Am I doing something wrong? I'm interested in giving a container its own IP address. Peter, this you have no idea.
01:02:42 Yeah. Ricardo, I thought I was gonna get out of here without any networking questions. With regard to you are not alone. Networking in containers is difficult, right, You got to deal with, with different networking technologies, even locally. So yeah, it's not easy. Is this easier in something like 'Swarm' or 'Kubernetes'? or something else that maybe controls it more? Now? It gets even more complicated. Okay. Once you step outside of if you have cloud native apps, right? I'm not a huge fan of that term. Because what the heck does that mean? But
01:03:11 right, if you don't want to permanently pay to read my app forever?
01:03:16 Yeah, exactly. But yeah. So if you're microservices, you're not you're using configuration outside of your app to control your app, right is the best. But I see a lot of 'Legacy apps', I'll do 'Legacy' in 'double quotes' also, right, because just meaning not more micro service type apps, right. And so you're trying to run those apps inside of containers, and they have unique networking needs unique volume needs, right? They're still writing to disk and reading from disk. And you can't control that without changing code. Or they have IP addresses or subnets. Kind of hardwired in, right. And you can't do a lot of that. So you got to configure your networking around your app with inside a container. And that's where the complexity comes in. So you know, give like, so in this scenario, given you're giving your container, its own IP address, it's a little bit about against best practices, right? So containers should be familiar. They should go away, right? You should think of them and I got to get a better analogy. My apologies to anybody if this offends you. But you know, we think of our we used to think of our VMs as pets, right? And we take care of them and keep them up and love and love them. And you know, hope they live forever. Right? configure them and then now you have containers, which are more like, let's say, I used to say cattle but I'm worried that offends people. So let's say consumable foods, right? You know, like you get in trouble elements. Yeah, so you get more like cake. Right? If I get cake in my house, I'm eating that. Right. It's gone. And if I want new cake, I gotta put new cake right.
01:04:37 So same goes bad. You don't try to rehabilitate it. You just get new cake.
01:04:40 Right, right. Yeah, you don't eat half the cake and then and then bake. You know, put another bake the half this is an analogy is going poor, but speak for yourself. We
01:04:47 refill our cake. You don't ever want to go down it okay, right. But
01:04:50 that's the idea. So if you're assigning an IP to a container, that 'Container' goes away, right? And the idea is if I'm 'Orchestrating' that I have 'Multiple Containers' we're all running. They shouldn't have the same IP Right, if one goes away, when it comes back up, you don't want to you know, so those are the type of things you're getting into. And Ricardo, you're probably shaking your head going. Yes, but but but but but and I told him that I need it. Yeah, I totally understand. And you can do it, but it's just a little more complex. Yeah. A little hard to answer on, you know, a couple minutes here. But yeah,
01:05:17 hit me up on Twitter. All right. So thank you so much for being here. Peter, super interesting, final call to action. People want to get started maybe making Docker more of their life, what do they do?
01:05:26 Definitely go hit me up on Twitter, say hello, love to talk to folks. Join our community jumping in community slack. If your questions like that, Ricardo, the communities there, there will be very, very helpful. A lot of our engineers, our developers are in there, all of our PMS are in there, ton of captains are in there. Docker captains are in there. And they're very, very helpful. So this is a great way to get plugged in, in the community. And then the last thing is Docker con is coming up here in the may sign up for 'Docker con', it's going to be an online live event. Last year, we're had 80,000 registrants a really, really big show. So you're not gonna want to miss a great content coming, you get to see more of my smiley face. And then CSP is still open, we're getting ready to close it on the 15th. So if you ever thought about giving a talk, right, please submit if you're a first time speaker, don't let that deter you. Right? 'Dockercon' is traditionally a pretty big conference and well attended, right and it might, it might, you know, first time speakers might not submit please do. We do read all of them. And we do take an account. You know what you've done in the past, but we also do look for new speakers right. 100% don't let that deter you for sure.
01:06:27 All right. Fantastic. Well, thanks for taking the time, Peter. It's been great to chat with you, buddy. Yeah.
01:06:30 Hey, thanks for having me on. I really enjoyed it.
01:06:32 You bet. This has been another episode of talk Python to me. Our guest on this episode was Peter McKee. And it's been brought to you by 'Datadog and us' over at Talk Python Training. 'Datadog' gives you visibility into the whole system running your code, visit 'talkpython.fm/data dog' and see what you've been missing. Throw in a free t shirt with your free trial. Want to level up your Python we have one of the largest catalogs of Python video courses over at talk Python. Our content ranges from true beginners to deeply advanced topics like 'Memory' and 'Async'. And best of all, there's not a subscription in sight. Check it out for yourself at training.talk python.fm Be sure to subscribe to the show, open your favorite podcast app and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, the Google Play feed at /play and the direct RSS feed at /rss on talk python.fm. We're live streaming most of our recordings these days. If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talk python.fm/YouTube. This is your host Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code