WEBVTT 00:00:00.001 --> 00:00:05.180 It's time to jump into the brave new world of containers and Docker with Patrick Shanizen. 00:00:05.180 --> 00:00:10.400 This is Talk Python To Me, episode number nine, recorded Monday, May 4th, 2015. 00:00:10.400 --> 00:00:40.380 Hello. 00:00:40.640 --> 00:00:46.200 And welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, 00:00:46.200 --> 00:00:48.020 the ecosystem, and the personalities. 00:00:48.020 --> 00:00:50.060 This is your host, Michael Kennedy. 00:00:50.060 --> 00:00:54.860 Follow me on Twitter, where I'm @mkennedy, and keep up with the show and listen to past 00:00:54.860 --> 00:00:57.220 episodes at talkpythontome.com. 00:00:57.220 --> 00:01:01.240 This episode, we'll be talking with Patrick Shanizen about Docker. 00:01:01.240 --> 00:01:05.400 Before we get to Patrick, I'm really excited to tell you that this episode is brought to 00:01:05.400 --> 00:01:06.260 you by Codeship. 00:01:06.740 --> 00:01:11.440 Codeship is a platform for continuous integration and continuous delivery as a service. 00:01:11.440 --> 00:01:16.220 Please take a moment to check them out at codeship.com or follow them on Twitter, where they're 00:01:16.220 --> 00:01:17.160 at Codeship. 00:01:17.160 --> 00:01:18.680 Now, let me introduce Patrick. 00:01:18.680 --> 00:01:22.320 Patrick Shanizen is a member of the technical staff at Docker Inc. 00:01:22.320 --> 00:01:28.640 He helps build Docker, an open platform for distributed applications for developers and sysadmins. 00:01:28.640 --> 00:01:31.300 Patrick is a software developer and storyteller. 00:01:31.380 --> 00:01:36.960 He has spent 10 years building platforms at Netscape and Sun, 10 more evangelizing platforms 00:01:36.960 --> 00:01:39.120 at Google, VMware, and Microsoft. 00:01:39.120 --> 00:01:44.300 His main professional interest is in building and kickstarting the network effects for these 00:01:44.300 --> 00:01:46.780 wondrous two-sided markets called platforms. 00:01:46.780 --> 00:01:53.400 He worked on platforms for portals, ads, commerce, social, web, distributed apps, and of course, 00:01:53.400 --> 00:01:53.980 the cloud. 00:01:53.980 --> 00:01:56.620 Patrick, welcome to the show. 00:01:56.620 --> 00:01:57.400 Hey, Michael. 00:01:57.400 --> 00:01:58.480 Thanks for having me. 00:01:58.480 --> 00:02:04.160 Hey, I'm really glad that you're here, and I'm super excited to have Docker on the show. 00:02:04.160 --> 00:02:09.720 Ever since I learned about Docker, there's a few times I've learned about some technology, 00:02:09.720 --> 00:02:14.080 and it has completely taken over me, and somehow I'm obsessed to learn it. 00:02:14.080 --> 00:02:19.300 And when I figured, when I came across Docker, when you guys announced it, I don't know, was 00:02:19.300 --> 00:02:20.180 it a year ago? 00:02:20.180 --> 00:02:21.720 Two years, about two years ago, right? 00:02:22.120 --> 00:02:27.460 Yeah, I came across it, and I'm just, oh my gosh, I need to learn this. 00:02:27.460 --> 00:02:29.960 I need to understand this, because this is going to be huge. 00:02:29.960 --> 00:02:34.620 And I really think Docker and these container technologies are going to change the way we 00:02:34.620 --> 00:02:36.400 build and deliver software. 00:02:36.400 --> 00:02:41.360 So let's start with telling the world, what is Docker, for those who don't know? 00:02:41.360 --> 00:02:41.860 Sure. 00:02:42.640 --> 00:02:50.500 So Docker is an open source project that was announced by Solomon Haidt, the founder of 00:02:50.500 --> 00:02:53.720 .cloud and Docker, two years ago at PyCon. 00:02:53.720 --> 00:02:58.940 So it's relevant for Python developers, because it all started at PyCon. 00:02:59.580 --> 00:03:06.400 And so Solomon and his team were building this platform as a service called .cloud, and they 00:03:06.400 --> 00:03:08.260 were struggling with that option. 00:03:08.260 --> 00:03:13.200 It's a time when, at that time I was at Google, I was doing a Google App Engine. 00:03:13.200 --> 00:03:17.260 There was Heroku, who's been bought by Salesforce. 00:03:17.260 --> 00:03:23.960 And so .cloud had trouble attracting developers to their platform as a service, which allowed 00:03:23.960 --> 00:03:27.920 you to deploy applications in PHP, Python, and multiple languages. 00:03:28.540 --> 00:03:33.480 Well, at the time, a platform like Google App Engine, for example, was limited to Python 00:03:33.480 --> 00:03:37.120 and Java as runtimes, and Go as well. 00:03:37.120 --> 00:03:43.760 And so they were struggling with adoption, but they had this gem inside of .cloud that they 00:03:43.760 --> 00:03:49.400 were using to build their platform as a service that was called Docker. 00:03:49.400 --> 00:03:58.400 And that was an easy way to package and distribute, build, ship, and run Linux containers. 00:03:58.400 --> 00:04:03.340 And so they open-sourced the project at PyCon two years ago. 00:04:03.340 --> 00:04:05.980 And then it took the world by storm. 00:04:05.980 --> 00:04:08.720 Like all developers starting adopting it. 00:04:08.720 --> 00:04:14.340 And the whole industry restructured around that notion of Linux container. 00:04:14.340 --> 00:04:25.080 So Docker essentially is a set of tools that allow you to build, ship, and run distributed applications based on Linux containers. 00:04:25.080 --> 00:04:34.020 So there's the Docker command line, which is at the same time, it's a unique binary that has a client and a daemon. 00:04:34.020 --> 00:04:49.840 And the daemon lets you get images for containers on your local machine or on a server and run them in a Linux container, like isolated from other workloads on that machine. 00:04:49.840 --> 00:04:54.780 So you can have your own file system, your own users, your own network namespace. 00:04:54.780 --> 00:04:59.780 And so Docker is based on Linux namespaces and C groups. 00:04:59.780 --> 00:05:17.840 And then you have Docker machine that lets you provision machines running the Docker engine on any cloud provider, like Google, Amazon, Microsoft, but also locally on your machine on VMware, VirtualBox, or Hyper-V. 00:05:17.840 --> 00:05:23.780 So Docker itself, Docker machine to provision the machines. 00:05:23.780 --> 00:05:41.840 Then there's another project called Docker Compose that allows you to assemble all these containers that you're using to build your microservices into a single unit that you can run on your laptop for development or ship to the cloud as a single unit. 00:05:41.840 --> 00:05:47.780 So typically you would have, for your typical Python app, maybe you're using Django or Flask. 00:05:47.780 --> 00:05:53.780 You would have one container that has Django, your Django application with its code in it. 00:05:53.780 --> 00:05:57.780 And you would link that to another container that has Redis on MySQL. 00:05:57.780 --> 00:06:03.780 And then you can start all this with a single command, Docker Compose. 00:06:03.780 --> 00:06:06.660 So you can, with Docker Compose, you can just start. 00:06:06.660 --> 00:06:07.460 Yes, exactly. 00:06:07.460 --> 00:06:11.760 So in Docker Compose, you say, I have my image for my Django app. 00:06:12.120 --> 00:06:18.060 Let's call it like, let's call it like, my Django app. 00:06:18.060 --> 00:06:28.940 I want to launch that with port 80 on the machine, on the server on which it's running, mapped to port 8080, maybe internally, where I'm running it. 00:06:28.940 --> 00:06:35.880 And then I want to tie that to a Redis instance that's using the Docker image Redis that's on Docker Hub. 00:06:35.880 --> 00:06:45.560 And then inside of my code, I'm just going to refer to Redis and automatically it will just tie that to the other container that's running. 00:06:53.560 --> 00:06:59.580 CodeChip is a hosted continuous delivery service focused on speed, security, and customizability. 00:06:59.580 --> 00:07:05.420 You can set up continuous integration in a matter of seconds and automatically deploy when your tests have passed. 00:07:05.420 --> 00:07:08.660 CodeChip supports your GitHub and Bitbucket projects. 00:07:08.660 --> 00:07:11.440 You can get started with CodeChip's free plan today. 00:07:11.440 --> 00:07:19.920 Should you decide to go with a premium plan, Talk Python listeners can save 20% off any plan for the next three months by using the code TALKPython. 00:07:19.920 --> 00:07:21.660 All caps, no spaces. 00:07:22.420 --> 00:07:29.000 Check them out at CodeChip.com and tell them thanks for sponsoring the show on Twitter where they're at CodeChip. 00:07:29.000 --> 00:07:48.000 So before we get into the details of how we run all that, you know, it's interesting that Docker kind of grew out of what was supposed to be kind of a larger project, right? 00:07:48.000 --> 00:07:48.360 Yeah. 00:07:48.360 --> 00:07:50.620 A platform as a service company, if you will. 00:07:50.620 --> 00:07:55.920 It was just, it was just, that was how you guys were trying to internally achieve greater density for your hosting. 00:07:55.920 --> 00:07:57.780 Yeah, that's exactly right. 00:07:57.780 --> 00:08:03.600 And the notion of running Linux containers has been around for a really long time. 00:08:03.600 --> 00:08:09.460 Or actually containers themselves have been around for a long time with three BSD gels. 00:08:10.300 --> 00:08:18.020 In the, like, 2005, there was already the notion of Solaris containers in Solaris. 00:08:18.020 --> 00:08:21.680 So that notion of containers existing for a long time. 00:08:21.680 --> 00:08:36.220 And that's the way Google is running its workloads inside of Google data centers, using Linux containers for isolation in order to be able to run workloads with a very high density, 00:08:36.220 --> 00:08:43.220 because it's using Linux or operating system virtualization. 00:08:43.220 --> 00:08:53.300 So that's a super important difference or distinction to make between these containers and virtual machines. 00:08:53.300 --> 00:09:02.400 So if I create a virtual machine, maybe it's a couple of gigs for the file system and the operating system and all my data and everything I've done to it. 00:09:02.400 --> 00:09:05.360 Maybe it uses two gigs of RAM when I start and stop it. 00:09:05.360 --> 00:09:07.740 It's got to boot up like a regular machine. 00:09:07.740 --> 00:09:10.320 You know, it's a big blocky thing. 00:09:10.480 --> 00:09:11.740 How big are these Docker containers? 00:09:11.740 --> 00:09:12.720 Yeah, they can. 00:09:12.720 --> 00:09:16.460 So Docker containers compared to virtual machines can be much smaller. 00:09:16.460 --> 00:09:21.180 They can be big as well, but usually they're much smaller. 00:09:22.280 --> 00:09:32.380 And another advantage they have over VMs is that the image format for Docker images, they are layered. 00:09:32.380 --> 00:09:41.060 So you build a layer, you can say, oh, I'm going to inherit from the Ubuntu layer, and then I'm going to install Python on that. 00:09:41.280 --> 00:09:45.840 And then I'm going to use pip to install my requirements on that. 00:09:45.840 --> 00:09:55.500 Each of these instructions generates a different layer, and when you're using the Docker pool to pull the image or Docker run to run that image, 00:09:55.500 --> 00:10:02.180 what it's doing is that it's looking in its local cache to see, oh, hey, I have Ubuntu already in there. 00:10:02.180 --> 00:10:09.160 So I'm just going to download the image that adds Python to it and then the one with the requirements for my app. 00:10:09.900 --> 00:10:15.320 So there's an additional density in terms of storage as well that goes with it. 00:10:15.320 --> 00:10:26.160 While when you're packaging your app for a VM, you're just building that monolithic VM image with everything from the version of Linux or Windows that you're running in there, 00:10:26.160 --> 00:10:35.060 the full operating system, plus your application code and packaging, and every VM you start has all these data. 00:10:35.540 --> 00:10:43.780 While if you're running like 10 different Python applications package as a container on one host, 00:10:43.780 --> 00:10:53.600 you would have one time the data for the Ubuntu VM, that's your base, and then maybe one layer for Python, 00:10:53.600 --> 00:11:00.640 and then small layers 10 times for the requirements and the code base for your Python application. 00:11:00.960 --> 00:11:02.960 So the images are lighter. 00:11:02.960 --> 00:11:05.800 And what that means is that they download much faster. 00:11:05.800 --> 00:11:09.180 Another aspect is that at runtime, it's much faster. 00:11:09.180 --> 00:11:15.100 So starting a container, once your image is cached locally, just take like a few milliseconds. 00:11:15.960 --> 00:11:23.860 While, as you know, to start a VM, it takes a good 20 to 30 seconds in good cases. 00:11:24.420 --> 00:11:25.960 Yeah, that's really amazing. 00:11:25.960 --> 00:11:28.560 And I like the layering concept. 00:11:28.560 --> 00:11:36.440 So every layer you add brings you closer and closer to the specific type of operating system and environment you need for your app. 00:11:36.440 --> 00:11:40.120 So at first, it's just Linux, and that could be anything. 00:11:40.120 --> 00:11:40.340 Yeah. 00:11:40.340 --> 00:11:43.140 And then it's Linux plus your version of Python. 00:11:43.140 --> 00:11:44.200 And so it's a little closer. 00:11:44.460 --> 00:11:47.320 And then maybe it's that plus Nginx. 00:11:47.320 --> 00:11:49.500 And then it's a little closer. 00:11:49.500 --> 00:11:56.360 And then it's that version of Linux plus Python plus Nginx plus all the packages your web app depends upon. 00:11:56.360 --> 00:11:57.740 And then finally, maybe it's your code. 00:11:57.740 --> 00:12:04.180 And so you can use every one of those layers as a building block to create new ones and things like that, right? 00:12:04.220 --> 00:12:05.500 Yeah, it's very composable. 00:12:05.500 --> 00:12:17.300 So once you have created your base layer, anybody in your team or if you make it public on Docker Hub can inherit from that and add their own stuff. 00:12:17.300 --> 00:12:27.540 Or you can go up in the chain of inheritance of layers and say, oh, I don't like the official Python layer. 00:12:27.540 --> 00:12:32.360 I'm going to see what operating system they're using to run it, and maybe I'm going to create my own. 00:12:33.080 --> 00:12:35.280 So you can branch off at any time. 00:12:35.280 --> 00:12:39.340 It's a very flexible system that gives you total flexibility. 00:12:39.340 --> 00:12:53.120 And at the same time, to me, the big differentiation is that the speed aspect to it, the fact that downloading images is much faster because images are layered. 00:12:53.120 --> 00:13:00.160 And starting a container is like 10 times or maybe 100 times faster, depending on the cases. 00:13:00.860 --> 00:13:04.660 When you make things faster, usually it changes the experience. 00:13:04.660 --> 00:13:10.100 And that's what developers found or realized when they start. 00:13:10.100 --> 00:13:13.920 When a developer starts to use Docker, usually they have this aha moment. 00:13:13.920 --> 00:13:26.600 Oh, then I can build all the time and create my own images and start breaking my application that was monolithic into smaller microservices that are easier to compose with something like Docker Compose. 00:13:26.600 --> 00:13:36.800 So it goes hand in hand with two big shifts in software engineering that happened in the past five years. 00:13:37.640 --> 00:13:38.640 One of them is microservices. 00:13:38.640 --> 00:13:47.540 One of them is microservices, the fact of breaking up your application into smaller services that can be scaled and developed independently. 00:13:47.540 --> 00:14:04.380 And the other one is DevOps, where dev teams and ops teams are working together with continuous integration and continuous delivery using the same tools to achieve continuous delivery of their application. 00:14:05.100 --> 00:14:09.480 And so the container is a really nice unit of collaboration between devs and ops. 00:14:09.480 --> 00:14:16.120 I think you're right that the size makes that really even possible to some degree. 00:14:16.120 --> 00:14:20.260 So let me give you my mental model for the DevOps story. 00:14:20.260 --> 00:14:24.460 And maybe you can tell me how I have that slightly wrong or improve on it or whatever. 00:14:24.680 --> 00:14:35.380 So in my mind, I'm thinking there's some developers building their app and probably they have some kind of continuous integration set up where they check in, some automated build happens, some test run, stuff like that. 00:14:35.960 --> 00:14:41.520 So I could build instead of just running my code directly, checking in and running my test directly on the, say, the build server. 00:14:41.520 --> 00:14:42.920 I could create a container. 00:14:42.920 --> 00:14:52.560 And when I do a check in, the build server could actually create one of these containers, push my code into it, run that code in that actual container environment. 00:14:52.560 --> 00:14:57.580 And if I'm happy, maybe I could automatically push that out to some cloud provider. 00:14:57.580 --> 00:15:02.620 And the thing that's actually running is the thing I tested and built locally. 00:15:02.620 --> 00:15:03.660 Yeah. 00:15:04.080 --> 00:15:08.000 So there's no sort of environmental differences or anything like that. 00:15:08.000 --> 00:15:09.140 It's just that is the thing. 00:15:09.140 --> 00:15:11.440 I can actually ship almost the infrastructure. 00:15:11.440 --> 00:15:11.980 Yeah. 00:15:11.980 --> 00:15:13.640 I think you got it right. 00:15:13.640 --> 00:15:31.080 One of the things I would add is that this kind of factoring out of applications into microservices works really well for people who follow the 12-factor app manifesto that Heroku published a few years ago 00:15:31.080 --> 00:15:35.080 where you try to build your apps as stateless as possible. 00:15:35.080 --> 00:15:38.840 You run data services in a centralized way. 00:15:38.840 --> 00:15:47.160 And then you configure the connectivity between different microservices with environment variables. 00:15:47.160 --> 00:15:57.600 And so typically what would happen is that you would run locally as a developer on a single box with a test MySQL container 00:15:57.600 --> 00:16:01.420 that has a volume attached with test data in there. 00:16:01.420 --> 00:16:13.700 And then when you just got pushed to a build server, the build server deploys that into a cloud QA environment where automated tests are deployed. 00:16:13.700 --> 00:16:21.600 And then there the container would be tied to a test MySQL database that maybe is maintained by operations. 00:16:21.600 --> 00:16:31.340 So this is where the split between dev and operations happens where the devs specify all the requirements that they have for their app in the container. 00:16:31.340 --> 00:16:36.600 And then the ops are providing the right plugs to the right services in different environments. 00:16:37.480 --> 00:16:50.880 And then eventually you would push that to production where typically there would be someone, a sysadmin, that would decide, hey, this build is ready to go to production. 00:16:50.880 --> 00:16:56.720 I'm just going to trigger the fact that in production the image is going to be pulled. 00:16:56.720 --> 00:16:59.120 It's the same image that I had on my dev box. 00:16:59.120 --> 00:17:03.780 But there it's configured to go to my production MySQL cluster. 00:17:03.780 --> 00:17:07.220 That's really amazing. 00:17:07.220 --> 00:17:11.220 So how do I deal with persistent data? 00:17:11.220 --> 00:17:12.800 You know, I've got this. 00:17:12.800 --> 00:17:18.900 Let's say I create one of these Docker containers that contains like MongoDB. 00:17:18.900 --> 00:17:22.060 MongoDB has to have a data store somewhere on the file system. 00:17:22.360 --> 00:17:25.000 How do I kind of deal with inversion that? 00:17:25.000 --> 00:17:26.600 Is that actually in the container? 00:17:26.600 --> 00:17:28.560 Is that somewhere else? 00:17:28.560 --> 00:17:34.600 How do I, you know, if the database schema changes as the new container comes up, what do I do? 00:17:34.600 --> 00:17:35.860 That seems a little tricky. 00:17:35.860 --> 00:17:36.520 Yeah, yeah, yeah. 00:17:36.520 --> 00:17:36.840 Definitely. 00:17:36.840 --> 00:17:47.480 So Docker has this notion of values where in your container you can mount a directory that is on your host machine. 00:17:47.700 --> 00:17:49.880 And this is where you would keep the data. 00:17:49.880 --> 00:18:00.820 And typically there's also another notion that is called a data container where you just launch a container that just has a volume attached to it. 00:18:00.820 --> 00:18:06.280 And then you mount the volumes from that container into your runtime container. 00:18:07.080 --> 00:18:18.200 And then changing database like from development to production would just be a matter of changing the volume of the container you're mounting your volume from. 00:18:18.200 --> 00:18:23.860 So that's the notion of volume that lets you have persistent containers. 00:18:24.400 --> 00:18:41.080 There's one issue with it in Docker today that lots of people are running into when they try to deploy in clusters, which is volumes can be attached only for local file systems, on the engine on which you are triggered, on the host on which you are triggered. 00:18:42.200 --> 00:18:49.920 And there's a company called cluster HQ, which is building an add-on to Docker called Fluckr. 00:18:49.920 --> 00:18:55.540 We're working with them to make it a plugin, like a real plugin in Docker. 00:18:55.540 --> 00:18:59.240 So there's work underway to make that happen right now. 00:18:59.720 --> 00:19:06.420 And what Fluckr is doing is that they're addressing this issue of moving persistent container from one host to another. 00:19:06.420 --> 00:19:20.540 If you're deploying in a large orchestrated Docker setup, where when the orchestrator tells the engine, hey, I want to move that Docker engine that is tied to your, 00:19:20.540 --> 00:19:31.200 Docker container that has a persistent volume from host A to host B, what they're doing is that they're using ZFS under the hood, 00:19:31.200 --> 00:19:34.580 and they're doing a ZFS snapshot, which are pretty fast. 00:19:34.580 --> 00:19:38.160 Then they're moving the snapshot to the other machine that can take a few hours. 00:19:38.160 --> 00:19:47.180 And once the snapshot is moved, they can start your server, take a diff of the snapshot and push it on the other side, which is faster. 00:19:47.180 --> 00:19:49.860 And then they restart your container on the other side. 00:19:50.220 --> 00:19:54.780 That sounds really helpful because it seems like there's a lot of complexity, more and more containers that get involved, 00:19:54.780 --> 00:19:56.540 a more horizontal scale that gets in there. 00:19:56.540 --> 00:19:56.840 Yeah. 00:19:56.840 --> 00:19:57.380 Yeah. 00:19:57.380 --> 00:20:05.480 Container persistence is really an interesting issue where there's lots of research right now and lots of companies looking into that and how to enable this. 00:20:05.480 --> 00:20:06.400 Excellent. 00:20:06.400 --> 00:20:08.660 So you've spoken a little bit about Docker Hub. 00:20:08.660 --> 00:20:10.740 Can you tell the listeners what that is? 00:20:10.740 --> 00:20:11.500 Yeah, definitely. 00:20:11.500 --> 00:20:19.420 So Docker itself is a client and a demon that's running either on your local machine or on your server. 00:20:19.900 --> 00:20:26.980 But what you're building with the Docker client is an image of the system on which you're running. 00:20:26.980 --> 00:20:32.880 Once your image is ready, typically you would push it to Docker Hub, which is a hosted service by Docker. 00:20:32.880 --> 00:20:34.540 So it's software as a service. 00:20:34.540 --> 00:20:40.820 It's a hosted service that Docker, that lets you share your images either with the world. 00:20:41.220 --> 00:20:42.480 You can make your image public. 00:20:42.480 --> 00:20:42.780 You can make your image public. 00:20:42.780 --> 00:20:47.720 And there are lots of open source and community images in there. 00:20:47.720 --> 00:20:50.200 Or you can host them privately. 00:20:50.740 --> 00:20:54.700 So Docker Hub is free for public images. 00:20:54.700 --> 00:21:02.860 But if you want a private image, you can buy a subscription that allows you to have several private images for your own projects. 00:21:03.660 --> 00:21:08.000 And then your colleague can just download the images from the hub as well. 00:21:08.000 --> 00:21:10.120 And it's integrated with Docker client. 00:21:10.120 --> 00:21:14.240 We also have a version that's called Docker Hub Enterprise. 00:21:14.740 --> 00:21:20.160 That enterprises who want to adopt Docker for their internal workflow can install behind the firewall. 00:21:20.160 --> 00:21:25.080 So they have complete control over the policies for the images that they're creating. 00:21:25.080 --> 00:21:25.820 Right. 00:21:25.820 --> 00:21:26.440 That makes sense. 00:21:26.440 --> 00:21:29.840 You might not want to just go grab a bunch of public images and download them. 00:21:29.840 --> 00:21:34.640 Is this actual source code or actual binary executables that I'm grabbing? 00:21:34.860 --> 00:21:37.760 Or is there another way to get an image from Docker Hub? 00:21:37.760 --> 00:21:53.080 So what you grab from Docker Hub, these images are actually tables that are gzip for each of the layers with a bunch of metadata explaining the Docker Engine how to reconstitute the image of your operating system. 00:21:53.080 --> 00:21:54.180 Okay, excellent. 00:21:54.180 --> 00:21:58.860 So it might tell me how to install Nginx or how to install Python 3 or something like that. 00:21:58.860 --> 00:21:59.420 Yeah, exactly. 00:21:59.420 --> 00:22:05.900 So we host official Docker images for many open source projects. 00:22:05.900 --> 00:22:08.640 So for example, for Python, there's an excellent... 00:22:08.640 --> 00:22:11.260 I used to take images from the community initially. 00:22:11.260 --> 00:22:15.080 And then I switched to the official image, which is called Python. 00:22:15.080 --> 00:22:22.040 And so in order to run Python in Docker, you just say Docker run Python. 00:22:22.040 --> 00:22:27.260 Eventually, you could qualify that to say if you want Python 2 or Python 3. 00:22:27.800 --> 00:22:34.760 Or there are labels on these images that let you specify a more fine-grained version of Python. 00:22:34.760 --> 00:22:40.020 And once you run the image, it comes with a... 00:22:40.020 --> 00:22:42.940 I don't remember which flavor of Linux. 00:22:42.940 --> 00:22:49.480 And then Python all installed with the tools for developing there or for running Python workloads. 00:22:49.480 --> 00:22:52.720 That sounds like a really easy way to get started for the Python developers. 00:22:52.720 --> 00:22:53.180 That's cool. 00:22:53.180 --> 00:22:56.860 Yeah, and that's one of the things that I love. 00:22:56.940 --> 00:23:01.100 So I joined Docker just two months ago. 00:23:01.100 --> 00:23:08.420 Before that, I was at Microsoft where my role as part of the developer experience team 00:23:08.420 --> 00:23:12.980 was to bring all the Docker partner ecosystem on Azure. 00:23:12.980 --> 00:23:19.660 And so as part of doing that, I built a bunch of tools to deploy clusters on Azure. 00:23:19.660 --> 00:23:22.540 And the tool I built for that was in Python. 00:23:22.540 --> 00:23:25.400 And I interacted with lots of startups using Python. 00:23:25.400 --> 00:23:32.460 And when I started talking about my tools to other Microsoft people and I started giving trainings about that, 00:23:32.460 --> 00:23:34.060 these guys were on Windows. 00:23:34.060 --> 00:23:36.060 They didn't have Python installed. 00:23:36.060 --> 00:23:38.260 They said, how do I run that script? 00:23:38.980 --> 00:23:45.260 And in order to make that easier, I just bundled all my script in a Docker image. 00:23:45.260 --> 00:23:51.280 And I just told them, hey, you just run that image and you'll have Python 2 installed, 00:23:51.280 --> 00:23:57.700 all the requirements for the tool that I built, like the Azure Python SDK, all pre-installed. 00:23:57.700 --> 00:23:59.760 And you don't need to install anything. 00:23:59.760 --> 00:24:03.700 It's just a Docker run away from running. 00:24:03.700 --> 00:24:05.060 That's really cool. 00:24:05.060 --> 00:24:09.440 It sounds like an interesting world to live in, working with a lot of the Windows stuff. 00:24:09.440 --> 00:24:15.900 And actually, one of the things I'd kind of like to talk about in a minute is the Docker view from different operating systems. 00:24:16.020 --> 00:24:18.480 I mean, if I'm on Linux, it makes perfect sense. 00:24:18.480 --> 00:24:19.800 It's kind of like a Linux thing. 00:24:19.800 --> 00:24:22.900 I can run Docker right on Ubuntu, things like that. 00:24:22.900 --> 00:24:23.160 Yeah. 00:24:23.160 --> 00:24:27.360 But if I'm on, say, let's say, on my Mac, what do I do there? 00:24:27.360 --> 00:24:31.960 Yeah, so on a Mac, you can download an installer from docker.com. 00:24:31.960 --> 00:24:37.740 So we always release all our versions on Linux, Mac, and Windows. 00:24:38.740 --> 00:24:47.360 And so you download that installer and what it's doing because the engine relies on primitives in the Linux kernel. 00:24:47.360 --> 00:24:48.840 It needs to run on Linux. 00:24:48.840 --> 00:24:53.840 And so we have this tool that's downloaded as part of the installer that's called Boot2Docker 00:24:53.840 --> 00:25:01.040 that starts behind the scene a VirtualBox VM running Linux where the engine is installed. 00:25:01.040 --> 00:25:06.480 And then you run the client on your Mac, and your client is targeting that VirtualBox instance. 00:25:07.380 --> 00:25:15.340 And then by changing the environment variables, you can just target, instead of targeting your local machine, 00:25:15.340 --> 00:25:24.560 your local VirtualBox instance on your Mac, you can target a box in Azure or Google Compute Engine or Amazon. 00:25:24.560 --> 00:25:26.220 So that's pretty easy. 00:25:26.220 --> 00:25:31.580 Yeah, in terms of operating system, one of the things that I'm super happy about that happened, 00:25:31.580 --> 00:25:37.300 that was launched a few weeks ago and that was announced at the build, 00:25:37.300 --> 00:25:44.220 conference last week from Microsoft, that Microsoft really bet heavily on Docker. 00:25:44.220 --> 00:25:52.620 And what they've done is that they have contributed a lot of code to Docker so that now the Docker client runs on Windows. 00:25:52.620 --> 00:25:56.440 So you can run the Docker client and then target Linux machines. 00:25:56.440 --> 00:25:58.860 You can use Boot2Docker on Windows as well. 00:25:59.200 --> 00:26:02.980 So you can run your Linux workloads on Windows with that and develop. 00:26:02.980 --> 00:26:04.780 That's cool. 00:26:04.780 --> 00:26:06.920 So that kind of puts it on par with OS X maybe. 00:26:06.920 --> 00:26:07.500 Yeah, exactly. 00:26:07.500 --> 00:26:09.260 Puts it on par with OS X. 00:26:09.340 --> 00:26:20.080 But they're going to run the Docker client. 00:26:20.080 --> 00:26:30.820 So that's cool. 00:26:30.820 --> 00:26:32.680 So that's really a big news. 00:26:32.680 --> 00:26:43.280 So Microsoft likes the Docker workflow for devs and ops so much that they wanted to give that to Windows developers and Windows sysadmins. 00:26:43.500 --> 00:26:50.420 So they modified Windows server to have the same isolation primitive that exists in Linux. 00:26:50.420 --> 00:26:53.860 And then they implemented the Docker daemon in terms of that. 00:26:53.860 --> 00:26:56.020 So that's not out yet. 00:26:56.020 --> 00:26:58.080 They just gave a demo last week. 00:26:58.080 --> 00:27:02.320 But it's going to shift with the next version of Windows server. 00:27:02.320 --> 00:27:04.720 And there will be a preview this summer. 00:27:05.140 --> 00:27:13.480 So that means that you'll be able to develop .NET applications running on Windows server inside of Docker containers. 00:27:13.480 --> 00:27:21.700 Which means that for developers and sysadmins in enterprise who are typically working half and half with Java and .NET, 00:27:21.700 --> 00:27:30.040 they'll be able to just deploy or to build their microservices in the technology that fits the best. 00:27:30.040 --> 00:27:32.000 Yeah, that's really cool. 00:27:32.000 --> 00:27:32.960 I'm glad you brought that up. 00:27:33.060 --> 00:27:35.440 I was watching the Build conference last week as well. 00:27:35.440 --> 00:27:40.140 And just for the listeners, because of time shifting, we're recording on Monday, May 4th. 00:27:40.140 --> 00:27:43.980 And Build, I think this was announced last Wednesday, so four or five days ago. 00:27:43.980 --> 00:27:54.780 And one of the very first things they announced at the entire conference was your CEO, Ben Golub, coming out and saying, you know, from Docker, saying, 00:27:55.000 --> 00:27:57.860 look, we're working with Microsoft doing amazing things. 00:27:57.860 --> 00:28:01.120 And we started out maybe trying to bring it on par with OS X. 00:28:01.120 --> 00:28:07.480 But the Microsoft guys wanted to go even farther and actually add features to Windows, the OS, 00:28:07.480 --> 00:28:11.860 that would allow it to have containers for its operating system and its processes. 00:28:11.860 --> 00:28:17.220 Because until this moment, there was no possibility of something like Docker on Windows for Windows apps. 00:28:18.080 --> 00:28:20.160 Yeah, yeah, it's pretty incredible. 00:28:20.160 --> 00:28:28.440 So, I mean, to me, that means like now we have the two major operating systems that run software and the servers in the data centers, 00:28:28.440 --> 00:28:31.080 both supporting containers, both working with Docker. 00:28:31.080 --> 00:28:32.660 And that's a huge boost for you guys, right? 00:28:32.720 --> 00:28:33.580 Yeah, yeah, it is. 00:28:33.580 --> 00:28:34.600 It is definitely. 00:28:34.600 --> 00:28:45.580 And I think it's very good for the Windows ecosystem as well, because it will bring them the same kind of tools that Linux developers and admins have been enjoying for a long time. 00:28:45.580 --> 00:28:51.040 And it really allows for more dense workloads on your servers and in the data center. 00:28:51.040 --> 00:28:52.200 Yeah, that's really cool. 00:28:52.200 --> 00:28:54.720 So the DevOps story just gets way better over there, I think. 00:28:54.720 --> 00:28:55.460 Yeah, exactly. 00:28:55.460 --> 00:28:58.460 So I'd like to talk a little bit about Python with you. 00:28:58.460 --> 00:29:01.700 But before we do, what do you think the future holds for Docker and containers? 00:29:01.700 --> 00:29:04.060 Where are things going in this whole world? 00:29:04.060 --> 00:29:09.580 Yeah, so we have DockerCon, which is our big conference that's happening in June. 00:29:09.580 --> 00:29:20.600 And one of the things we've been working on, as I explained to you before, Docker takes care of a lot of aspects, but it still needs to mature in lots of areas. 00:29:20.600 --> 00:29:27.960 And especially when people start putting it in production, there are lots of needs that are still unfulfilled by the tool. 00:29:27.960 --> 00:29:33.040 So one of the big aspects that we've been working on recently is to make it pluggable. 00:29:33.580 --> 00:29:43.580 So we're working on a plugin model with people like Weave, who are building virtual networking, or ClusterHQ, who are building portable volumes, 00:29:43.580 --> 00:29:57.120 so that they can run as plugin in Docker Engine, and so that you would be able to define networks for your containers that span your own data center, 00:29:57.820 --> 00:30:02.020 and maybe Azure and Google, or like two public clouds. 00:30:02.020 --> 00:30:08.200 So that's on the Weave side, and on the ClusterHQ side, it's like moving volumes from one to the other. 00:30:08.200 --> 00:30:14.820 We're also working on making sure Docker Machine lets you provision Docker daemons everywhere. 00:30:14.820 --> 00:30:18.140 There's been lots of improvement to Docker Compose. 00:30:18.600 --> 00:30:26.940 And also, one of the big topics that's happening right now is once you start building all your apps as microservices running in containers, 00:30:26.940 --> 00:30:29.740 the next step is how do you orchestrate them? 00:30:29.740 --> 00:30:38.760 And I have a whole deck on that that I'd send you where there's at least six or seven containers in that space. 00:30:39.760 --> 00:30:44.920 Again, as I used to say, the whole industry reorganized around this. 00:30:44.920 --> 00:30:53.960 Before, people were building platforms as a service, and that whole industry refocused towards orchestrating Docker containers. 00:30:53.960 --> 00:30:58.600 So the unit now is the microservice package as a container. 00:30:58.600 --> 00:31:04.300 And in the orchestrators, you have Docker Swarm, which is one from Docker itself. 00:31:04.500 --> 00:31:07.900 So it's a native orchestration solution that works multi-cloud. 00:31:07.900 --> 00:31:15.140 There's Kubernetes from Google that's inspired by what's running internally at Google, and that's open source. 00:31:15.140 --> 00:31:18.900 There's Apache Mesos, done by Mesosphere. 00:31:18.900 --> 00:31:22.740 And they're running in production in lots of different places. 00:31:22.740 --> 00:31:24.540 They're coming from Twitter originally. 00:31:24.540 --> 00:31:26.720 I think Netflix runs it as well. 00:31:26.720 --> 00:31:34.080 And there are smaller players like DS or Tutum, which is running on Azure as well. 00:31:34.080 --> 00:31:38.960 So there's lots of activity in that space of our Docker container orchestration. 00:31:38.960 --> 00:31:41.560 Yeah, that seems like where the real challenges lie. 00:31:41.560 --> 00:31:45.760 And if that gets easier and easier, then, you know, the sky's the limit. 00:31:45.760 --> 00:31:46.640 So that's amazing. 00:31:46.640 --> 00:31:52.060 Yeah, on the Docker side, we're trying to make the whole experience easier, like with machines, 00:31:52.060 --> 00:31:55.340 like one command to provision new machines in different clouds. 00:31:55.560 --> 00:31:58.080 And then it's integrated with Swarm. 00:31:58.080 --> 00:32:01.720 So you can say, oh, I want all these machines to be in a single cluster. 00:32:01.720 --> 00:32:08.120 And then with Docker Compose, you can say, oh, I want to, or there's the start of an integration 00:32:08.120 --> 00:32:14.140 where you can say, my app that's composed of multiple services, please deploy that to that Swarm instance. 00:32:14.140 --> 00:32:19.240 And then it will just provision it in the right place in the cluster where there's some room. 00:32:19.720 --> 00:32:24.780 So we're really trying to make the experience much easier for developers and sysadmins. 00:32:24.780 --> 00:32:25.260 Cool. 00:32:25.260 --> 00:32:25.500 Yeah. 00:32:25.500 --> 00:32:30.800 The more you guys encourage and make it easy to break our apps into small little services, 00:32:30.800 --> 00:32:33.140 then the challenge becomes linking them together, right? 00:32:33.140 --> 00:32:35.500 So it sounds like you're doing some great work there. 00:32:35.500 --> 00:32:35.960 Fantastic. 00:32:35.960 --> 00:32:36.200 Yeah. 00:32:36.280 --> 00:32:42.020 And one advantage I will have to add that there is to this microservice approach and that Docker enables 00:32:42.020 --> 00:32:51.420 is once you start breaking your app in microservices, you can start innovating in terms of what language 00:32:51.420 --> 00:32:54.140 and platform you're using for each of the microservice. 00:32:54.280 --> 00:33:02.240 So as opposed to being stuck, for example, with a Django app running in Python 2.7 and you made that choice 00:33:02.240 --> 00:33:10.040 and there's some legacy code that cannot be ported to Python 3, you could say, oh, I just break my app in 10 different microservices 00:33:10.040 --> 00:33:12.540 and for this microservice, I don't have any legacy. 00:33:12.540 --> 00:33:14.260 I'm going to write it in Python 3. 00:33:14.920 --> 00:33:25.760 And for the developers, it's just a matter of saying, oh, I'm inheriting from Python 3 as opposed to Python 2 when you're creating your image. 00:33:25.760 --> 00:33:26.980 Oh, that's really neat. 00:33:26.980 --> 00:33:31.540 So maybe it's a gateway for a little more flexibility in the technology to... 00:33:31.540 --> 00:33:31.760 Yeah. 00:33:31.760 --> 00:33:40.520 And I think especially for the Python community where that big gap between Python 2 and Python 3 developers happen, 00:33:40.820 --> 00:33:49.020 I think this breaking up of large monolithic applications into smaller microservices where you have more freedom to test new stuff 00:33:49.020 --> 00:33:53.060 may be a good way of introducing Python 3 to your environment. 00:33:53.060 --> 00:33:53.720 For sure. 00:33:53.720 --> 00:33:57.420 Do you see anything like that happening in the data side of things? 00:33:57.420 --> 00:34:02.500 So, you know, if I'm using MySQL and relational databases in some big monolithic way, 00:34:02.500 --> 00:34:09.060 like, could my different microservices be using, like, NoSQL, say, MongoDB or Redis here and there, 00:34:09.060 --> 00:34:12.500 and then maybe one other part still using a relational database? 00:34:12.500 --> 00:34:12.900 Oh, yeah. 00:34:12.900 --> 00:34:15.980 I've seen a lot of that where people are starting... 00:34:15.980 --> 00:34:17.700 Once your microservices... 00:34:17.700 --> 00:34:23.700 Your microservice has a smaller footprint and, like, a single functional goal, 00:34:23.700 --> 00:34:28.660 then it's easier to say, oh, I'm going to back that up with Redis or MongoDB 00:34:28.660 --> 00:34:31.960 as opposed to storing everything in MySQL 00:34:31.960 --> 00:34:40.000 because maybe I'm building an e-commerce application and I need the relational aspect for parts of it, 00:34:40.000 --> 00:34:51.380 but maybe for, like, tracking referrals or the social media aspect of how people are tweeting about products on my website, 00:34:51.380 --> 00:34:57.800 maybe these small services can use different types of databases or data backends. 00:34:57.800 --> 00:35:01.920 Yeah, that seems like a perfect way to choose the right database, the right job there. 00:35:01.920 --> 00:35:03.700 Cool. 00:35:03.700 --> 00:35:06.260 So you guys, do you use Python internally? 00:35:06.260 --> 00:35:09.840 You know, obviously you can run Python in Docker containers really well, 00:35:09.840 --> 00:35:11.920 but do you guys use it as a language for yourself? 00:35:11.920 --> 00:35:12.980 Actually, we do. 00:35:12.980 --> 00:35:14.160 Not everybody. 00:35:14.160 --> 00:35:19.460 So Docker itself is written in Go as well as Docker Machine and Swarm. 00:35:20.180 --> 00:35:26.020 But Docker Compose is actually written in Python, so it's a Python project. 00:35:26.020 --> 00:35:26.380 Nice. 00:35:26.380 --> 00:35:26.880 And how's that? 00:35:26.880 --> 00:35:28.980 People are pretty happy with Python internally? 00:35:28.980 --> 00:35:33.600 Yeah, and I think they even tried to call it to Go at some point, 00:35:33.600 --> 00:35:37.340 but then they said, hey, we have high velocity with Python, 00:35:37.340 --> 00:35:42.000 we have a code base that people understand well, so let's stick with that. 00:35:42.800 --> 00:35:45.460 So I think Compose for now will stay in Python, 00:35:45.460 --> 00:35:49.180 and the developers who are working on it are pretty happy with it. 00:35:49.180 --> 00:35:50.260 Yeah, that's really cool. 00:35:50.260 --> 00:35:54.120 So if, you know, I'm a listener and I'm out there hearing this, 00:35:54.120 --> 00:35:56.260 and I'm like, this is awesome, I have to get started. 00:35:56.260 --> 00:35:56.880 What do I do? 00:35:56.880 --> 00:35:57.600 How do I get started? 00:35:57.600 --> 00:36:00.880 Yeah, to get started, you go to docker.com. 00:36:00.880 --> 00:36:02.300 We have lots of tutorials. 00:36:02.300 --> 00:36:05.860 If you're on a Mac, you just download Docker for Mac, for Linux. 00:36:06.380 --> 00:36:08.980 Depending on the distro, either there are packages 00:36:08.980 --> 00:36:11.080 or you can download the binaries directly. 00:36:11.080 --> 00:36:13.760 And for Windows, there's an installer as well. 00:36:13.760 --> 00:36:17.440 There's also another project you may want to take a look at, 00:36:17.440 --> 00:36:18.500 that's called Ketmatic. 00:36:18.500 --> 00:36:25.380 That one runs only on macOS, and it's a Docker client GUI. 00:36:25.380 --> 00:36:30.600 So it shows you all the containers that you can find on Docker Hub, 00:36:30.600 --> 00:36:32.620 and you can say, oh, just start this one, 00:36:32.720 --> 00:36:36.600 launch a terminal where I can script it or connect to it, 00:36:36.600 --> 00:36:41.000 and it shows you the logs, the volume that you have attached, 00:36:41.000 --> 00:36:43.180 so you can go edit the files and all that. 00:36:43.180 --> 00:36:48.140 So that's a very easy way to get started, Ketmatic, I would say. 00:36:48.140 --> 00:36:49.260 And else? 00:36:49.260 --> 00:36:51.780 Yeah, I have Ketmatic installed, and I enjoy it. 00:36:51.780 --> 00:36:52.160 It's nice. 00:36:52.160 --> 00:36:53.800 It's clean and works well. 00:36:53.800 --> 00:36:55.740 Yeah, there's lots of stuff to add in there, 00:36:55.740 --> 00:36:59.320 but it's really a good start, and it's very easy to get started. 00:36:59.320 --> 00:37:00.020 Excellent. 00:37:00.800 --> 00:37:05.000 So I think it's probably a place to call it a show, Patrick. 00:37:05.000 --> 00:37:08.720 What else would you like to add while we've got a chance to talk to everyone? 00:37:08.720 --> 00:37:10.820 I mean, thanks very much for the opportunity. 00:37:10.820 --> 00:37:16.400 I really encourage Python developers to go take a look at that. 00:37:16.400 --> 00:37:20.940 To me, one of the angles that's most interesting for Python developers 00:37:20.940 --> 00:37:26.560 is giving you the portability for your customers to run your code anywhere 00:37:26.560 --> 00:37:30.680 without having a complicated environment to set up. 00:37:30.680 --> 00:37:36.020 As long as they have Docker, they can just start your app packaged as a container. 00:37:36.020 --> 00:37:39.360 And the other angle that's, I think, pretty important 00:37:39.360 --> 00:37:44.660 is starting to introduce Python 3 microservices into your app. 00:37:44.660 --> 00:37:49.960 If all your apps are packaged as containers, it just doesn't matter. 00:37:49.960 --> 00:37:52.940 You don't need to set up complicated environments. 00:37:52.940 --> 00:38:00.380 It's just a change of the label in the from directive in your Docker file 00:38:00.380 --> 00:38:02.180 to start using Python 3. 00:38:02.640 --> 00:38:07.220 I am very excited about Docker and this whole container world. 00:38:07.220 --> 00:38:09.920 And I appreciate you taking the time to share it with everyone. 00:38:09.920 --> 00:38:10.560 Thanks, Michael. 00:38:10.560 --> 00:38:11.360 You're welcome, Patrick. 00:38:11.360 --> 00:38:12.080 Bye. 00:38:12.080 --> 00:38:12.640 Bye. 00:38:13.980 --> 00:38:16.880 This has been another episode of Talk Python To Me. 00:38:16.880 --> 00:38:22.240 Today's guest was Patrick Shanazin, and this episode has been sponsored by Codeship. 00:38:22.240 --> 00:38:26.680 Please check them out at Codeship.com and thank them on Twitter via at Codeship. 00:38:26.680 --> 00:38:28.460 Don't forget the discount code for listeners. 00:38:28.460 --> 00:38:29.180 It's easy. 00:38:29.180 --> 00:38:31.620 Talk Python, all caps, no spaces. 00:38:31.620 --> 00:38:37.140 Remember, you can find the links from this show at talkpythontome.com 00:38:37.140 --> 00:38:40.040 slash episodes slash show slash nine. 00:38:40.040 --> 00:38:43.900 And if you're feeling generous, please check out our Patreon campaign 00:38:43.900 --> 00:38:48.120 at patreon.com slash mkennedy to contribute and support the show. 00:38:48.120 --> 00:38:49.900 And be sure to subscribe to the show. 00:38:49.900 --> 00:38:52.300 Visit the website and choose subscribe in iTunes 00:38:52.300 --> 00:38:56.080 or grab the episode RSS feed and drop it into your favorite podcatcher. 00:38:56.080 --> 00:38:58.800 You'll find both at the footer of every page. 00:38:58.800 --> 00:39:00.500 This is your host, Michael Kennedy. 00:39:00.500 --> 00:39:01.380 Thanks for listening. 00:39:13.820 --> 00:39:14.100 Bye. 00:39:14.100 --> 00:39:14.100 Bye. 00:39:14.100 --> 00:39:14.100 Bye. 00:39:14.100 --> 00:39:14.100 Bye. 00:39:14.100 --> 00:39:20.140 first, develop first, develop first, develop first, develop first. 00:39:20.140 --> 00:39:22.860 Developers, developers, developers, developers. 00:39:22.860 --> 00:39:23.100 you 00:39:23.100 --> 00:39:24.100 Thank you.