WEBVTT 00:00:00.001 --> 00:00:03.360 Have you heard about the It Works on My Machine certification program? 00:00:03.360 --> 00:00:06.440 This is a really awesome certification for developers. 00:00:06.440 --> 00:00:07.960 It was created by Joseph Cooney. 00:00:07.960 --> 00:00:08.920 Here's how it works. 00:00:08.920 --> 00:00:11.700 Step one, compile your application code. 00:00:11.700 --> 00:00:16.200 Getting the latest version of any recent code changes from other developers 00:00:16.200 --> 00:00:19.020 is purely optional and not a requirement for certification. 00:00:19.020 --> 00:00:22.720 Step two, launch the application or website that's just been compiled. 00:00:22.720 --> 00:00:26.920 Step three, cause one code path in the code you're checking in to be executed. 00:00:27.040 --> 00:00:30.660 Now, the preferred way to do this is with ad hoc manual testing 00:00:30.660 --> 00:00:33.780 of the simplest possible case for the feature in question. 00:00:33.780 --> 00:00:37.720 And you may omit this step if the code change was less than five lines 00:00:37.720 --> 00:00:40.580 or if in the developer's professional opinion, 00:00:40.580 --> 00:00:43.180 the code change could not possibly result in an error. 00:00:43.180 --> 00:00:45.840 Now, check in your changes to version control. 00:00:45.840 --> 00:00:47.440 Congratulations, you're certified. 00:00:47.440 --> 00:00:52.240 On this episode of Talk Python To Me, you'll meet Florian Motlik from CodeChip. 00:00:52.240 --> 00:00:56.260 He's here to tell us all about continuous integration and continuous delivery. 00:00:56.520 --> 00:00:59.740 Maybe you can help keep you and your team from getting certified as well. 00:00:59.740 --> 00:01:04.360 This is episode number 38, recorded December 10th, 2015. 00:01:04.360 --> 00:01:34.340 Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem, 00:01:34.340 --> 00:01:35.560 and the personalities. 00:01:35.560 --> 00:01:37.700 This is your host, Michael Kennedy. 00:01:37.700 --> 00:01:39.700 Follow me on Twitter where I'm @mkennedy. 00:01:39.700 --> 00:01:46.100 Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via at Talk Python. 00:01:46.100 --> 00:01:50.000 This episode is brought to you by Hired and DigitalOcean. 00:01:50.000 --> 00:01:56.040 Thank them for supporting the show via Twitter where they're at Hired underscore HQ and at DigitalOcean. 00:01:56.680 --> 00:01:57.340 Hey, everyone. 00:01:57.340 --> 00:01:59.060 Thanks again for listening today. 00:01:59.060 --> 00:02:02.440 I have a few prize winners to announce before we get to the show. 00:02:02.440 --> 00:02:06.300 Two weeks ago, JetBrains and I gave away three PyCharm professional licenses. 00:02:06.300 --> 00:02:11.300 The lucky winners were Drew Kerr, Ben Kelly, and Seref Alvinides. 00:02:11.800 --> 00:02:17.280 And in episode 37 on computer security, Justin Seitz gave away a copy of Black Hat Python. 00:02:17.280 --> 00:02:20.160 The lucky winner there was Arek Fyla Kowalski. 00:02:20.160 --> 00:02:22.140 Congratulations to everyone. 00:02:22.140 --> 00:02:27.180 And if you want to be able to win these giveaways, all you have to do is be a friend of the show. 00:02:27.420 --> 00:02:31.740 Just visit talkpython.fm, click Friends of the Show in the navbar, and sign up. 00:02:31.740 --> 00:02:33.860 Now, let me introduce Flo. 00:02:33.860 --> 00:02:39.420 Florian Motlik is the co-founder and CTO of CodeShip, a continuous delivery service. 00:02:39.420 --> 00:02:46.680 While studying and working in the Technical University in Vienna, he stumbled into running test infrastructure for other teams and hasn't stopped since. 00:02:46.680 --> 00:02:53.760 He sometimes blogs on the CodeShip blog at blog.codeship.com or on his personal blog, flomotlik.me. 00:02:54.360 --> 00:02:57.720 You can also find him on Twitter, where he's at Flo Mollik. 00:02:57.720 --> 00:02:59.160 Flo, welcome to the show. 00:02:59.160 --> 00:03:00.800 Thanks. Thanks for having me. 00:03:00.800 --> 00:03:05.920 Yeah, I'm really excited to talk about building software, testing software, and automating the whole thing with you. 00:03:05.920 --> 00:03:07.060 So it'll be fun. 00:03:07.060 --> 00:03:08.060 Yeah, totally. 00:03:08.060 --> 00:03:09.580 For sure. 00:03:09.580 --> 00:03:17.200 So we're going to talk about continuous integration, continuous delivery, maybe throw in a little Docker, infrastructure, all those kinds of things. 00:03:17.200 --> 00:03:20.180 But before we get into those topics, maybe give me your story. 00:03:20.180 --> 00:03:23.060 How do you get into software and programming and this whole thing? 00:03:23.060 --> 00:03:24.060 Sure. 00:03:24.060 --> 00:03:24.100 Sure. 00:03:24.100 --> 00:03:31.040 So I mean, as most people at some point, I started playing games on my dad's old computer. 00:03:31.660 --> 00:03:38.460 And so that got me into a little bit of scripting and stuff like that and trying to look a little bit under the hood at like 10, 11 or something. 00:03:39.180 --> 00:03:40.140 And then when I was 15. 00:03:40.140 --> 00:03:45.320 So the school that I did had a computer science branch, basically, that I went to. 00:03:45.320 --> 00:03:51.840 And so there I really started to get into programming and what it actually means to develop software. 00:03:52.360 --> 00:03:59.160 So that was in school from like 15 on. 00:03:59.160 --> 00:04:07.300 So I'm originally from Vienna and then started software engineering. 00:04:07.300 --> 00:04:14.820 then, basically, and then basically, and at some level, really, the whole web development side of things really captured my interest. 00:04:14.820 --> 00:04:19.840 And so that's web development and infrastructure is what I've done since, basically. 00:04:20.280 --> 00:04:25.840 Oftentimes, I ask people how they got into Python because, you know, a lot of my guests are explicitly doing Python. 00:04:25.840 --> 00:04:27.560 But you guys are doing all sorts of languages. 00:04:28.360 --> 00:04:31.360 Can you maybe just tell me what areas you work in, what technologies? 00:04:31.360 --> 00:04:32.320 Sure. 00:04:32.320 --> 00:04:38.560 So in the past, basically, like I started with the typical Java, like school and university stack. 00:04:38.560 --> 00:04:40.200 So been doing Java for a long time. 00:04:40.200 --> 00:04:41.760 At some point, we switched. 00:04:41.760 --> 00:04:45.480 So when we started CodeShip, I looked at various technologies out there. 00:04:45.480 --> 00:04:54.060 And since it was a small startup and I was the only developer, like I looked into Ruby and Rails for getting stuff up and running quickly. 00:04:54.060 --> 00:05:00.700 So most of our stack, including our back end and like the build infrastructure that we're running for customers, was written in Ruby. 00:05:00.700 --> 00:05:05.980 And then in terms of production systems, like we've started moving more and more stuff to Go. 00:05:05.980 --> 00:05:08.080 So we've re-implemented a lot of the back end systems. 00:05:08.080 --> 00:05:09.480 A lot of that stuff is now in Go. 00:05:09.480 --> 00:05:12.240 So that's on the part of like developing our own systems. 00:05:12.240 --> 00:05:18.900 But then since we basically rerun software and we run the tests for many, many other people and support a lot of different technologies. 00:05:19.500 --> 00:05:28.000 I've dabbled in Python, in Java, of course, a lot of it, in Ruby, in many other languages that we support in our stack. 00:05:28.000 --> 00:05:39.200 So been playing with and enjoying the ecosystem of many different languages, but mostly programming in either Ruby, Java in the past or a little bit of Go now. 00:05:39.200 --> 00:05:41.680 Although I don't get to code a lot anymore, sadly. 00:05:42.500 --> 00:05:43.380 That is unfortunate. 00:05:43.380 --> 00:05:48.180 So I've worked for very large companies and very small companies. 00:05:48.180 --> 00:05:52.360 And I've been in a situation, like you just described, where I'm the only developer. 00:05:52.360 --> 00:05:58.120 And I have to say, it's pretty cool to start something new where you're the only developer. 00:05:58.120 --> 00:06:03.880 And you kind of get to look across the entire tech landscape and say, I can pick anything. 00:06:03.880 --> 00:06:04.740 Where do we start? 00:06:04.740 --> 00:06:07.380 You know, that's a really fun place to be. 00:06:08.020 --> 00:06:12.720 You work for CodeShip, obviously, and you guys have been a sponsor of the show for a long time. 00:06:12.720 --> 00:06:13.600 So thank you for that. 00:06:13.600 --> 00:06:17.360 Why don't you tell everyone what you guys do at CodeShip? 00:06:17.360 --> 00:06:18.480 Sure. 00:06:18.880 --> 00:06:27.100 So the main idea, I mean, just in terms of the messages, CodeShip is a continuous integration, hosted continuous integration and continuous delivery service. 00:06:27.100 --> 00:06:35.540 The main point there is that basically when you build software, you want to test it as fast and as often as possible to really know when something breaks. 00:06:35.540 --> 00:06:36.500 Like you want to do tests. 00:06:36.500 --> 00:06:39.580 You want to do test-driven development or write tests constantly. 00:06:39.960 --> 00:06:48.020 And then you need a separate system that whenever you change any of your code, push anything into the repo, that it runs all of your tests, all of your builds, basically. 00:06:48.020 --> 00:06:52.360 And doing that on your own local development machine is kind of a hassle. 00:06:52.360 --> 00:06:53.520 And oftentimes it's not done. 00:06:53.520 --> 00:06:54.520 Like we're all humans. 00:06:54.520 --> 00:06:57.960 Like we only think this small change will not break my whole application. 00:06:57.960 --> 00:06:59.160 And then it does. 00:06:59.160 --> 00:06:59.820 Yeah. 00:06:59.820 --> 00:07:01.000 Of course it does. 00:07:01.000 --> 00:07:03.300 And, you know, maybe it doesn't break on your machine. 00:07:03.300 --> 00:07:05.280 It breaks on someone else's machine. 00:07:05.280 --> 00:07:05.620 Exactly. 00:07:05.620 --> 00:07:06.360 Exactly. 00:07:06.360 --> 00:07:10.340 Have you seen the It Works on My Machine certification program? 00:07:10.340 --> 00:07:11.980 No, I haven't actually. 00:07:11.980 --> 00:07:16.740 So everyone out there listening, you should search Google for Works on My Machine certification. 00:07:16.740 --> 00:07:21.880 It's a really great certification program that basically marks you as a bad developer. 00:07:21.880 --> 00:07:23.300 Nice. 00:07:23.300 --> 00:07:24.000 I have to look at that. 00:07:24.000 --> 00:07:25.100 No, I haven't heard of that one before. 00:07:25.100 --> 00:07:25.700 Yeah. 00:07:25.700 --> 00:07:28.020 It's really like that whole thing is a huge problem. 00:07:28.020 --> 00:07:30.760 And so people really like the... 00:07:30.760 --> 00:07:33.080 And that's where the idea came from. 00:07:33.080 --> 00:07:37.780 Like we ran or I ran like CI for a lot of companies or like at university before. 00:07:37.780 --> 00:07:39.780 And it's just such a hassle. 00:07:39.780 --> 00:07:45.180 It's just so painful to run it yourself and maintain that stuff when you actually want to do... 00:07:45.180 --> 00:07:50.160 Like you actually want to get productive stuff done and like work on your customers' projects or like your own product. 00:07:50.160 --> 00:07:51.520 And yeah. 00:07:51.520 --> 00:07:52.880 So that's where it came from. 00:07:52.880 --> 00:07:57.120 Instead of having everybody else like do the boring task of maintaining the test infrastructure, 00:07:57.120 --> 00:08:01.040 we take that for them and then do it much better for everybody basically. 00:08:01.040 --> 00:08:01.520 Yeah. 00:08:01.520 --> 00:08:02.480 That's fantastic. 00:08:02.480 --> 00:08:03.680 When did you guys get started? 00:08:03.680 --> 00:08:06.240 We started in 2010. 00:08:06.240 --> 00:08:09.340 We talked a little bit about continuous integration. 00:08:09.340 --> 00:08:12.500 Maybe we could dig into that a little bit more. 00:08:12.500 --> 00:08:14.520 Like what is continuous integration? 00:08:14.520 --> 00:08:15.620 What are the steps? 00:08:15.620 --> 00:08:17.160 And like what's involved in that? 00:08:17.160 --> 00:08:20.500 Continuous integration as a term is heavily overloaded. 00:08:20.540 --> 00:08:23.980 I think at this point, a lot of people when they talk about continuous integration, 00:08:23.980 --> 00:08:31.640 especially in like small to medium sized teams, are talking about automated testing and running your automated tests on a separate system. 00:08:32.500 --> 00:08:41.040 So I think that's really the main goal of making sure that whenever anything changes on your code base, whenever you push anything anywhere, 00:08:41.040 --> 00:08:52.620 that it's run by a separate system, a separate system that makes sure that everything works and notifies you that the whole build has been working fine and everything has been working correctly. 00:08:53.280 --> 00:08:55.860 That's the main goal of the CI system. 00:08:55.860 --> 00:08:56.460 Right. 00:08:56.460 --> 00:09:05.180 I think a lot of people initially, you know, back when you guys started around 2010, saw that almost as a, here's an automated compile step. 00:09:05.180 --> 00:09:13.720 Maybe we'll run a couple of tests, you know, depending on whether you're working for with a group that's actually written the test. 00:09:13.720 --> 00:09:14.000 Right. 00:09:14.860 --> 00:09:18.880 And people have been adding more and more to it over time. 00:09:18.880 --> 00:09:19.180 Right. 00:09:19.180 --> 00:09:20.680 Yes, absolutely. 00:09:20.680 --> 00:09:25.960 So I think that's, that's something where like, I think continuous integration can be seen as more than just running your unit tests. 00:09:25.960 --> 00:09:32.040 I think that's something where in the end, like the way I would define is the important part is that at every point, 00:09:32.040 --> 00:09:39.040 the team should know that your application works as expected for your customers. 00:09:39.040 --> 00:09:40.400 I think that's the whole goal. 00:09:40.400 --> 00:09:41.900 Like, and that can be unit tests. 00:09:41.940 --> 00:09:49.480 That can be like functional level tests that can be deploying into a staging environment and like testing against that staging environment just to make sure it works there. 00:09:49.480 --> 00:09:51.100 I think it's the end goal. 00:09:51.100 --> 00:09:56.680 I think it's the definition that we really like is less about like the technical aspect of continuous integration. 00:09:56.680 --> 00:10:01.740 And I don't really like defining the continuous integration as like a technical process. 00:10:01.740 --> 00:10:09.960 It's more of like this thing needs to make sure that whenever I run something, that I never want to change something in my, my system. 00:10:10.140 --> 00:10:13.820 I can actually do that and not break anything for my customers. 00:10:13.820 --> 00:10:16.960 It should be very customer or like user driven. 00:10:16.960 --> 00:10:22.540 Like we want to make sure that we don't break anything because that's, that's actually like, that's why we, we build software. 00:10:22.540 --> 00:10:25.160 Like we want users to use the things that we've built. 00:10:25.160 --> 00:10:27.440 We don't want to break things for them constantly. 00:10:27.440 --> 00:10:30.380 And we want to move fast, like move fast and break things. 00:10:30.480 --> 00:10:33.180 But even better if you just move fast and don't break things. 00:10:33.180 --> 00:10:33.620 Right. 00:10:33.620 --> 00:10:37.440 Maybe you break them, but maybe you don't send the broken pieces to your customers. 00:10:37.440 --> 00:10:38.780 Yes, exactly. 00:10:38.780 --> 00:10:39.120 Exactly. 00:10:39.120 --> 00:10:41.160 Get, get that information very quickly. 00:10:41.160 --> 00:10:51.040 And I think especially like break things maybe in terms of like features that you remove or when you want to change your product, but not really like break things as in show a 500 error page. 00:10:51.160 --> 00:10:53.540 Like that just shouldn't be something that, that actually should be happening. 00:10:53.540 --> 00:10:58.120 And I think it's, I mean, all of us have like experienced that all, all the time. 00:10:58.120 --> 00:11:08.540 Like things when, when features break, when things don't work as expected anymore, when like there's a downtime, just a drain on like the product. 00:11:08.540 --> 00:11:10.520 You don't like the product as much anymore. 00:11:10.520 --> 00:11:13.420 And it just doesn't feel good at that point. 00:11:13.420 --> 00:11:27.160 And I think continuous integration and the process of really making sure from a customer's perspective that your application works is usually important and can really put a lot of, a lot of stop basically to unnecessary work that your team does. 00:11:27.160 --> 00:11:27.960 And has to do that. 00:11:27.960 --> 00:11:31.440 I mean, we've all been in those situations where we broke something. 00:11:31.440 --> 00:11:34.700 And so now the whole team has to jump onto fixing that thing. 00:11:34.700 --> 00:11:37.640 And like, it takes so much resources from the rest of the team. 00:11:37.640 --> 00:11:42.480 If you can capture that early and make sure that it actually works and it, and there's no issue. 00:11:42.480 --> 00:11:45.320 It just, it puts a lot less stress on the team. 00:11:45.320 --> 00:11:47.400 There's a lot less overwork that we need to do. 00:11:47.400 --> 00:11:52.920 So that's, I think the really important part of automating all of this and just making sure it works. 00:11:52.920 --> 00:11:56.800 It basically lets you focus on the fun part of software development. 00:11:56.800 --> 00:11:58.000 Writing cool new stuff. 00:11:58.000 --> 00:11:58.080 Yes. 00:11:58.080 --> 00:12:01.600 Not fixing things that were not broken previously. 00:12:01.600 --> 00:12:02.840 Absolutely. 00:12:02.840 --> 00:12:03.400 Absolutely. 00:12:03.400 --> 00:12:12.940 There's no worse thing than regressions that come in where things that are working or have been working are now broken for no particular reason, just because there wasn't anything in place to catch that. 00:12:13.500 --> 00:12:16.720 And so that's, that's, it's really important to make sure that that doesn't happen. 00:12:17.140 --> 00:12:17.580 Yeah. 00:12:17.580 --> 00:12:17.580 Yeah. 00:12:17.580 --> 00:12:26.800 I've noticed a lot of really interesting cultural behavioral shifts that seem to be out of proportion with the effort of setting up continuous integration. 00:12:26.800 --> 00:12:36.840 So for example, I've seen teams that they release every three months or every six months, you know, to me, that sounds insane, but that's how they were working. 00:12:36.840 --> 00:12:37.080 Right. 00:12:37.080 --> 00:12:38.660 Because they wanted to test it. 00:12:38.660 --> 00:12:43.960 They wanted to make sure they wanted to sort of put it in a box and say, okay, now we're ready to send out this next thing. 00:12:43.960 --> 00:12:55.620 And, you know, maybe the next week somebody created some great feature, but well, that's going to come out in three months because it's just too much effort to go through the release test deploy cycle. 00:12:55.760 --> 00:12:55.960 Yes. 00:12:55.960 --> 00:12:59.900 To justify going through that just to get this one feature out. 00:12:59.900 --> 00:13:11.760 But if, if all of that becomes push a button or push to a branch and it just happens, well, maybe all of a sudden when that feature is tested, it's ready to roll and you just push it out. 00:13:11.760 --> 00:13:11.940 Yep. 00:13:11.940 --> 00:13:25.520 And so I've, yeah, I've seen teams that have sort of gone from releasing these big, painful, slow things like, hey, we're all going to come in Sunday night when no customers are there, shut everything down, make sure the integration stuff works. 00:13:25.520 --> 00:13:29.040 It's to just much more fluid work. 00:13:29.040 --> 00:13:31.520 And I think that's great for the software. 00:13:31.520 --> 00:13:33.720 It's also great for the people that have to do it. 00:13:33.720 --> 00:13:37.080 It's way more fun to see your work that you do come right out. 00:13:37.080 --> 00:13:37.240 Yeah. 00:13:37.240 --> 00:13:38.440 Absolutely. 00:13:38.440 --> 00:13:45.080 I think that the whole team productivity and team happiness is a huge part of like automating your, your whole workflows and everything around it. 00:13:45.080 --> 00:13:53.440 Because I mean, in the end, the best people that we have in our team, and if you want to build like a high class engineering team, they don't want to fix broken things. 00:13:53.440 --> 00:13:55.540 They want to build interesting, new and cool stuff. 00:13:55.540 --> 00:14:08.460 So if you have a process that is continuously like bringing down the happiness in the team and continuously having the team have to shift resources between like building new stuff, but then fixing old stuff, building new stuff, fixing old stuff. 00:14:08.840 --> 00:14:11.300 That's just going to drive the best people in your team away. 00:14:11.300 --> 00:14:15.080 And I think that's a really good way to drive the whole engineering team into the ground. 00:14:15.220 --> 00:14:32.240 So I think making sure that automation and how we release processes all really help to strengthen team happiness and putting those into like the perspective of team happiness is, is really crucial because I mean, and also that that's something that we've seen that we've seen with customers that we've seen in our own team. 00:14:32.240 --> 00:14:40.580 Like when the build, like when the tests get unstable or something like that, that's a huge drain on like team productivity, team happiness. 00:14:40.800 --> 00:14:46.940 And just, it takes so much mental energy from the team and it just, it just puts people in a bad position. 00:14:46.940 --> 00:14:50.340 And I think that it really, like, it's not just about the technology. 00:14:50.340 --> 00:14:51.960 It's not just about the customers. 00:14:51.960 --> 00:15:03.640 It's a big part of that is that, but it's also about putting it into perspective of how can I make my team happy by not having them to do any like menial, boring tasks that I really don't want them to do on a daily basis. 00:15:03.640 --> 00:15:08.740 So that's some of the things that I think are really important when setting up all that automation. 00:15:08.740 --> 00:15:20.500 And, and we've seen that ourselves where we have teams basically like their release, even small teams, like their release process was like a week of like going through Excel, like having Excel sheets that had manual test cases in there. 00:15:20.500 --> 00:15:29.500 And like that took a whole week, like that, that took, like they took their backend, like backend, main backend engineers and just had them run through that because they didn't have any more people. 00:15:29.500 --> 00:15:36.000 Like their way of parallelizing all of that was basically getting more people, more engineering people in and like clicking through the application. 00:15:36.000 --> 00:15:51.820 And I mean, that obviously doesn't scale and like, and they've, but, but I think it's, it's so often like that path is just so set in stone for some teams that, that it really has to like, they really have to get the help to like be taken out of it and like see a different path, how it can automate this. 00:15:51.820 --> 00:15:55.820 And I'm sure like the developers were aware of how this could be done differently. 00:15:55.820 --> 00:16:07.980 But I think if you don't put it in terms of like making the team actually happy and just see it as a, that's just more time spent by the team and like not see it as, Hey, if we do this like two more times, like these three people will just leave. 00:16:07.980 --> 00:16:10.220 They just, they just won't keep up with this anymore. 00:16:10.220 --> 00:16:12.980 I think it's a much easier sell to actually do this. 00:16:12.980 --> 00:16:13.380 Yeah. 00:16:13.380 --> 00:16:14.760 That's a really interesting point. 00:16:15.540 --> 00:16:16.340 It seems crazy. 00:16:16.340 --> 00:16:24.000 Like as soon as you said Excel, like I started to get a little bit of a headache and, you know, it was just like, you don't want to do that. 00:16:24.000 --> 00:16:24.220 Right. 00:16:24.220 --> 00:16:33.220 Like that is, that is not what people signed on for, but you know, some places they seem to develop this culture where that's how software is built. 00:16:33.220 --> 00:16:35.040 And, you know, it comes probably from a good place. 00:16:35.040 --> 00:16:36.640 Like we can't break things. 00:16:36.640 --> 00:16:41.240 You know, maybe you're a bank or something and, and if you're down, it's, you lose a hundred thousand dollars an hour. 00:16:41.240 --> 00:16:42.780 Like you obviously want to avoid that. 00:16:43.660 --> 00:16:50.160 But you know, things like continuous integration and continuous delivery and all that sort of stuff can really help there. 00:16:50.160 --> 00:16:50.960 Yeah, totally. 00:16:50.960 --> 00:17:01.420 And I think it's also like, it doesn't necessarily, as you mentioned, we actually like there's like we, we, we worked with teams where they had exactly that, that problem that they were contractually obliged to not release. 00:17:01.420 --> 00:17:07.860 Like at a specific point in day, like they, they cannot release from, I think it was like 10 in the morning to like 6 PM or something like that. 00:17:07.860 --> 00:17:11.840 There is just like in the contract, you cannot release at that time. 00:17:12.000 --> 00:17:13.120 Just, you're not allowed to. 00:17:13.120 --> 00:17:15.660 So there is no continuous delivery for them. 00:17:15.660 --> 00:17:20.240 Like that's just not, or at least not directly to production because that's, that's not what their business is doing. 00:17:20.960 --> 00:17:30.040 But what they, they've done is just simply released to a staging environment, like continuously have something that is pretty much exactly what is, what production is released there on a continuous basis. 00:17:30.200 --> 00:17:38.680 And then run like tests against that staging environment to make sure it actually works then so that they can release like during the night or in the very early morning hours. 00:17:38.680 --> 00:17:42.340 And actually know that, that this works and then they're fine with that. 00:17:42.660 --> 00:17:49.360 So I think there's different ways how to set that up, how to make that work, that, that teams can do and that the teams can use. 00:17:49.620 --> 00:17:53.240 but it's really about like, how can I make my customers happy? 00:17:53.240 --> 00:17:54.640 How can I make my team happy? 00:17:54.640 --> 00:17:59.060 And how can I get the most like productivity and focus out of my team? 00:17:59.140 --> 00:18:05.440 Because I mean, in the end that's, we're in a, we're now in a world with software where like everything is basically software. 00:18:05.440 --> 00:18:08.880 Like every company, like an Uber is taking over the tax industry. 00:18:08.880 --> 00:18:13.120 Netflix and Amazon are taking over like basically everything or like the TV industry. 00:18:13.120 --> 00:18:26.980 So like a lot of these industries were in the past, they had a much stronger, part in like the real world, to call it the real world, are not taking over by companies that are software companies that are just intermediaries between like different people selling stuff. 00:18:27.400 --> 00:18:38.960 And so everything is software competition moves a lot faster in all of those fields because it's just so much easier and cheaper like AWS and all those other cloud services make it so much cheaper to get started with things. 00:18:38.960 --> 00:18:41.480 And then we have so little talent. 00:18:41.480 --> 00:18:44.800 Like there's so few engineers generally out there. 00:18:44.800 --> 00:18:46.200 Like everybody's constantly hiring. 00:18:46.200 --> 00:18:56.980 Everybody's constantly trying to find their people that it's just crazy to spend time of your engineers on doing something that some workflow or automated system could, could do for you. 00:18:56.980 --> 00:18:59.180 Because it's like, you need to get faster. 00:18:59.180 --> 00:19:00.720 Like you cannot slow down. 00:19:00.720 --> 00:19:03.620 Like you're just not allowed to slow down if you're in the software world anymore. 00:19:03.620 --> 00:19:04.760 It's just not, it's just over. 00:19:04.760 --> 00:19:06.720 Somebody else will come in and just eat your lunch. 00:19:06.720 --> 00:19:19.940 This episode is brought to you by Hired. 00:19:19.940 --> 00:19:26.400 Hired is a two-sided curated marketplace that connects the world's knowledge workers to the best opportunities. 00:19:26.560 --> 00:19:35.580 Each offer you receive has salary and equity presented right up front and you can view the offers to accept or reject them before you even talk to the company. 00:19:35.580 --> 00:19:41.940 Typically, candidates receive five or more offers in just the first week and there are no obligations ever. 00:19:41.940 --> 00:19:44.020 Sounds pretty awesome, doesn't it? 00:19:44.020 --> 00:19:46.080 Well, did I mention there's a signing bonus? 00:19:46.340 --> 00:19:50.200 Everyone who accepts a job from Hired gets a $2,000 signing bonus. 00:19:50.200 --> 00:19:54.500 And as Talk Python listeners, it gets way sweeter. 00:19:54.500 --> 00:20:02.080 Use the link Hired.com slash Talk Python To Me and Hired will double the signing bonus to $4,000. 00:20:02.080 --> 00:20:03.780 Opportunity's knocking. 00:20:04.260 --> 00:20:07.400 Visit Hired.com slash Talk Python To Me and answer the call. 00:20:07.400 --> 00:20:18.120 Yeah, that's for sure. 00:20:18.120 --> 00:20:22.060 Speed is definitely a foundation of your business itself, right? 00:20:22.280 --> 00:20:23.380 You mentioned Netflix. 00:20:23.380 --> 00:20:31.940 Those guys have tremendous amounts of software driving and cloud deployments and whatnot driving their whole business. 00:20:31.940 --> 00:20:39.100 And they have the policy that as an engineer, you can release whenever you want. 00:20:39.100 --> 00:20:40.080 Yep. 00:20:40.080 --> 00:20:42.660 You decide, yes, it's done. 00:20:42.660 --> 00:20:43.260 Out it goes. 00:20:43.260 --> 00:20:44.960 And they have all the processes in place. 00:20:44.960 --> 00:20:51.820 So it's that kind of stuff you can sort of make much more doable with continuous integration and delivery, right? 00:20:51.820 --> 00:20:53.320 Totally agree. 00:20:53.320 --> 00:20:57.860 Yeah, I think having those processes in place where I think for once putting a trust in the engineers, 00:20:57.860 --> 00:21:04.060 but also having things in place where maybe you just want to release this to like 1% of your customer base and just to try it. 00:21:04.060 --> 00:21:05.860 And like give those tools to your engineers. 00:21:05.860 --> 00:21:10.880 And it's obviously like at Netflix scale, that's a lot more doable than it is on like other people's scale, 00:21:11.440 --> 00:21:19.580 where like you can divert like 1% of your customer, like your requests to that specific new release. 00:21:19.580 --> 00:21:25.460 But I think in general, like I think teams should really think through like how can we support our team in like testing new things, 00:21:25.460 --> 00:21:28.140 in experimenting with new things, with moving faster. 00:21:28.140 --> 00:21:34.980 And yeah, continuous integration and continuous delivery are definitely an important part there to make sure that you can actually release there. 00:21:34.980 --> 00:21:39.780 And on the other hand, that you don't actually break stuff for your customers and you just don't break that buy button. 00:21:39.780 --> 00:21:41.600 So customers can actually give you money still. 00:21:41.600 --> 00:21:42.580 That's right. 00:21:42.580 --> 00:21:53.340 I think the other thing that would really helps people have confidence moving faster and just going with it is the ability to undo something, right? 00:21:53.340 --> 00:21:57.680 If the ability to say, oh, that's a bad, something broke, right? 00:21:57.680 --> 00:21:59.060 We thought we had the system in place. 00:21:59.060 --> 00:22:01.360 Turns out not quite everything was caught. 00:22:01.360 --> 00:22:06.880 And just push a button and roll it back as well, rather than, oh, we're going to be down for a day if we break this thing. 00:22:06.880 --> 00:22:08.100 Yep, absolutely. 00:22:08.440 --> 00:22:10.580 You need to have a way to either. 00:22:10.580 --> 00:22:21.260 So what we've seen often with continuous delivery, once people move to a point where like you can release so quickly, is that when you release like small batches at a time, then oftentimes people just don't. 00:22:21.260 --> 00:22:23.240 They don't really roll back. 00:22:23.240 --> 00:22:27.840 But oftentimes they just push forward with like, because you just released something that is incredibly small. 00:22:27.840 --> 00:22:30.540 So if there's a bug out there, you find it immediately. 00:22:30.540 --> 00:22:32.520 It's typically pretty quick to fix it. 00:22:32.520 --> 00:22:34.680 You fix it and push that new fix out there. 00:22:34.960 --> 00:22:40.480 So that's something that we've seen a lot of teams go to in terms of the process where you definitely need a way to roll back. 00:22:40.480 --> 00:22:43.000 Like that definitely needs to be in place. 00:22:43.000 --> 00:22:48.980 But we've seen a lot of teams that then just push forward and like just deploy a new version very quickly. 00:22:49.340 --> 00:22:55.340 Because you only deploy so small things that it's really easy to find like what the actual problem is. 00:22:55.340 --> 00:23:01.160 You don't deploy like work of like three weeks where like you have no idea, like something breaks and you have just no idea where it happens. 00:23:01.160 --> 00:23:05.680 It's just a combination of like five different things coming together and then breaking your application. 00:23:05.900 --> 00:23:07.420 But it's just actually a small thing. 00:23:07.420 --> 00:23:16.360 So because you're pushing so often and so fast, you can actually push more often and faster because you can find stuff so quickly and so easily. 00:23:16.360 --> 00:23:17.800 Yeah, I think that's a really good point. 00:23:17.800 --> 00:23:23.240 You talked about releasing on shorter cycles and sort of catching bugs early. 00:23:23.240 --> 00:23:35.620 I think, you know, the way that people run into trouble and the way that deployments become not fun at all is you save all of your work for three months and then you put it out on the real system. 00:23:35.620 --> 00:23:37.520 And then you figure out what works and what doesn't. 00:23:37.520 --> 00:23:38.660 And you find these problems. 00:23:38.660 --> 00:23:43.340 You have to go back and like completely analyze the whole system. 00:23:43.340 --> 00:23:43.680 Right. 00:23:43.800 --> 00:23:45.320 You know, what did you do the last three months? 00:23:45.320 --> 00:23:46.680 Well, let's start here. 00:23:46.680 --> 00:23:51.740 And the opposite of if I've been working for an hour, I put something out right then. 00:23:51.740 --> 00:23:53.380 Well, you probably know exactly what it is. 00:23:53.380 --> 00:23:55.340 Just like you said, it's super easy to fix, right? 00:23:55.340 --> 00:23:56.280 Yep. 00:23:56.280 --> 00:23:57.020 Yep, totally. 00:23:57.020 --> 00:24:05.740 And I think that's even something even for teams that have those three month release cycles, like where like maybe the business for some reason can only ship every three months. 00:24:05.740 --> 00:24:13.220 But then you need to find a way where push it to a staging environment, push it to like a QA environment, like push continuously somewhere. 00:24:13.480 --> 00:24:23.660 And make sure that that somewhere runs correctly and have some like sanity checks, like write a few test cases that just go through the happy paths and like one sad path of that application. 00:24:23.660 --> 00:24:32.040 And yeah, so happy path and sad path is basically like a test that should work and a test that shouldn't work as expected and walking through the application. 00:24:32.040 --> 00:24:40.360 And if you just, even though, you know, you're just releasing this in three months, but you know, today, like all the changes that have been happening over the three months have been pushed somewhere. 00:24:40.360 --> 00:24:41.800 They've been tested somewhere. 00:24:41.800 --> 00:24:43.220 Somebody looked at those somewhere. 00:24:43.220 --> 00:24:44.540 On a continuous basis. 00:24:44.540 --> 00:24:47.740 So once the release comes, there is no surprise anymore. 00:24:47.740 --> 00:24:50.020 There is no, no, nothing new that's coming in. 00:24:50.020 --> 00:24:51.720 It just, it's all been running somewhere. 00:24:51.720 --> 00:24:54.060 And, and I think that's something that every team can do. 00:24:54.060 --> 00:25:00.900 I think that's, that's, that there might be like, maybe somebody has some exotic example where this is absolutely not possible for some reason. 00:25:01.160 --> 00:25:06.480 But I think for the vast majority of teams, that's something that should not be too hard to do. 00:25:06.480 --> 00:25:18.800 And I think most people should be able to do that in some way or another, just to get that feedback early as possible to like make that feedback loop as small as possible and, and, and get going there. 00:25:18.800 --> 00:25:26.180 I find usually when people say that it's not possible, that usually is like some kind of sign that something's not quite put together right in the app. 00:25:26.640 --> 00:25:27.160 Right. 00:25:27.160 --> 00:25:37.860 So there's no way to test this part because it calls into this thing, which goes into this whole infrastructure, which does that other, you know, maybe, you know, you need some, some way to break your app up. 00:25:38.100 --> 00:25:46.280 Some design patterns, dependency injection, things like this, that will actually allow you to break your app into the small pieces that you can test and should be testing. 00:25:46.280 --> 00:25:47.080 Agreed. 00:25:47.080 --> 00:25:50.700 If it's too complex to test it, then it's probably just too complex for anything. 00:25:50.700 --> 00:25:53.740 Yeah, that's, that's a great, great way to put it. 00:25:53.740 --> 00:25:57.640 I have my own opinions on this, but I'll ask, I'll get yours. 00:25:57.640 --> 00:26:02.980 Is there like a size of a team where you should start doing continuous integration? 00:26:02.980 --> 00:26:05.800 Like if there's only two of us, should we do this? 00:26:05.800 --> 00:26:06.460 Is it worth it? 00:26:06.740 --> 00:26:08.480 And is there an upper bound? 00:26:08.480 --> 00:26:12.740 Like, you know, if you have 200 people, it's like just too, too crazy, too complicated. 00:26:12.740 --> 00:26:17.460 Yeah, I think I'm, I'm obviously like totally biased in that regard. 00:26:17.460 --> 00:26:23.960 But I think, yeah, there is definitely, I think you should do CI on, on every level of, on every team size. 00:26:23.960 --> 00:26:28.160 I think it just, the approach to CI becomes different on, on different team sizes. 00:26:28.160 --> 00:26:35.780 I think that especially like if you're one or two people, like early product, just a proof of concept or something, 00:26:35.880 --> 00:26:41.940 you might just want to do something where you write like a few tests and then it depends on like, do you have customers or not? 00:26:41.940 --> 00:26:48.440 But in the end, like you, you don't want to break stuff for your customers, especially not your early customers, because they're the ones, even though they expect it to break. 00:26:48.740 --> 00:26:52.580 They're the ones that can be the most committed evangelist for your product. 00:26:52.580 --> 00:26:57.880 So if you break it for them, then you just lose basically your whole marketing team in the early days. 00:26:57.880 --> 00:27:04.680 Like the evangelist, the people that you, the outside people, the outside users and customers, your early users are your marketing team. 00:27:04.680 --> 00:27:07.440 They are the ones who are going to talk to everybody else about your product. 00:27:07.440 --> 00:27:09.340 So you don't want to break it for them constantly. 00:27:09.600 --> 00:27:21.340 So what you can do there, instead of writing like a million small unit tests, you just make sure that you have like 10, 15, 20, very high level tests that make sure like the most important workflows of your application aren't broken. 00:27:21.340 --> 00:27:26.160 Like, so people can sign up, people can like, whatever is the main business function of your product. 00:27:26.160 --> 00:27:27.080 People can do that. 00:27:27.080 --> 00:27:30.820 Like the main things and, and, and those can be easy to write. 00:27:30.820 --> 00:27:32.460 They, they can be on a very high level. 00:27:32.460 --> 00:27:43.320 So they don't break constantly, even though you're changing your application and they just make sure that like your customers can do at least the most important stuff on a constant basis or like the, especially early users. 00:27:43.320 --> 00:27:51.200 Like if some small feature on the sidelines gets broken at some point, like they'll miss it and they'll tell you, but it's probably not going to break everything. 00:27:51.200 --> 00:27:58.480 For them and they're probably not going to like leave the product or like leave the, the, the, like evangelizing for you that quickly. 00:27:58.480 --> 00:28:06.060 But you should write something where like at least the most important and main paths of your application are covered so that you can push faster. 00:28:06.060 --> 00:28:12.800 I think that's also something like in your early times, like you want to move so fast, like you want to push your product out the door as quickly as possible. 00:28:12.800 --> 00:28:20.680 And then again, like if you push something out the door that breaks other features and you don't know it and you like, you only know, like after a couple of days that it happened, 00:28:20.680 --> 00:28:36.020 and you don't know exactly what broke, like it takes so much time away from, from actually doing something productive that spending a little time on like writing good tests and writing tests from the right perspective of the user on a very high level actually makes a lot more sense. 00:28:36.020 --> 00:28:43.820 So I think the larger the team gets, the larger the code base gets, you want to like go lower in, in that stack in terms of like high level tests. 00:28:43.820 --> 00:28:52.880 So you go away from like, maybe in the beginning, you just write tests like with Selenium or something where you actually like, you just point a browser at your application and just click through your application automatically. 00:28:52.880 --> 00:29:13.040 So you actually know that like these things work on a relatively low level as well and not just on the higher level, but you still want to keep that higher level tested very well. 00:29:13.040 --> 00:29:16.060 So you know, because in the end, like that's what customers care about. 00:29:16.060 --> 00:29:23.040 Like a customer doesn't care if the unit test for a validation on a model works or doesn't like, that's not important to them. 00:29:23.040 --> 00:29:25.640 The important part is, does the workflow work? 00:29:25.640 --> 00:29:26.520 Does the product work? 00:29:26.520 --> 00:29:27.240 Does the feature work? 00:29:27.540 --> 00:29:30.220 So that needs to be tested in my opinion at first. 00:29:30.220 --> 00:29:43.600 And once you've tested like those few happy and sad paths on the upper level, then you can go like, okay, I've tested like that a validation in generally fires and, and, and error message gets shown on the web UI. 00:29:43.600 --> 00:29:50.280 So now let's, let me write like 10 more tests about like all these different validations that we run on the model before we save it. 00:29:50.280 --> 00:29:55.480 and, and before that works, because I don't want to test all of them on a very high level because that would be really slow. 00:29:55.720 --> 00:30:01.680 So you want to have that mix of like slow, very high level tests to make sure that from a customer's perspective, it works. 00:30:01.680 --> 00:30:12.480 And then lots and lots of like smaller level tests that verify specific things that you, you wouldn't want to verify on the higher level because it would just take too long. 00:30:12.480 --> 00:30:17.880 So that's at least the mix that I, that I found and that we found works pretty well. 00:30:17.880 --> 00:30:23.800 And that really captures and catches a lot of problems early on that, that might impact your, your customers. 00:30:23.900 --> 00:30:28.000 That's a really pragmatic and powerful way to look at things. 00:30:28.000 --> 00:30:33.480 I guess the way I would kind of summarize that is don't let the perfect be the enemy of the good. 00:30:33.480 --> 00:30:38.860 You know, don't let, don't let the goal for saying we must have a hundred percent test driven development. 00:30:38.860 --> 00:30:43.900 We must have 95% code coverage push you into a point where you say, well, but we have to go faster. 00:30:43.900 --> 00:30:44.820 We're just going to do zero. 00:30:45.900 --> 00:30:46.800 Yeah, no, totally. 00:30:46.800 --> 00:30:47.240 I agree. 00:30:47.240 --> 00:30:53.920 And I think that's, it's, it's a misconception, misconception that that makes you faster, that not writing test makes you faster. 00:30:53.920 --> 00:30:56.560 It just makes you, it only makes you faster. 00:30:56.560 --> 00:30:59.260 If you don't count the times where you have to actually fix things. 00:30:59.460 --> 00:31:03.040 Like if you do that, then it's just not going to make you any faster. 00:31:03.040 --> 00:31:15.600 It's just going to make you slower in, in my opinion or, and in my experience, like that's what we, the experience that we've had from, from the teams that I've worked in and from many, many, many different customers that I've talked to. 00:31:15.600 --> 00:31:18.680 And I've worked with that writing, you have to do it the right way. 00:31:18.680 --> 00:31:27.560 Like, and I've seen teams that have gone up from like early teams that have written thousands and thousands of low level unit tests, but their application still broke constantly. 00:31:27.560 --> 00:31:30.560 Like that's not really the right way to go. 00:31:30.560 --> 00:31:39.120 Like the, in my opinion, the right way to go is really see it from a, from a customer perspective and whatever the customer needs to make sure that it validates well and that it works well. 00:31:39.120 --> 00:31:40.680 That's what you should focus on. 00:31:40.680 --> 00:31:41.960 Yeah, that's a good point. 00:31:41.960 --> 00:31:55.040 And my rule of thumb is kind of when I, when I look at the part that I'm trying to test or think about whether I should write a test for it, I think is the customer actually paying me for this part of my application? 00:31:55.040 --> 00:31:57.380 You know, is, is this my core business? 00:31:57.380 --> 00:31:58.920 So like, is this the trading engine? 00:31:58.920 --> 00:32:11.500 If I work at like a, a stock based company or is this, you know, the continuous integration execution part, if I work at CodeShip or something like, does the, the, about our team page work or not? 00:32:11.500 --> 00:32:16.460 Like, you know, I don't really care and I'm not going to put energy into it, but if people are paying me for that part, right? 00:32:16.460 --> 00:32:18.980 My company, then probably that part should have a test. 00:32:18.980 --> 00:32:19.780 Yeah, exactly. 00:32:19.780 --> 00:32:28.140 And I think it can be simple as like, open up the page and see if it, if the page throws an error, like it doesn't have to be like, that's like for, for a lot of things. 00:32:28.140 --> 00:32:34.040 And that's something, for example, like when we deploy something new, like we deploy into a staging environment and just simply open up the application. 00:32:34.260 --> 00:32:43.320 And that captured a lot of different problems from, or like not a lot, but like from time to time, it captures something where maybe we've done something wrong in the configuration of the page and just the whole web app doesn't load. 00:32:43.540 --> 00:32:52.900 And it really pays off to just load one single website or like one single page of the application just to make sure like, does this thing actually boot when we push it somewhere? 00:32:53.100 --> 00:32:58.680 And that definitely captured a couple of times where like, we would have pushed something out that, that wouldn't have, that would have gone bad. 00:32:58.680 --> 00:33:00.020 And it didn't. 00:33:00.020 --> 00:33:02.940 And it was very easy to catch and very easy to deploy a fix for that. 00:33:02.940 --> 00:33:05.860 And, and it never saw, saw the light of a customer. 00:33:05.860 --> 00:33:08.220 But I think that putting that, that in place. 00:33:08.220 --> 00:33:16.360 And I think especially the about page is, I think a good example, because that's something like you wouldn't put a lot of effort into setting up an infrastructure to test your about page. 00:33:16.640 --> 00:33:30.500 So that should just something that should be there because you need it for many other things as well, like to, to just make sure that your about page is actually loaded and can actually be shown is something that shouldn't take more than a few seconds basically to implement in, in a test. 00:33:30.500 --> 00:33:35.300 But it's really critical because you are, you also don't want to like, you don't want your about page. 00:33:35.300 --> 00:33:40.820 Like if somebody looks up your website and the about page breaks, that's kind of like, that's not really a good sign. 00:33:40.820 --> 00:33:44.940 Like, especially for a tech product, like I would at least expect the about page to load. 00:33:44.940 --> 00:33:48.180 And I think that's some of the things where like, it should be easy. 00:33:48.180 --> 00:33:50.040 It should be fast to, to test that. 00:33:50.040 --> 00:33:52.500 Then there should be like a lot of different ways how you can do that. 00:33:52.500 --> 00:33:55.800 But yeah, your developers shouldn't have to spend a lot of time on, on doing that. 00:33:55.800 --> 00:33:56.720 Yeah, I agree. 00:33:56.720 --> 00:34:04.220 And like you said, just making requests to the page and seeing if a 200 rather than a 500 comes back actually can do quite a bit. 00:34:04.220 --> 00:34:05.940 Yeah, that catches a lot of stuff. 00:34:05.940 --> 00:34:23.560 This episode is brought to you by DigitalOcean. 00:34:23.560 --> 00:34:27.600 DigitalOcean offers simple cloud infrastructure built for developers. 00:34:27.820 --> 00:34:34.880 Over half a million developers deployed to DigitalOcean because it's easy to get started, flexible for scale, and just plain awesome. 00:34:34.880 --> 00:34:41.100 In fact, DigitalOcean provides key infrastructure for delivering Talk Python episodes every day. 00:34:41.380 --> 00:34:48.980 When you, or your podcast client, download an episode, it comes straight out of a custom Flask app built on DigitalOcean, and it's been bulletproof. 00:34:48.980 --> 00:34:58.480 On release days, the measured bandwidth on my single $10 a month server jumps to over 900 megabit per second for sustained periods, and there's no trouble. 00:34:58.480 --> 00:35:02.260 That's because they provide great servers on great hardware at a great price. 00:35:02.760 --> 00:35:10.780 Head on over to DigitalOcean.com today and use the promo code TALKPYTHON, all caps, no spaces, to get started with a $10 credit. 00:35:10.780 --> 00:35:28.540 In the early days, back in 2008, 2009, 2010, it was all about continuous integration and making sure that we could check in our code, 00:35:29.060 --> 00:35:36.120 that if we're using a compiled language, it compiles for writing tests, the tests run in an automated sort of way. 00:35:36.120 --> 00:35:43.180 But since then, people have been moving on to actually turning this whole system into actually a deployment mechanism, right? 00:35:43.180 --> 00:35:46.200 Like I can check in and magically stuff is in production. 00:35:46.200 --> 00:35:48.180 And that's called continuous delivery, right? 00:35:48.180 --> 00:35:58.380 The main idea with continuous delivery is that you keep your application in a state which either can be or is deployed somewhere constantly. 00:35:58.940 --> 00:36:02.600 And I think the important part that can be through CI, that can be by deploying it to a staging environment, 00:36:02.600 --> 00:36:06.560 but that can also be by just pushing it out there on a continuous basis. 00:36:06.560 --> 00:36:08.100 How would I go about that? 00:36:08.100 --> 00:36:13.960 Like I totally know how I check into Git and I set up a continuous integration server to pull that and run my tests. 00:36:13.960 --> 00:36:19.320 But how do I get that, say, into my, like if I wanted to do continuous delivery, 00:36:19.320 --> 00:36:25.220 how would I actually make changes to my server in AWS, for example, or DigitalOcean, something like that? 00:36:25.680 --> 00:36:36.840 So I think the important part there is that to take one quick step back and something that we find really important for continuous delivery in general is how you start triggering those changes. 00:36:36.840 --> 00:36:49.480 Basically, I think continuous delivery and the whole deployment part of continuous delivery really needs to be also fully automated and done by like the CI or continuous delivery system for you. 00:36:49.480 --> 00:36:58.720 Because in the end, like what you want, even like on a daily basis, like if you deploy all the time, you don't actually want your developers to think about like, okay, now I actually have to deploy this. 00:36:58.960 --> 00:37:06.580 Like what you want them to do is here is code and like I'm merging code into master and just do stuff to it. 00:37:06.580 --> 00:37:11.520 Like, but, but you want them to like focus on the code all the time, like on a, on a daily basis. 00:37:11.520 --> 00:37:19.520 You, you don't want anybody to have to think about like deployment strategy and how, like which script do I run in which way and where do I get which credentials, all that kind of stuff. 00:37:19.680 --> 00:37:25.180 So I think the important part and the way we see it is in terms of triggering it is through the repository. 00:37:25.180 --> 00:37:32.000 So we think the repository, is a really great way to capture the intent of releasing something. 00:37:32.000 --> 00:37:39.780 Like if you merge something from a feature branch into the master branch, like there is some intent to like test it, but also put it somewhere. 00:37:39.780 --> 00:37:48.960 So you can use that and just say, basically whenever you're on a specific branch, like run all the test, commands and then also release them to, to production. 00:37:49.100 --> 00:37:53.380 But at that point, like the developer can already be working on something completely different. 00:37:53.380 --> 00:38:03.700 and, and just is already jumped into the next task and, and works on the next thing, next thing, or, or gets a coffee or has a chat with somebody else on the team, but they don't have to actually think about this. 00:38:03.700 --> 00:38:06.440 I think that's, that's a core part of continuous delivery. 00:38:06.440 --> 00:38:10.780 because you really only want your developers to focus on code on a daily basis. 00:38:10.780 --> 00:38:18.520 That also helps with a couple of other things because it's easy to move between different parts of the infrastructure then because so for example, it doesn't matter. 00:38:18.520 --> 00:38:22.780 If you're working on like, if you want to release changes to your DNS system or your main web application. 00:38:22.780 --> 00:38:27.980 If all the things you need to do is merge code from a feature branch into the master branch and then it gets released. 00:38:27.980 --> 00:38:30.020 Like that's easy to understand. 00:38:30.020 --> 00:38:34.700 for anybody and anybody can go into any part of the infrastructure application. 00:38:35.140 --> 00:38:36.400 And, and actually do it then. 00:38:36.400 --> 00:38:39.400 And from then on, it depends, totally depends on like the specific technologies. 00:38:39.400 --> 00:38:49.240 If you want to use something like a platform as a service, like a Roku or an AWS elastic beanstalk or have your own servers, different ways, different tools, different abilities. 00:38:49.240 --> 00:38:58.080 But I think the important part is really capturing it in an automated fashion, but in a way where like it just happens kind of automatically for, for developers. 00:38:58.080 --> 00:39:01.920 Like they know it happens, but they don't have to consciously think about it. 00:39:01.920 --> 00:39:08.600 Like at some point, like that merge button on GitHub in a pull request is just like you're merging code and then you're off to the next thing. 00:39:08.740 --> 00:39:12.180 And you don't really actually think about the deployment at all at that point anymore. 00:39:12.180 --> 00:39:12.600 Yeah. 00:39:12.600 --> 00:39:13.840 That's, that's really cool. 00:39:13.840 --> 00:39:22.680 Deploying modern software can involve a lot of infrastructure and a lot of pieces, especially if you're doing like microservices, it can involve a whole bunch of them. 00:39:22.680 --> 00:39:23.160 Right. 00:39:23.160 --> 00:39:37.780 And so if you can automate that, even if it takes you a week to automate that, and normally it would take you a half hour to do a deploy, the speed that you get on the other side of that, the ability to, like you said, push a button, merge to a branch and just magically new software is out. 00:39:37.780 --> 00:39:38.800 That's, that's really cool. 00:39:38.800 --> 00:39:38.980 Right. 00:39:38.980 --> 00:39:39.360 Totally. 00:39:39.360 --> 00:39:45.540 I think that's, it's just more of the, I think it's more of that focus and productivity for your engineering team. 00:39:45.540 --> 00:39:48.020 Like that's something that is always important. 00:39:48.020 --> 00:40:00.060 It's really, really, really important that the engineering team can focus as much as possible on getting new stuff out there and building new features and product for, for customers and anything else that is in the way. 00:40:00.060 --> 00:40:05.760 And that is manual testing that is thinking about the release process in terms of like, how do I actually get this out the door? 00:40:06.320 --> 00:40:07.940 That's all wasted time. 00:40:07.940 --> 00:40:09.320 Like that just shouldn't be. 00:40:09.540 --> 00:40:19.800 And it's not just wasted time on like existing engineers who actually know that, but actually when you onboard a new engineer and you have to walk them through like every single way that you deploy all the different systems. 00:40:19.800 --> 00:40:23.420 It, that's just a huge time investment that you need to do. 00:40:23.500 --> 00:40:29.280 And, and on the other hand, if you just tell them like, yeah, open up a feature branch and like that, then just merge it into the master branch. 00:40:29.520 --> 00:40:36.220 And like, this is just a technology, like technologically, you, you know, like it's just a, I don't know, Django or a rails application. 00:40:36.220 --> 00:40:38.120 Like, you know how to work with that. 00:40:38.120 --> 00:40:44.080 Like our infrastructure might be a little bit different than what you've seen in the past, but you don't really have to worry about that at all. 00:40:44.140 --> 00:40:46.820 Like just do your code and like, just merge it in. 00:40:46.820 --> 00:40:51.800 And over time you'll learn more about like how actually the whole system works. 00:40:51.800 --> 00:40:56.020 And I think it takes away some of that stress of onboarding new people. 00:40:56.020 --> 00:41:04.340 And that, like that, that wish that a lot of companies have where like you, you have somebody new and like they should release something on day number one, which I'm not the biggest fan of. 00:41:04.340 --> 00:41:11.120 But at least in the first week, I think that it just puts a lot of pressure, unnecessary pressure on everybody to have them release on day number one. 00:41:11.120 --> 00:41:18.920 But I think it removes a lot of that, that, that, the pressure, like the problem with like showing somebody how that stuff actually works. 00:41:18.920 --> 00:41:30.720 And instead, like you tell them five minutes how the workflow works and then they couldn't do it anywhere from, as I mentioned, the DNS system to like your main application, to your back end application, to whatever. 00:41:30.720 --> 00:41:32.080 Yeah, that makes a lot of sense. 00:41:32.080 --> 00:41:37.960 You talked about earlier having different levels of tests and running them possibly at different times. 00:41:37.960 --> 00:41:43.860 Obviously, you would want to run your tests before you go through and push it through the continuous delivery path. 00:41:43.860 --> 00:41:52.120 Do you see people having a different set of tests for like a regular feature branch check-in versus this final step? 00:41:52.120 --> 00:41:55.340 Would people write, like run more tests if maybe their tests are slow or something? 00:41:55.340 --> 00:41:56.120 Sometimes. 00:41:56.120 --> 00:41:59.120 I think that's mostly due to time constraints, though. 00:41:59.120 --> 00:42:02.500 I think that's, and I think that's also something like that. 00:42:02.500 --> 00:42:04.520 I mean, that's one of the challenges that we try to solve for them. 00:42:04.520 --> 00:42:08.920 It's basically giving them a system that makes it so fast that they can run anything in parallel. 00:42:08.920 --> 00:42:23.520 But yeah, we definitely see that where people like they have a lot of like they run their maybe unit tests on every feature branch and run like their full on like integration or like functional testing suite only on like the master branch. 00:42:24.020 --> 00:42:25.560 This definitely happens. 00:42:25.560 --> 00:42:33.040 But yeah, I think so from what I've seen, I haven't seen a team that says I don't want to run it on like all the tests on my branch. 00:42:33.040 --> 00:42:36.360 It's more I can't run it there because it's just too slow. 00:42:36.660 --> 00:42:43.240 So I think it's a lot of it is just down to like optimizations in the test suite and like making it faster and making it more parallelized, all that kind of stuff. 00:42:43.240 --> 00:42:44.220 Yeah. 00:42:44.220 --> 00:42:49.140 If it takes two hours to give you feedback, even on your feature branch, it's maybe better to save it for later. 00:42:49.140 --> 00:42:51.600 So you know right away whether it's mostly good, right? 00:42:51.600 --> 00:42:52.880 Yeah, exactly. 00:42:52.880 --> 00:42:53.160 Exactly. 00:42:53.240 --> 00:42:54.680 And I think that's so what we've seen. 00:42:54.680 --> 00:43:00.040 I think like a team should get their whole build on like a feature branch. 00:43:00.040 --> 00:43:03.400 I'm not including deployments on a master, but just like running their tests. 00:43:03.400 --> 00:43:09.140 If you can get it down to like two or three minutes, that's below the level of like where people actually have to think about it. 00:43:09.180 --> 00:43:16.920 So when I was thinking about it, like as a developer, like I code something locally, I commit something locally, I push it into the repository. 00:43:16.920 --> 00:43:25.460 And if I get an answer, if that works or not in like two or three minutes or in less than that, I actually, I don't think about it actually again. 00:43:25.460 --> 00:43:27.220 Like I'm still coding on the next thing. 00:43:27.220 --> 00:43:33.080 And at some point, like a notification pops up that tells me, hey, everything worked on that thing that you just pushed or a notification pops up. 00:43:33.080 --> 00:43:34.680 It didn't work. 00:43:34.680 --> 00:43:37.820 And like you have to like fix something that you broke on that feature branch. 00:43:38.060 --> 00:43:43.820 But it's not like while I'm still working on the other thing, I'm not thinking like, hey, did this already finish or is this still running? 00:43:43.820 --> 00:43:46.360 I think that's like two or three minute like level. 00:43:46.360 --> 00:43:47.860 I think anything beyond that. 00:43:47.860 --> 00:43:51.340 Then the problem is that you're sitting there, you're coding on your stuff. 00:43:51.340 --> 00:43:54.180 You're thinking, hey, wait, did my build already finish? 00:43:54.180 --> 00:43:55.440 Let's just look that up. 00:43:55.440 --> 00:44:00.200 So you go to a browser, you open up the build, you open up the build page, and you've just lost a little bit of time. 00:44:00.200 --> 00:44:02.480 You lost focus from your development effort. 00:44:02.900 --> 00:44:12.360 And like over time, like if you have a large team and like a lot of people that have to do this constantly, that just is a lot of focus lost from actually developing. 00:44:12.360 --> 00:44:27.060 So I think we like our goal is really to get any team of any size to a point where you can actually run the builds in like this two or three minute time window where you're not actually actively thinking about the results of your build. 00:44:27.400 --> 00:44:36.300 And you just basically keep working and you just constantly keep working and you just get notified, hey, this thing worked and now this thing worked and now this thing worked without having to go back. 00:44:36.300 --> 00:44:40.280 And I think that's something that in my opinion, people really should strive for. 00:44:40.280 --> 00:44:44.100 Again, it's all down to productivity and focus for your engineers. 00:44:44.100 --> 00:44:50.680 Anything that takes even slight productivity and focus away from your engineers is something that actually hurts your bottom line. 00:44:50.680 --> 00:44:54.000 And so that just you have to get rid of that. 00:44:54.060 --> 00:44:58.740 Yeah, there's a lot of talk in programming about flow and focus and getting into the zone. 00:44:58.740 --> 00:45:05.220 And these things just can really kill that if you're like half still thinking, well, the thing that I did, is it actually okay? 00:45:05.220 --> 00:45:07.420 Or can I go forward assuming it's okay? 00:45:07.420 --> 00:45:11.040 Or, you know, do you know because the build finished that you're fine, right? 00:45:11.540 --> 00:45:12.180 Exactly, exactly. 00:45:12.180 --> 00:45:16.260 And if it just takes 15 minutes, like you're just going to come back, like you're just going to open up the page. 00:45:16.260 --> 00:45:20.360 And like then you open the page, you wait for the build to finish, like you open up Reddit or something. 00:45:20.360 --> 00:45:24.080 And like it's just like all that kind of stuff is just it's taking away that focus. 00:45:24.080 --> 00:45:27.540 And I think it's really important to keep that flow, to keep that focus going. 00:45:27.540 --> 00:45:27.980 Sure. 00:45:28.160 --> 00:45:32.660 So what's the longest or the craziest build you've seen running? 00:45:32.660 --> 00:45:39.600 Definitely people who were like, so where it's been over, like we ran it through the night. 00:45:39.600 --> 00:45:43.000 And at some point, like we just we just stopped because it just took too long, basically. 00:45:43.000 --> 00:45:45.500 I mean, there's definitely things where like it takes hours and hours. 00:45:45.740 --> 00:45:57.500 And I think a lot of it comes down to, especially with the teams that we work with, they're not like huge enterprise, like 48 hours straight, like running through like 100% CPU usage. 00:45:57.500 --> 00:45:59.580 It's often just comes down to optimization. 00:45:59.580 --> 00:46:05.720 I think oftentimes teams, they build their test suite and it gets progressively slower. 00:46:05.720 --> 00:46:09.540 Like no team starts out with like a two minute build. 00:46:09.540 --> 00:46:12.660 And then like three weeks later, it's like at an hour. 00:46:12.660 --> 00:46:14.400 Like it just happens over a longer time. 00:46:14.520 --> 00:46:18.620 Like it's two minutes, then it's three minutes, then it's five minutes, eight minutes, 15 minutes. 00:46:18.620 --> 00:46:21.160 And just it gets slower and slower and slower. 00:46:21.160 --> 00:46:26.200 And the team never really does the decision to like make this faster and look into it. 00:46:26.200 --> 00:46:34.100 So I've certainly seen teams where the tests have been running and they've been running slow, but they've only been running slow because they didn't really look it. 00:46:34.100 --> 00:46:42.460 And I mean, that has been the case with us, like a couple of like or a year ago, like one of our engineers joined and he's really strong on like optimizing test suites. 00:46:42.720 --> 00:46:49.220 Like that's, that was, he's been doing like for a while and he got our test suite down from 15 minutes to like three or something. 00:46:49.460 --> 00:46:52.460 And that's, and we're pretty good with like test suites. 00:46:53.400 --> 00:46:56.140 And I think that's something where like he just invested the time. 00:46:56.140 --> 00:46:57.920 He had the experience, knowledge about it. 00:46:57.920 --> 00:46:59.840 Obviously he then blogged about it on our blog. 00:46:59.840 --> 00:47:01.580 So we always try to keep that up. 00:47:02.720 --> 00:47:09.080 But that really helps like spending time on actually making your test suite faster and seeing where it's slow and how to make it faster. 00:47:09.080 --> 00:47:10.680 And actually like setting an SLA. 00:47:10.680 --> 00:47:15.400 Like I think as a team, you should set an SLA on your test suite. 00:47:15.400 --> 00:47:20.120 And if it just takes longer than that SLA, you just put resources on it to make it faster. 00:47:20.780 --> 00:47:27.800 Because otherwise, if you're just progressively letting it get slower and slower and slower, again, it'll just stop the process. 00:47:27.800 --> 00:47:31.280 Like continuous delivery really relies on your test suite being fast. 00:47:31.580 --> 00:47:36.240 Because if it's not, then your developers have to constantly wait for the results of what's happening. 00:47:36.240 --> 00:47:42.680 And there's nothing worse than having to wait on like the build result getting in before you can open up a pull request. 00:47:42.680 --> 00:47:50.480 And like you can ask other people for like feedback or maybe somebody has already looked into it and like given you feedback and done a code review. 00:47:50.480 --> 00:47:54.840 But then the build comes back half an hour later and tells you, wait, this actually failed. 00:47:54.840 --> 00:47:56.520 Like you have to recode it in some way. 00:47:56.620 --> 00:48:00.660 And then they do have to do another, go over it and do a code review. 00:48:00.660 --> 00:48:03.920 And like that just takes so much productivity away from you. 00:48:03.920 --> 00:48:04.180 Yeah. 00:48:04.180 --> 00:48:05.160 And wait again, right? 00:48:05.160 --> 00:48:06.860 Wait that 15 minutes again. 00:48:06.860 --> 00:48:08.120 Exactly. 00:48:08.120 --> 00:48:14.240 So like once you feel like, and I think having a strict rule of like our build should be that fast. 00:48:14.240 --> 00:48:16.800 And then dedicating resources to it. 00:48:16.800 --> 00:48:21.940 It's so much cheaper than like just waiting, like putting the resources on features and just waiting. 00:48:21.940 --> 00:48:24.460 Like it's not going to magically become faster. 00:48:24.640 --> 00:48:27.240 And it's just going to block your team like crazy all the time. 00:48:27.240 --> 00:48:33.820 And it's just so frustrating if the test suite is slow that you really should dedicate resources to it. 00:48:33.820 --> 00:48:35.380 It's just, it's developer productivity. 00:48:35.380 --> 00:48:36.940 Developer productivity is really important. 00:48:36.940 --> 00:48:40.460 Developers are really expensive and it's really hard to get new ones. 00:48:40.460 --> 00:48:47.280 So you really, really should make sure that your team is productive, your team is happy and can actually work on the most important tasks. 00:48:47.280 --> 00:48:52.920 And yeah, making your test suites fast is definitely something that pays off very quickly. 00:48:53.140 --> 00:48:54.320 Yeah, totally agree. 00:48:54.320 --> 00:49:03.700 When you're talking about continuous delivery, the putting the new version of the source code or the compiled bits on the server and making that run, it's pretty straightforward. 00:49:03.700 --> 00:49:07.840 But usually it gets really sort of tricky when you're talking about databases. 00:49:07.840 --> 00:49:08.520 Yep. 00:49:08.740 --> 00:49:09.800 What do you see people doing there? 00:49:09.800 --> 00:49:12.320 Putting a lot of it into services. 00:49:12.320 --> 00:49:18.920 I think that's something that's a relatively consistent theme that we've seen is like maintaining databases is really, really hard. 00:49:19.580 --> 00:49:28.220 I think continuous delivery is definitely something where you can't, whenever you store state, that's really hard to continuously deliver. 00:49:28.220 --> 00:49:32.380 And so I think a lot of teams, and that's certainly us as well. 00:49:32.380 --> 00:49:43.280 We just put that off to a service and we just let Heroku in our case or Amazon RDS deal with that because they have the infrastructure and the knowledge to actually do that. 00:49:43.280 --> 00:49:50.280 If you run your own data system, it's definitely possible to do that with the follower, like do some level of replication and then switching that over. 00:49:50.720 --> 00:49:52.460 But it's generally really, really hard. 00:49:52.460 --> 00:50:04.900 And the question is, whenever I look at like how often we deploy our web application and how often we would have to upgrade our database, it's not really that big of a deal to deploy. 00:50:04.900 --> 00:50:15.360 Like you don't really have to deploy like some infrastructure parts and parts where you store state that often compared to like the things that you that don't store state and that you can actually push constantly. 00:50:15.740 --> 00:50:24.400 So I think a lot of teams don't even have to really spend a lot of cycle on like, how do I actually continuously deploy like my database? 00:50:24.400 --> 00:50:35.100 Because oftentimes it's just fine enough to do it like on a Sunday morning and like do a one hour window there where you like just do a follower, switch over and then stop out the other one. 00:50:35.100 --> 00:50:40.600 So I think that's something where I think it's again, like don't let perfect be the enemy of good. 00:50:41.020 --> 00:50:50.240 And I think really focusing the continuous delivery on the parts that you actually can control very easily, mainly being like everything that you don't have stayed in makes sense. 00:50:50.240 --> 00:51:02.980 And then I think putting the other systems into like a service, putting it into like a Roku, into an RDS, at least for us, I think that's like, I'm not, I'm not in the business of maintaining databases. 00:51:02.980 --> 00:51:04.280 Like that's not my product. 00:51:04.280 --> 00:51:06.000 I just need a database. 00:51:06.000 --> 00:51:11.380 So if somebody else can provide a really good product around the database, that's totally worth it for me. 00:51:11.380 --> 00:51:12.880 And so that I don't have to do that. 00:51:12.880 --> 00:51:19.520 And again, put my developers on a much more productive for our customers, much more productive and for our business, much more productive path. 00:51:19.520 --> 00:51:23.200 So I think that's how we typically try to deal with it. 00:51:23.200 --> 00:51:35.900 But I think, yeah, for most companies, if you really, really have to do like zero downtime database updates, then you're, you're probably in a position where like you have enough resources to do that. 00:51:35.900 --> 00:51:47.640 Because if you, if you're really small, but you have to do like zero downtime database deployments, you're something on the business sounds a little bit off because that sounds really hard to do. 00:51:47.640 --> 00:51:51.780 Like if you have little customers and you're a small team, but the customers don't pay you a lot. 00:51:51.780 --> 00:51:59.880 But on the other hand, they, they still can require like really hard technical things like continuously updating or like zero downtime deploying new databases. 00:51:59.880 --> 00:52:00.980 Like that's not easy. 00:52:00.980 --> 00:52:05.660 Like then there's probably, you're not making, like you should, you should totally get more money from them. 00:52:05.660 --> 00:52:09.760 But on the other hand, I think that's, yeah, just don't worry too much about it. 00:52:09.760 --> 00:52:13.420 If you can just get away with a like Sunday early morning update. 00:52:13.420 --> 00:52:15.880 I think it's just, it's just really hard for small teams to do that. 00:52:15.880 --> 00:52:16.960 Yeah, it definitely is. 00:52:16.960 --> 00:52:20.380 So one of the things I saw you guys talking about is Docker. 00:52:20.380 --> 00:52:28.640 And I think Docker is going to become increasingly important in the whole way that we host and maintain and evolve our applications. 00:52:28.640 --> 00:52:30.080 What's the story of Docker? 00:52:30.080 --> 00:52:32.640 So generally I fully agree. 00:52:32.640 --> 00:52:46.300 I think it's, it's definitely a very important part of how we will build Docker and like containers and generally like very low level and easy to do virtualization is a big part of how we're going to build infrastructure in the future. 00:52:46.460 --> 00:53:07.880 So for me, a lot of the things that are important there are that when the system can maintain itself, it just, you, you easily push a container out that has everything in there that to actually run your application, which also makes that separation between developers and operations much easier because the developers can actually fully control what's actually running as part of their, their application. 00:53:08.080 --> 00:53:09.080 But it's still fast. 00:53:09.080 --> 00:53:09.420 But it's still fast. 00:53:09.420 --> 00:53:11.220 So that's really important. 00:53:11.220 --> 00:53:17.380 And on the test and continuous integration side, obviously, it's really key for us in the future. 00:53:17.380 --> 00:53:20.360 So we've just rebuilt our whole system basically to run on top of Docker. 00:53:20.820 --> 00:53:27.560 And, and, and the main idea there is that developers should be able to control their build environment fully. 00:53:27.560 --> 00:53:29.640 And each repository is different. 00:53:29.640 --> 00:53:33.120 Each repository and each application needs a different setup. 00:53:33.120 --> 00:53:36.560 And, and Docker can definitely provide that. 00:53:36.560 --> 00:53:38.560 And that's just not limited to like code chip. 00:53:38.560 --> 00:53:50.640 I think that's, many people are using it for many different ways in their, their test and, and, and CI setup, but it's really about giving the development team a lot more power about the infrastructure that the code is running in. 00:53:50.640 --> 00:53:57.280 and so that the operations team, like there's a clear separation between like how to run the whole container. 00:53:57.280 --> 00:53:58.820 Like that's just a clear interface. 00:53:58.820 --> 00:54:02.960 And then whatever's inside of it, the development team has full control over that. 00:54:02.960 --> 00:54:12.580 And I think that's like that separation of concern that you don't just throw code over the wall basically, and just let the operations team or somebody else deal with it, is really key. 00:54:12.580 --> 00:54:21.620 And then you can do a lot of, a lot more automation around it, like a lot more health checks, a lot easier way to like, run several instances of that, get it up and running again. 00:54:21.620 --> 00:54:26.400 When something goes down, really use the most of your resources, in your machines. 00:54:26.600 --> 00:54:36.120 So I think Docker and, and the way I think it's less about just the technology of Docker and like Docker as a company, although I think they're doing great and, and they're providing great, a great product. 00:54:36.120 --> 00:54:51.180 I think the more important part there is that it really enables, developers and the whole team to take full control over their environment and work in a different way where it's not just about like, let's update those three servers that we're running, but just. 00:54:51.180 --> 00:54:54.780 Let's push this artifact into like our environment. 00:54:54.780 --> 00:55:00.760 And then our environment decides where, when, and in how many items this artifact is running. 00:55:00.760 --> 00:55:08.920 So I think that's something that is, is really critical where you have a lot of more separation of concerns there where you don't have to think about everything all the time. 00:55:08.920 --> 00:55:16.420 And there's just a system taking care of, of a lot of this, and nice abstractions in place, to make this, this run and work. 00:55:16.420 --> 00:55:17.560 Yeah, that's cool. 00:55:17.560 --> 00:55:23.620 It seems like Docker and continuous delivery go well together because now you're almost able to deliver the infrastructure. 00:55:24.340 --> 00:55:25.440 As part of this, right? 00:55:25.440 --> 00:55:29.120 Here's the Docker container for this version plus my code, right? 00:55:29.120 --> 00:55:30.640 Yeah, absolutely. 00:55:30.640 --> 00:55:35.540 You just have a bunch of servers running and that can be in a cloud that can be as a cloud service that can be on your own infrastructure. 00:55:35.540 --> 00:55:40.280 And all that thing provides is just a way to run arbitrary Docker containers. 00:55:40.280 --> 00:55:45.140 And I think that's really powerful because again, like the development team can take full charge. 00:55:45.140 --> 00:55:52.740 Like no, no operations person has to know which version of Java runs in there and they don't have to be involved in our Python or Ruby or whatever. 00:55:52.900 --> 00:56:04.540 They don't have to be involved in like updating that or making sure that like all the different projects that are running as part of the company all need to use the same Java version because otherwise like stuff will break all the time. 00:56:04.540 --> 00:56:20.360 Like it totally frees every team, every repository, every developer basically to use whatever is best for that specific problem and not have to deal with like all different complexities of running application, like many different applications on the same kind of infrastructure. 00:56:20.360 --> 00:56:23.660 And virtualization in the past has definitely gone a long way there. 00:56:23.780 --> 00:56:27.080 But then it's just like, it takes a lot more resources to do that. 00:56:27.080 --> 00:56:32.960 And with Docker and containers, it just takes a lot less resources that the underlying technology has been, is proven. 00:56:32.960 --> 00:56:34.980 Like Google runs their operating system on it. 00:56:34.980 --> 00:56:38.440 Facebook runs, runs their operating, like their data centers on it. 00:56:38.440 --> 00:56:41.780 So I think we can say that this actually scales to whatever. 00:56:42.360 --> 00:56:44.880 I think we just need the tools to make it scaled down. 00:56:44.880 --> 00:56:47.660 Like we just need the tools to make it scaled down to like small teams. 00:56:47.660 --> 00:57:04.000 And I think that's what really Docker is providing a great tooling for that we can use containers for like a lot of different stuff, a lot of different workflows, a lot of different processes in our either production systems, but also in, in our whole build environment, in our whole build system. 00:57:04.000 --> 00:57:17.560 So I think that's where it's, it's really about control and giving the developers the utmost level of control about everything they're running as part of their application and not have other people either take away control or have to worry about that control at all. 00:57:17.560 --> 00:57:19.280 So I think you're absolutely right. 00:57:19.280 --> 00:57:25.900 I think Docker just moves a little bit more of that control back to the software team. 00:57:25.900 --> 00:57:30.580 And by virtue of that moves it into this continuous integration story, right? 00:57:30.580 --> 00:57:46.800 So we can build out these Docker containers and I can say, you know, look, if we're going to switch from, let's say Python 3.4 to 3.5 and use some of the new features there, I don't have to make sure I coordinate with the infrastructure team to make sure that the right versions of Python are on the servers we're going to push to. 00:57:46.800 --> 00:57:52.340 When we do continuous delivery, we just say, well, we're just going to push a different version of the container and boom, it works, right? 00:57:52.340 --> 00:57:53.320 Yeah, exactly. 00:57:53.320 --> 00:57:57.880 And on different branches, maybe like on a different branch, you want to test against the new version of Python. 00:57:58.020 --> 00:58:04.900 Because like on the master, you're still running against a different version of Python, but on the feature branch, because you want to update, you're running against new version there. 00:58:04.900 --> 00:58:09.060 So you want the whole build environment and the whole deployment environment to run on a different version. 00:58:09.060 --> 00:58:13.260 So nothing other than that, than just changing it in a config file and using a different one. 00:58:13.480 --> 00:58:24.020 That's a really interesting component to it as well, because, you know, as you move forward, you're sort of moving forward in time on your infrastructure and all the other pieces as well on your, your branch. 00:58:24.020 --> 00:58:34.500 And so if you need to go back one month and run code from back then, if maybe solve a bug that somebody reported on a different version of your code or something, you, you know, to really do that. 00:58:34.600 --> 00:58:39.140 Genuinely, you have to like somehow roll back your system to that level, right? 00:58:39.140 --> 00:58:47.000 But if, if the Docker container specification is part of that, that check-in history, you can literally roll back to what you built it on, right? 00:58:47.600 --> 00:58:48.320 Yeah, absolutely. 00:58:48.320 --> 00:58:50.000 You can have the exact same. 00:58:50.000 --> 00:58:56.220 I think that's, I mean, and as I mentioned before, like that's, that's why we built a new system on top of Docker to be able to exactly support that. 00:58:56.220 --> 00:58:57.080 Like that should be easy. 00:58:57.080 --> 00:59:03.200 Like you should be able to run, to roll back to like your build workflow, your code, your environment that you built, like everything. 00:59:03.200 --> 00:59:11.760 You should be able to roll back everything and like have a complete history of, of everything that happened as part of your, your build and even production system. 00:59:12.260 --> 00:59:24.300 Because only then, like, as you mentioned before, like if you want to do a rollback, if you want to fix something, if you want to look into something, debug something, there isn't really any other way to do it than to be able to go back historically and like really look up what exactly happened there. 00:59:24.300 --> 00:59:24.940 Yeah. 00:59:24.940 --> 00:59:25.120 Yeah. 00:59:25.120 --> 00:59:25.540 Very cool. 00:59:25.540 --> 00:59:28.300 So yeah, we'll see more Docker in the future. 00:59:28.300 --> 00:59:28.700 I'm sure. 00:59:28.700 --> 00:59:30.860 I'm very sure. 00:59:30.860 --> 00:59:31.660 Yeah, definitely. 00:59:31.660 --> 00:59:33.020 Okay. 00:59:33.020 --> 00:59:35.240 So we're getting kind of to the end of the show. 00:59:35.240 --> 00:59:40.400 Do you have a final call to action or something you'd like to make sure people go out and try and do? 00:59:41.360 --> 00:59:48.660 So two things on, on, on, on, on the one side of obviously if, if people are interested in like a Docker based, CI system, give us a call. 00:59:48.660 --> 00:59:51.680 we're happy to show you guys, what we're doing there. 00:59:51.680 --> 01:00:04.800 But I think on a, on a non coaching basis, I think we really, really, really need to need to make sure that as a community, we focus more on the productivity and focus of our engineering teams. 01:00:04.800 --> 01:00:07.920 Like that needs to be number one, like customer success, obviously. 01:00:07.920 --> 01:00:14.240 And then, but that is driven and done through productivity and focus of, of our engineering team. 01:00:14.240 --> 01:00:20.460 So I think that anything, and we deeply need to analyze on a daily basis, like how is our engineering doing things? 01:00:20.460 --> 01:00:21.800 What can we change? 01:00:21.800 --> 01:00:22.740 What's in their way? 01:00:22.740 --> 01:00:24.240 How can we improve the processes? 01:00:24.240 --> 01:00:25.620 What can we automate? 01:00:25.620 --> 01:00:28.080 How can we automate operations more? 01:00:28.080 --> 01:00:32.940 How can we automate automatic health checks so that there's no downtimes and no issues or problems there? 01:00:32.940 --> 01:00:41.540 Like how can we do all of that, so that they can actually focus on building stuff for our customers that actually makes us money and is interested in new technology. 01:00:41.540 --> 01:00:45.200 So I think that's something that is, is really key and really important. 01:00:45.200 --> 01:00:57.820 And that would be my main takeaway for, for people like really, really focus on those processes to make sure that your team can actually focus as much as possible on to building stuff for your customers. 01:00:58.160 --> 01:00:58.920 Yeah, that's great. 01:00:58.920 --> 01:01:06.600 I really love the focus on team happiness because I think it often gets lost in the technical bits of this story, right? 01:01:06.600 --> 01:01:08.360 Yes, absolutely. 01:01:08.360 --> 01:01:10.600 Flo, it's been great talking to you. 01:01:10.600 --> 01:01:11.680 Thanks for being on the show. 01:01:11.680 --> 01:01:13.300 Yeah, thanks for having me. 01:01:13.300 --> 01:01:13.720 It was great. 01:01:13.720 --> 01:01:14.560 You bet. 01:01:14.560 --> 01:01:14.960 Talk to you later. 01:01:14.960 --> 01:01:18.320 This has been another episode of Talk Python To Me. 01:01:18.320 --> 01:01:20.180 Today's guest with Florian Motlick. 01:01:20.180 --> 01:01:23.380 And this episode has been sponsored by Hired and DigitalOcean. 01:01:23.380 --> 01:01:25.300 Thank you both for supporting the show. 01:01:25.300 --> 01:01:27.840 Hired wants to help you find your next big thing. 01:01:28.000 --> 01:01:36.540 Visit Hired.com slash Talk Python To Me to get five or more offers with salary and equity presented right up front and a special listener signing bonus of $4,000. 01:01:36.540 --> 01:01:42.100 DigitalOcean is amazing hosting blended with simplicity and crazy affordability. 01:01:42.100 --> 01:01:48.100 Create an account and within 60 seconds, you can have a Linux server with a 30 gig SSD at your command. 01:01:48.100 --> 01:01:49.940 Seriously, I do this all the time. 01:01:49.940 --> 01:01:51.660 Remember the discount code too. 01:01:51.660 --> 01:01:53.880 It's Talk Python, all caps, no spaces. 01:01:54.400 --> 01:02:00.520 You can find the links from today's show at talkpython.fm/episode slash show slash 38. 01:02:00.520 --> 01:02:03.100 Be sure to subscribe to the show. 01:02:03.100 --> 01:02:05.460 Open your favorite podcatcher and search for Python. 01:02:05.460 --> 01:02:06.620 We should be right at the top. 01:02:06.620 --> 01:02:10.300 You can also find the iTunes and direct RSS feeds in the footer of the website. 01:02:11.260 --> 01:02:15.460 Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix. 01:02:15.460 --> 01:02:18.420 You can hear the entire song on talkpython.fm. 01:02:18.420 --> 01:02:20.980 Just look for music in the nav bar. 01:02:20.980 --> 01:02:23.080 This is your host, Michael Kennedy. 01:02:23.080 --> 01:02:24.620 Thank you so much for listening. 01:02:24.620 --> 01:02:26.900 Smix, take us out of here. 01:02:26.900 --> 01:02:47.780 I'll see you next time. 01:02:47.780 --> 01:02:48.280 you 01:02:48.280 --> 01:02:48.940 I'll put it in. 01:02:48.940 --> 01:03:18.920 Thank you.