Zero to production ready: a best practices process for Docker packaging

Hello, my name is Itamar Turner Charing, and today I'll be talking about Zero to Production Ready, a best practices process for Docker packaging. And you can learn more about me, you can learn a whole lot more about Docker packaging. for production at pythonspeed. com, my website. And so Docker packaging is quite complicated, quite complex, uh, for two reasons. And the first reason is that Docker is at the intersection of 50 years of technologies , starting from Unix in the 1970s, so some of the signal handling issues that are involved in Docker packaging go back to designs from the 70s that Linux then borrowed.

Networking, TCP/IP comes from the 80s. There's Python. All these technologies then fed into Docker, which was created in the 2010s, and then all our packaging tools Don't even want to talk about the twenty twenties. But all of these different technologies built up over literally almost fifty years all intersect in one place in Docker packaging. And so that just inherently makes it a complicated task. The second reason the Docker packaging is complicated is the Docker packaging is also at the intersection of multiple organizational processes, multiple processes involved in writing and deploying your software. So whether you're writing or testing or deploying or getting feedback from

production all of these different uh processes and systems, they all in some sense interact with how you do packaging. And so that again makes Docker packaging more complex. And uh ongoing theme about in this talk is that you need to be thinking about not just the specific tools and technologies you're using, not just the configuration files, but also the bigger picture processes. the way you interact with your software, the way you maintain your software, the way you deploy your software, all of these different processes are going to intersect with your packaging as well. And so as a result of this complexity, there's just a massive amount of material to cover if you're talking about best practices for Docker

packaging for production. And this talk is not even an hour. I've got a list of 70, it's probably up to 80 packaging best practices that I've been accumulating over the time. At the end of the talk, I will give a link to my website where I document many of them. If I was doing this as training, like it would take a day and a half and I wouldn't even be able to cover all of them. So we can't actually learn about all of these best practices in a single talk. It's too much to cover, too many different things, just can't be done. But what we can do is talk about the process. Because there are so many different uh best practices to apply. It can start become difficult to know where should you start, what should you prioritize, what to do first, what to do second.

And so for example, it is not uncommon for you to look at your Docker image and say Oh no, my Docker image is two gigabytes. This is terrible. And then you spend a day making it much smaller, and that's great. And then you get interrupted and you have to move on to your next task because something's on fire at your job and you have to go deal with that. And you've neglected to think about security of your Docker image. And really Security is probably more important than image size, although you should try to make your image smaller, and we will talk about that too. So because you have a little bit of time, you might be interrupted at any moment. What you want to do is have a process for taking your application from no Docker image at all to

a, you know. the ideal perfect Docker image, we want it to be an iterative process. You want to start with the most important parts, you want each step to build on the previous one. You want to to have a process where if you're interrupted halfway you've prioritized the more important parts and you're okay taking a break and working on a different part and coming back to it later. And so for the rest of the talk we'll be covering a process. And the process goes something like this. First you get something working. If you don't have a working Docker image, it doesn't matter if it's secure, if it's small. In fact, it's easier to make a small image if it doesn't actually run any code. So you need to have, as a very first step, you need to run your application.

If you're not running it, it's useless. Once it's running, you can start thinking about security. Without security, you can't run it anywhere publicly. And then you're going to start running and build doing automated builds. You're going to want multiple people to be building images. Then you're thinking about how it's going to run in production. How do you make it easier to debug? Just as more and more people are using it, as you do more and more deploys. As time passes, you need to start thinking about reproducible builds because your dependencies are going to change. And then finally it's time for optimization, so optimizing for developer time by having faster builds, and just optimizing developer time and resources and bandwidth by having smaller images. And so this process is fairly generic. In some cases you might want to do things in a different order.

As you become more of an expert in this process, you might end up doing something from stage 5 and stage 1 because you know you have to do it. And so this is sort of a general guideline, but it's a good starting point, which should work well for most people in most situations, and then you can customize it for your particular needs. So the first step is getting something working. And again, if you don't have a working application that you can actually run inside a Docker container. Then none of the other steps, the security, the performance, none of it matters if your application can't run. That's the goal. That's what you're trying to do And so this is an extremely simple Docker file, just an example of where you might get started. So you choose a bit

you use from Python 3. 9 Slim Bullseye. So you're using, we'll talk about this a bit later, but Basically, you're using the quote official Docker image for Python based on Debian stable for Python 3. 9. Copy all your files in install your code and then you run your server and that's it. And there are some best practices here that are more like requirements because you need them to get things working. We'll give some examples next. And more broadly, for each of the uh six steps in this process, we'll start by sort of introducing it. I'll give some examples of best practices and keep in mind these are just examples. There are actually many more best practices I don't talk about either because they're more specialized or just because just

as I said we don't have time And finally, and occasionally I will also talk about how you should think about this in the terms of the bigger processes of your organization development, deployment, and so on. Because again, a lot of this is not just about writing the Docker file or the scripts, it's also about um ensuring you have organizational processes in place. So guessing working. Here's some and next we'll look at some examples of things you need to do. So, you know, you're running a Django application, uh just it's a web at web server, you want uh browsers to be able to talk to it. And so you want to make sure that it is accessible from the network, from the outside network. And if you listen on localhost

127. 001 , That will prevent clients from talking to the con uh the web server running inside the container. And the issue is that 127. 001, the local host the loopback interface is specific to a specific machine. And a container is basically pretending, up to a point, to be its own little machine with its own little networking stack. And so if you listen on 127. 001 inside the container, that's the container's local host, which is of course different than your regular computer's local host, which is different than some other computer's local hosts. So if you want to make sure your accessible, your your server is accessible from the outside, you want to listen on 0.

0. 0. 0, which basically means all the interfaces and that will include the external IP address of the running container. And if you do that, you'll be able to connect to your web server. This is the common networking failure that you need to make sure you configure correctly for your container to work. Another example, and this one's more of a best practice and saying that's strictly necessary, it is not uncommon to use shell scripts to Configure your your Docker container. For example, you might have an entry point script, the script that sort of starts everything up And the problem with shell scripts is that they're really quite broken by default. For example, if there is a bug in your Python script and you have a typo And you use a variable that doesn't exist, your

Python program will not keep running. It'll throw an exception, a name error exception. And then you'll get you'll know there's a bug and then you can fix it. In a shell script, if you have a typo and you type in the name of command wrong, by default the shell script will just keep running. It'll say unknown command and then it'll keep going Similarly, if you use an undefined variable, it'll keep going. And so by setting these threading these three commands at the beginning of each of your shell scripts, you can also do them on one line. We'll see an example later. By doing these three uh commands, set minus e, set minus u, and set minus o pipefill, you make shell scripts a little less broken. which means they're more likely to work and it is if you have a large shell script it is difficult to add these later.

So you want to do this from the start. And more broadly, if your shell script is getting more than trivial, you might want to rewrite it in Python. Just have a script Python script that just starts everything up and then launches your actual application. So you've gotten your con Docker container basically working. It can start up your application. Great. What's next? So the next step in most cases is to make it more secure. A container that whose security you don't trust is not something you can deploy publicly anywhere. You want to test it out even uh in some restricted way, you want to know that you have some basic security. And if you're gonna be deploying it to your production system, you really want to know that it's secure.

Without security, it doesn't matter how good everything else is, you're gonna feel uncomfortable deploying it. So that's going to be the next step. And so let's look at some examples of what you can do to make your Docker image more secure. So, one thing you can do is make sure that your container does not run as root Now, a container is, as we said, is a bit of an i sort of an isolated system within your operating system. It pretends that it's its own little different computer as much as it can. So it has its own users, its own little file system, its own network stack. So it is in many ways separate that from uh the host machine that's hosting it And root inside a container is somewhat more restricted than root on your main machine.

So running as a root, which was what most uh Docker containers do by default, is not quite as bad as running your application as root on your host machine. Nonetheless, by running as root, you are giving an attacker a larger attack service. And so if someone takes over your process remotely, If you're running as root in the container, it'll be much easier for them to take over the host machine than if for if they are running as a non root user And so in general what you want to do is, once you've installed all the different , if you 're installing some Debian system packages like a Debian Red Hat, once you've done all that. What you want to do is create a new user Unix user in your Docker file and then use the user command in your Docker file to say

from now on all commands should run as this new user. And then both run commands and the entry point script will run as that new user. And the result of this is that Uh if someone does take over your uh running process, it'll be harder for them to escape and move into the um outside host it'll be harder for them to take over the whole machine and that's just you know an easy result to get from just adding four lines to your Docker file Another uh security thing you want to do is install updates to the system packages. Now, in to some extent the base images

provided, for example, by Python from for Python by Docker, uh they do get updated every few weeks typically, uh so they will over time get security updates. But it's not always immediate. Uh and so that means is there might be a gap of a few weeks where some critical security update was released by Debian just the base Linux distribution in this case, or released by Red Hat or whoever. And that security update won't be on the Docker image you're using. So when you do from Python 3. 9 Slim Bullseye, or using a Debian image, you want to make sure that all the security updates from Debian are actually installed. And so the first thing you want to do And your NACRO file is probably run up to getUpgrade, or the Red Hat equivalent, in order to install the security updates so you're not

uh running your software without all the security patches that you need to be secure. So you know you don't want to be running a version of OpenSSL that has a remote exploit in it that will let someone take over your computer. And so taking a step back and thinking about security as a process, uh a Docker image is an immutable artifact. The idea is that once you've created your Docker image, you don't update it. And so security updates are going to require a new image. We'll repeat this again when we talk about automated builds next. And so you need an ongoing process to know that security updates have happened.

Like, do you know when Django has a security update? How will you know? Do you have some process automated or organizational to make sure that happens? Once JaneGo has a security update, you're gonna have to rebuild your image. update your image to use a new version of Django and you have to rebuild your image, then you have to redeploy your application. And this is a thing that you need to, a process you have to build, a process you have to think about. And so security isn't just about doing some configuration files. It's also a thing that has to be integrated into your development processes. It's a thing you have to plan for. It's a thing you have to do in an ongoing basis even after you've created your initial Docker image. So that's security. So so far we've gotten our Docker image to the point where it runs our application.

That's great. We've done some steps to make it more secure, so we have some confidence that it doesn't have obvious security vulnerabilities, at least from the perspective of packaging. And so the next step is to do automated builds You don't want to have to manually build your Docker image each time. If it's more than just you working on the application, you want other people in your team to be able to build the images. And so the next step is to make image building automatic and integrate into your build RCI system. And so the idea is every time you do maybe a pull request, maybe every time you do a push. to your main branch, you will run the tests. And here we can see a little script that's going to do that. So we run the tests , we build an image, and then we push it to an image registry, the place where we're going to store the images.

And when you are further along, you might also at this point run some integration tests and maybe automatically deploy the new image. So not just do continuous integration, do continuous deployment, where whenever you merge into the main branch, things get deployed and go live within 20 minutes or what have you. And so let's look at some examples of automation stuff, best practices you might want to do now that you're working on automating your builds. So if you're more automating your builds, this is a good opportunity to also automate looking for security vulnerabilities. So here are some tools that you might want to use. There might be others

Bandit is a Python code analyzer. So not really Docker specific or packaging specific, but good thing to run. You can run it on your code base and it'll find uh use of pickle, uh, it'll look for SQL injection attacks. So just a good way to analyze your code and look for potential security movies. Safety is a command line tool that will look at your dependencies and it has a database of known security issues and it'll tell you, oh, you're using this version of requests that has a node security bug you should upgrade to a newer version of secure of the requests package. And if you're using uh let's say GitHub, GitHub has this built in as a service. So you can go to your GitHub settings for your uh repository and it will uh

do security scans and like it can even do pull requests to update your dependencies to newer versions. And Trivi is a more Docker-specific tool. You could point it at a Docker image and it will scan the system packages. You can scan for JavaScript packages. You can also do Python packages increasingly, like they're working on it. And it'll tell you about node security issues. So you might say, oh, the version of OpenSSL in this Docker image is insecure, you should update OpenSSL. So another thing you might want to automate is the way you name your Docker images. So it's not uncommon for different developers to be working on different branches for different features.

So imagine you have issue one, two, three. If someone filed a bug, you want to add more Cowbell. And so a developer creates a branch, one, two, three, more cowbell. And they do write some code in that branch and then they open a pull request. And so you want to build a Docker image based on that branch. But you don't want the Docker image for the CowGill branch to mistakenly overwrite the Docker image for your production branch. You don't want to, if you have, especially if you have automatic deploys, you don't want to automatically deploy feature branches until they're actually merged. And so what you can do to deal with this is you can name your Docker images based on the Git branch. And this is what this script does. So

this script basically says we're gonna call this command git rev parse abbref head. And if you're wondering how I got it, I got it off of Stack Overflow, because that's how you learn how to use git. So this command will get the current branch of the code. Then we build the image and we build the image the part after the colon, which is known as the tag. uh is based is the same as the git branch. And so you have your image colon main and you can have your image colon one to three more cogbell. And then you push that image. And so now your um GitHub registry, you have all these different variants of your image and each one is per branch and they don't stomp on each other and if you need to test your branch you have a Docker image for your branch.

So, another example of a best practice that you want to do as part of automating your builds, remember we talked about how Docker images are immutable. And so once you've created one, it doesn't change. We also talked about how you need to install security updates, your system packages. And so the combination of these two things means you need an ongoing process where let's say once a week, or maybe in response to each security vulnerability, you rebuild your image from scratch and then you redeploy it. And because Docker 's caching, uh it's very easy to mistakenly when you rebuild your image to use the cached version, then you won't actually get the security updates. So when you rebuild it, you have to rebuild it without caching.

And then by using dash-nocache, dash-pull, and then push your image and that will ensure that so you're rebuilding it from scratch that ensures that the app get upgrade actually runs that you get the security updates and then you can redeploy. Another example of something you might want to do as part of CI as part of automation. Um Docker has this feature, as I mentioned, where it when you rebuild an image, it'll cache um the uh image locally and then we rebuild the image it'll say oh your image is unchanged I don't have to uh in this in this part of it like the pip install requirements at text

will say oh requirements at text is unchanged so I don't have to reinstall the pip install I don't have to rerun the pip install I can use a Docker layer cache the rebuild will be faster And so when you rebuild, your rebuild will be faster. But that only happens if the image is locally in the cache. So if you're doing development in your local machine, this works just fine. But if you're doing automated builds in CI, CI, especially if you're using cloud CI like GitHub Actions or GitLab, GitLab CI. Each run is in a new Docker image or maybe a new virtual machine. That means the cache is empty. And so even if you could rebuild faster, you won't because your cache is empty. So you want to do is rebuild the cache. And so should ignore this just as a little

typo. And so what you want to do is. When you run in CI, before you rebuild your image, you want to do a Docker pull to download the previous version of your image. And the or true says that if there doesn't it's on the cache at all, it's the first time you're running it, it'll just won't break the build. And then when you run the build, you do a dash-dash cache from your image. Um and what that means is When it rebuilds, it'll rebuild to the warm cache and rebuilds and CI will be much faster. In practice, if you're using something like BuildKit, which is a new Docker build system, there's actually some more command line options you want. So um You'll probably want to look at my website for full instructions on how to do this. And again, as a reminder, at the end of the talk, I will give you a link to my website where I have much more details about all these best practices, or most of them at least

And again, stepping back and thinking about process. Packaging interacts with different ways your team develops code. and automated builds and CI are one of the most significant places these interactions happen. And so as you're doing this, you want to be thinking about like doing run tests before you make the Docker image. After I can be using tests with a Docker As we saw, you want to think about how branches work. And so automation of the build of your Docker package is a good place to think about how packaging your Docker images you can integrate with their larger development processes with testing, with deployment, with PRs, with code review, and so on So you've gotten your image working, just the basics. That was step one. Step two is you've added security.

Step three, now you have automated balance. And so now that you have all these automated builds, now that you're creating all these images, maybe it's even running in production at this point, you want it to actually run well. Like you don't want things to crash. You don't want things to be slow, you want shutdown to be fast, you want startup to be fast. And since you are going to have problems, you want to make sure you have logging and debuggability. Since you're going to have multiple images, you want to be able to identify images. And so as a catch-all, the next step is operational correctness and debuggability, making sure that working with these images, debugging things in production identifying images, all these things are easier. And for example,

if you have a bug in your Python code, you'll get a trace back. Because we'll say on line one, two, three, and function foo, and module blah, you got a zero division error. And so then you can look at your code and say, what's going on? Why didn't we check for zero? Where did the zero come from? And you can fix your code, and that's fine. If you have a bug in C code, your program will crash silently And this can happen because Python's written in C. Database adapters like for Postgres or MySQL they're often written in C. Using something like Matplotlib to generate graphs written in C using NumPy or Pandas written in C. So all this code can in theory have bugs. Your program crashes and there's nothing in the logs. And then you don't know how to debug it.

You don't even know what library crashed It's silence. But Python actually has a built-in way to deal with this called the fault handler module. And basically all you have to do is set a magic environment variable As we're seeing this example Docker file, env Python, Python fault handler equals one, you add this to your Docker file. And now every time you're your pr um you have a segfault in your Python program, uh it will try and usually succeed in giving you a Python trace back that will end get printed on standard error and therefore typically end up in your logs. And so if your program crashes due to a segfault you'll actually be able to identify where in your Python code at least the problem came from. Then you can say, oh, my Postgres adapter crashed.

That's where the problem is. You have a starting point to start looking at what the problem is. Another thing you might want to do is make it easier for your images to be identified So we talked about tagging our images based on branches, but the problem is that tags, uh names you give to your Docker image are not actually stored in the image itself And so it can be difficult just given a Docker image to tell what the tag was originally. But to deal with this, Docker has a feature called labels where you can embed metadata inside the image itself And that's always in the image. So you can always look inspect the image and look at it and say what's your labels and identify where it came from. So here we can see a tweak you can do to your build script

where you get to current commit in the current branch. When you build your image, you use the dash dash label command to Docker build to add labels to your build. And then Your images will always know which git commit and which git branch it came from. And so you can then give it a Docker image reproduce exactly which code was responsible of creating that Docker image. Another best practice you can do at this point is run a smoke test. If you think about the sort of what kind of tests you might be writing, unit tests check your sort of low-level Python code. An integration test might check like your overall system. So it might say, when I send a request to my server and it talks to a database, it gives me back the right response.

And the JavaScript works correctly with it. And you can test that with Selenium or something. But in between those, you might have a situation where your Docker packaging fails, and you don't want to find out your Docker packaging fails by your integration test timing out after 30 minutes saying couldn't reach server. And so it's nice to have a test specifically for your Docker packaging. And this can be a really simple test, just a smoke test, that is you uh turn it on and see if smoke starts pouring out. Um so in the case of a Django server you might just Start the Docker image and rest send an HTTP query to it. And if that HTTP query comes back, you know that you've done enough at least start Django and that's probably good enough to know your Docker packaging isn't completely broken. It won't catch everything, but it'll catch like the 90% of problems that will just break your integration test

altogether And that means you have a faster feedback loop. Another fun example of making your system work better in production. G Unicorn has a heartbeat system. It will send heartbeats to workers and then if that doesn't get a heartbeat back, it'll decide the worker's dead. And those heartbeats go through a special file that um is written to the file system. By default is written in temp. And if you look through the docs, it'll tell you that if the um Worker temp files are on an actual file system, not a RAM disk. Um this can cause delays and your workers might freeze and it will

just y your web server will become more l unresponsive for a few seconds at a time. If you're running in Docker, this is pretty much always the case. And so you can fix this by um this command line argument you say worker tempter is slash dev slash shm, which is a little tiny RAM uh disk, like a um little file system that's in memory and that way you're not hitting like uh cloud file system that can occasionally be slow, thus making your whole web application slow. If you are shutting down your server, it's very easy to use the wrong syntax or use the write your shell script the wrong way such that shutdowns are slow. So then it'll take 10 seconds for your

Web server it's shut down, it'll get killed with kill minus nine, so you won't be able to do any clean cleanup and shutdown. The way you get around this is by using the uh square brackets syntax for your entry points And then if you have a shell script, you have to make sure to use exec to run your program instead of just running it directly. And this is because a whole bunch of interactions involving signal delivery And just Unix signals are very hard to get right. And so basically you just do these two things and now your Docker container will shut down faster and more cleanly. So to recap where we are, um, we've gotten our Docker container working, we've added security, we've made automated builds. We've made it run better in production.

And so, you know, this may take a while and time passes and you're working on your doctrine container. And if it's been two weeks, Uh the libraries you depend on probably won't change dramatically. Probably there won't be a Django release in the two weeks you're working on it. But over the course of six months, there will be new releases of some of your dependencies. Over two years There'll be like two major releases of Python and like there'll be a new version release of Django and just things will change. And so All these you want to do you you want to upgrade your dependencies, but you want to do so in a controlled manner. You don't want to Just always be using the latest version because then something might break. But you don't want to never change because then you won't get things like security updates.

So here's an example of the ways you can deal with reproducibility. When you create your Docker image, you need to typically choose a base image, which is usually based on some Linux distribution. And you want that base image to be fairly stable. You don't want it to change, like make major changes or break things, but you also want to get security updates. And so a good basis for uh Docker images is some sort of release of Linux that has security updates and backwards compatibility for a few years. So Ubuntu long-term support, Debian stable, Red Hat Enterprise Linux. um they will all provide security updates and some minus stability. There are also the official Python images for

come from Docker, which are based on Debian Stable, but give you access to a whole wider variety of Python, not just the ones that are based in uh that are included in Debian Stable So for example, Python colon 3. 9 slash slim slash bullseye is the official Python Docker image. Version 3. 9, it'll usually use the latest 3. 9 point release. It's running in Debian Bullseye, which is Debian stable. Debian releases are named after Toy Story characters. That's where bullseye comes from. And slim means it's the smaller version of this image, so it just takes less uh disk space. And if you use that, um you will get uh a pretty stable base image Things won't change out from other new, they'll be backwards compatible, but also you'll get security updates when you um install them.

And reproducibility, as you can imagine, is not just a thing you can do at the start when you create your Docker image, it's an ongoing process you have to work on. And at the one extreme you can say every time you rebuild, you're going to install the latest version of everything. The latest version of Django, the latest version of your Postgres adapter, latest version of Pandas, just install the latest version. What that happens is if there's a new incompatible release of Django, your application will break. So you want to do is you want to pin or freeze your dependencies and install any specific versions. So I'll give an example of doing that next. But freezing your dependencies also has its problems because if you freeze them for too long you won't get security updates.

Uh if you wait like two years, upgrades start becoming uh more dangerous because that's two years worth of changes. And look there's long-term support releases for certain packages like Django has long-term support. But I checked and Django 2. 2 uh sort of stopped getting security updates in April apparently So if you're on Django 2. 2, you have some stability, but as of April, you're not going to have any security updates, and that's going to be a problem. And so what you need is an organizational process to continuously update your Python dependencies. So you pin them in place. So if you build rebuild an image today, rebuild them tomorrow. they'll use the exact same versions, but you also every three months, let's say, go and upgrade a bunch of dependencies so you don't end up two years in realizing that you have to do this major upgrade of every single one of your Python dependencies.

And same thing with a version of Python you're using. You want to be upgrading it on an ongoing basis. So like you know, once a year, make sure you're going from 3. 8 to 3. 9 to 3. 10. So we talked about pinning Python dependencies. Like if you say you have a requirements file that has a Django in it, you're gonna get the latest version. So you can use tools like piptools, for example. Pip tools will take a file called requirements. in with just Django, and it'll turn it into requirements. txt with a specific version of Django but also a specific version of a version of all its dependencies. So your things you import and all their dependencies will be pinned in unchanging. And then you can rerun so you you keep both these files around and every once in a while you update your dependencies by regenerating your requirements

at text. And there's other tools that do this like pipenv and uh poetry. And so you have a Docker container that's secure and you have automated builds and it works well and you figure out a process to do reproducibility and security updates. and version updates. And so at this point your Docker image is functionally done. Like it is does everything it should. And so now you can focus on optimization. Your time is expensive. Bandwidth can cost money too, storage costs money too, and so you want to spend some time at this point. The final step are in making your images smaller and making builds faster. So you can save some money, save some time, save some bandwidth. So one common piece of advice you will get for making smaller images is to use Alpine Linux.

And the problem is that is Alpine Linux cannot currently use wheels from PyPy. If you're on Linux, people can upload to PyPy, Pype I. They can upload pre-compiled packages of your Postgres adapter, of pandas, what have you. And then you don't have to compile everything on your computer when you install them. However, those won't work on Alpine Linux because it uses a different standard C library than most Linux distributions. And so when you install packages, they'll have to be compiled from scratch. And so that can take your build from 30 seconds to 1500 seconds because now you have to compile massive amounts of code. There's some work in progress to allow people to upload wheels that will work on Alpine Linux

So it may be at some point this problem will be fixed, but until it is , you should avoid using Alpine Linux for your Docker images because it will slow things down. Excuse me. Um another example of ways to speed up your build. If you look at this Docker file, we're copying and requiring set text, then we're installing Gcc, uh, and then we're installing requirements. What that means is every time requirements at text changes, that invalidates the layer cache we talked about that Docker uses for fast rebuilds. And so the apt get installed GCC will have to re-happen again, even though it does not in any way depend on requirements at text. So what you want to do is you only want to copy files in right before you use them.

So here we see first we do the app get install, then we copy requirements at text in And so that means we get better caching, we get faster rebuilds, uh, because requirements at text changing doesn't prevent caching of the GCC install. Similarly, if you just copy your code in call pipinstall. And then all your dependencies are in setup. py Every time your code changes, you're gonna have to reinstall all of your dependencies again from scratch. It's faster if you have a separate requirement set text. Copy it in, we install the requirements at text, and then we copy all your source code in and install things. And so that ensures that changes to your source code won't require reinstalling all your dependencies, which means rebuilds of your Docker image will be faster because

Docker can use caching. Another thing you can do that will both speed up builds and make for slightly smaller system images is when you install a Debian package By default it'll install a bunch of things it thinks you ought to have, recommendations. That just makes your image larger. You probably don't need them. So you can just add dash dash no install recommends to app can install and then you'll install fewer packages, leading to faster bailout, smaller images. Another example, when you run pip to install packages, it will typically cache a copy of the downloaded file locally. And the idea is you're probably going to create a new virtual env again in 10 minutes. So by keeping cache of downloaded packages, you can just look in the cache, it won't have to download it again, speeds things up.

In a Docker image, you are not going to be creating a virtual again in 10 minutes. You only ever are going to install these packages once. They're just wasting this space. They're just making your Docker image larger. So you can add the option dash dash no cache there. When you pip install, and that will mean pip won't store extra copies on disk. Your disk image will end up being smaller, just easy change, make things a bit more optimized. So those are some examples of some of the best practices. And to recap the process we're going through here, get something working, make it secure. Have it run in CI and with automated builds, make it work well better in production with debuggability, correctness we operate

for production. have reproducible builds and finally you focus on optimization so faster builds smaller images and just to repeat something I've been saying throughout this talk but what's worth repeating again Creating a Docker package requires creating things like a Docker file, build scripts, and so on. But it also, and just as importantly, involves interacting with and creating pro organizational processes. So you're going to interact with your development processes, how you do uh version control. We talked about branches, how you do testing, how your automated builds work. You have to add new processes. You have to add processes for security updates. You have to have processes for dependency updates. So creating your Docker file does not mean you're done. You are going to have to work with these processes on an ongoing basis pretty much forever until you stop using this Docker image in production.

These processes have to be there, they have to keep going. So that's my talk. And I realize that's a lot. And so there's a prose version focusing on the sort of high-level aspects of this talk at pythonspeed. com slash product slash stockerprocess. It's like a free PDF, it gives about 10 pages and just focuses on the process aspect. And if you go to pythonspeed. com slash Docker, I have a whole bunch of Docker articles about Docker best practices And so you'll find articles about keeping your cache warm, about keeping your image small, things I haven't even talked about, like multi-stage builds. Lots of articles there, lots of best practices. I encourage you to go look at both of these. Thank you very much, and thank you for listening to my talk.