An Intro to Docker for Djangonauts

All right. I have my speaker notes today, which is wonderful. Um good morning. We're we're moving from Daniele 's wonderful, beautiful talk about art and naivete , and we're moving from those concepts to things that are very, very more practical. So I hope that this change of pace is is comfortable for everyone. I'm going to try to make it a little bit whimsical too, so we still get to keep a little bit of that spirit that Danielle has left us with. Ooh, I have to turn this on. There we go. So you've probably figured out by now that my name is Lacey and I'm one of the organizers of DjangoCon US. I also work for a company called RevSys. We're a Django consultancy based in Kansas in the United States. I work remotely full-time. But yeah, so if you'd like to talk to me

later today about about Django or DjangoCon or the work that we do at RevSys, please seek me out. I won't be taking questions at the end of this talk, but I would really love to chat with you at the break or at lunch or some other time really about anything that you'd like to talk to me about. But right now we're going to talk about Docker. And um Docker is is a little bit confusing for people, and so um I 'm really happy that Daniela used some images from this same comic because I've used a couple too. But the the punchline from this particular slide is that someone um wants to create this um this this side-by-side display, but this you know this this tablet that they're using doesn't have split screen support. So they say, I downloaded the SDK and the IDE, registered as a developer, and started reading the language's docs.

And then I realized it would be way easier to just get two smaller phones and glue them together. And on that day I achieved software enlightenment. And then their friend says, but you never learned to write software. And the original person says, no, I just learned how to glue together stuff that I don't understand. And um this is this is kind of how a lot of people feel about Docker. At its core, Docker feels a lot like magic. And whenever you take tutorials on Docker, a lot of those tutorials consist of, all right, just copy this file and and put it at the top of your project. And then run this command and you're good to go. And that will get you through a a Hello World app, but once you start actually using Docker, it doesn't leave you with enough information to feel confident debugging, to make the changes that you need to. Whenever you're just told to copy and paste a chunk of code, you

you aren't empowered to um with the information to um to do other things with that. So that's part of what we're going to do today is we're going to get very detailed and get kind of a micro level of what the things that you're copying and pasting in these tutorials are doing. in the hope that that leaves you with the ability to go forth on your own and um and do more things with Docker So one of the things that Docker does is it allows you to separate your dependencies from one another, especially in different projects, which is this similar thing that virtual environments do, but Docker handles it a little bit differently, and we'll talk about that Separately or differently from a virtual environment, Docker shares your operating system. A virtual environment usually builds its own guest operating system on top of yours.

Docker just shares yours and goes up from there, so it's a little bit lighter weight in that way. But one of the things I really, really love about Docker is that it puts everyone on the same page on your team. So whenever you're using a virtual environment, you still have to have your own system Python. And you and your teammates might have slightly different minor versions or even very different major versions. And that might work on some of your projects, but you'll you'll run into bugs and you you get into that situation where you're having a problem and your coworker pulls down your code and they run it and it works on their machine, you know, so that's you don't you're not really left with very much with that. With Docker, you're defining everything down to the Python version very, very specifically. So all of your coworkers will be running things in the same environment, all of those will be identical, and it makes you it makes it a lot easier for you to help each other.

And the other thing that I love is that you don't actually need to have a system Python installed at all with Docker. You can just download Docker, and Docker takes care of everything else. So you you don't have to have anything on your machine anymore, which is really wonderful. But the thing that is a little bit tough about Docker is that it does have a pretty steep learning curve. And I know because um it took me a while to feel really comfortable with it. I only started using Docker About six or seven months ago, and I said yesterday in my lightning talk that the best person to give a talk on a given topic is the person who was most recently confused by that topic. And that's me. That's the spirit in which I proposed this talk, is that I was frustrated and it took me a while to get my feet under me and feel like I knew what I was doing. And I'm hoping to save you from a little bit of that pain.

I will say though that if you consider yourself a Docker expert, if you if you use Docker day in and day out, you might not learn anything new in this talk, but I hope that you maybe will have an idea about the things about Docker that are confusing. So you're able to help your coworkers and people later on. And so I did talk a little bit about virtual environments and I'm I'm not asking you to abandon virtual environments. I still like them. I have a system Python installed. I use virtual and wrapper. I really love that work on command. It's very, very convenient. But there's another XKCD comic that kind of shows the different paths that our our Python environments can take. And the punchline for this one is, my Python environment has become so degraded that my laptop has been declared a super fun site. And I've definitely been in that place before

too. So the way that we're going to talk about Docker today is by talking about Harry Potter, because that's a completely natural thing to do. To talk about the concepts of images and containers, which is a Docker concept We'll be talking about polyjuice potion. And if you're not familiar with the Harry Potter books or with this specific potion, the way that Polyjuice Potion works is you you brew this potion. And then you get someone else's hair, a hair of someone that you would like to impersonate, and you add it to that potion. Then you drink that potion and you look like that person and you get to impersonate them and get up to all kinds of hijinks So in this situation, Docker is the potion. Docker is the vessel through which everything happens

And then the Docker image, the the thing that you want to turn into, um is sorry, Docker the image is the person that you want to turn into. So whenever you start a a project in Docker , You start by defining your image. The image is um it defines the basic software of your project. The Docker docs say that it is the executable that contains all of the packages that your software needs. Docker also has this concept called a container. And the container is you. Like whenever you're you're drinking this potion And you're running around looking like somebody else, you are running an image in a container, right? Like you're not actually that other person, you just look like them for a little bit. So Docker would say that the container is the runtime instance of the image.

But like I said, you never actually run the image itself. You run a little copy of the image inside of a container. And this is a pretty important concept to understand. So I'm going to put it up here one more time. The image is the executable with all of the packages of your project. It gets your project ready to run. The container is the thing that actually runs. It's the runtime copy of your image that you've defined. But how do you define your image? So in Docker there's this file that you add to the the project level, the same level as your manage. py, and it's just called Docker file. It's Docker with a capital D. There is no file extension. And that's where you define all of the specifics about your project. So you can think about the Docker file as the DNA.

That's that person's hair that you would like to turn into. That's the definition of your project. And in the Docker file, you specify a lot of different things. You specify the version of Python that you want to use, you can set environment variables, you can install specific requirements, things like that. And we'll get into more detail about that in just a moment. So again, I'm going to have these definitions up here just because these are concepts that are pretty important to know. So just to repeat, the image is the definition of your project. The container is what your project is actually running in based on your image, and the Docker file is the file that defines what your image is going to look like. And so this is where this analogy kind of comes into play. And so if you're familiar with Harry Potter, then you can you can kind of look at this a little bit.

You use the hair to tell this particular potion. how to change into somebody else and then you drink it to actually turn into that potion. With Docker, you use the Docker file to tell Docker how to build your image and then you run your project in a container. So that's kind of where this metaphor comes from. And I'm gonna take a quick moment here and grab some water. So this is the basic layout of a Docker file. You will see Docker files that look more complicated than this. There are concepts called multi-stage builds that we won't get into.

So your uh production Docker file whenever you start using Docker will probably look a little bit different than this. It will create um contain more lines But this is this is all that you need to get started with Docker. This will this will get you up and going and then you can explore the more complicated things So we're going to go through this quickly line by line just to make sure that we understand. This from statement from Python 3. 6 you probably guessed correctly that this means that we want to use Python 3. 6. That's the version that we want to use. And the reason that we get to do this is because Python publishes all of their versions as images on Docker. So just like Whenever you're born, you inherit DNA from your parents, and just like in Python classes, we can inherit from one another, we can base images on top of other images.

So we get to say in our Docker file, I want to grab everything that's in this other image that's published on the Docker Hub, and I want to make use of that. This is an environment variable that basically says that you want to be able to see your output in the console. So if you want to see what you're doing, this is a handy line to have. Now, we talked about containers, about how your project is going to be running in its own container. And what that means is that your container doesn't really have access to the code as it lives on your machine. So you have to tell the container, hey, my my code is here and I want you to have that. So in this line, we're taking the requirements file that's in our current directory. And we're telling Docker, I want you to copy that requirements file into a directory called code that is going to be in the container

Now at this point, the container doesn't actually have that directory. The copy statement will automatically create the directory if it doesn't already exist and then copy. Now we want to actually install our requirements because again we're running in a container, we're not running in a virtual environment, so we have to tell the container, you know, these are tell the image that whenever we we run our container, we're going to want all of our dependencies. And after that, we copy the rest of our code into that directory called code that lives on the container. And you don't ri necessarily have to do these things separately. Like you could just copy everything into code. And then run pip install to um install your requirements. But it's nice to separate that so that if if you're getting an error, you get to isolate that a little bit more easily.

Now, whenever you're using Docker, you can still do things like run management commands. And I think that most of us are used to running things like dot slash manage dot pie shell or something like that. But now that we have all of our code in the container in this extra directory called code, we would have to type slash code slash manage dot pie, et cetera. And that's a little bit annoying. So we can get rid of that annoyance by setting our working directory as that code directory in the container. So whenever our container starts, it will automatically stick us in that directory, and then we can run commands the way that we're used to. We want to expose our port so that we're able to see that port 8,000 that we need whenever we're running the server And then we can put our Python command right there in the Docker file.

So we can say, hey, you know, I want to run my server whenever my image is created, whenever my container starts. I want my server to go ahead and start running for me. So now we have the the DNA for our project, we have our our Docker file, we're ready to build our image. So we have this file, but the file itself doesn't create the image. We have to tell Docker, hey, I want you to create this image now. So to brew that potion, we run this command, this dockerbuild dot. Docker build everything that's in this directory. And whenever you run that command, you'll see a lot of output. Um I have bolded the ones that are kind of important. But you run Docker build, and then you see that it's it's you know sending the build to a context, it's going through all of these steps, and then the very last line, if everything has gone well, will be successfully built

some kind of random name. And that name is the name of your image. And that doesn't really make a lot of sense, right? Like that's that's just a random string of letters and numbers that that's not meaningful to us. So Docker provides us with a way to give our images a name so that we can keep track of them more easily. If you add that T flag, then you can tell Docker, hey, I I want you to give this image a name so that I can access it more easily And that way you don't have to tell Docker to show you all of your images, scroll through, find the right one, copy it, and then use that that Docker ID. You can just use the name of your project or a name that's meaningful to you. And we we can see this this paradigm before, right? Like if we go back, we can see that it says successfully tagged Hogwarts colon latest.

That's not the first time that we've actually seen that paradigm. When we created our Docker file, our very top line was from Python colon 3. 6. That's because that's how Docker names things, right? So whenever you create an image, Docker automatically assigns that image a version. If you don't assign it a specific version, that version will just be called latest. It's the most recent version. For Python, Python has published all kinds of versions of its images. So you have the specific Python images for the different versions, but you can also download smaller Python images if you don't need all of Python. So you can make your your image um lighter weight. But the way that you'll see that is the image, a colon, and then the tag, and another word for tag is basically version.

So now we get into the concept of layers. Docker is layered, which is a pretty important concept to grasp. And you can you can kind of get a sense of this layeredness whenever you ran that Docker build command. You saw that Docker was building step one out of eight, step two out of eight. Every line in your Docker file, every single line that has anything in it is its own layer in your image. So Docker builds those one at a time. So the very first time through, Docker builds each layer by hand. So it's it's going through, it's it's crafting everything, it it just builds everything by hand. But the second time through, if if nothing has changed, Docker knows it doesn't need to rebuild it from scratch, right? So it just goes ahead and uses a cache, it will run very quickly, and that works really well.

But if you change something, Docker will use a cache until it gets to the thing that has changed, and then it can't use a cache anymore at all. So if you change something in the middle, it will use a cache until that you get to that middle, and then everything after that gets rebuilt by hand again. And the reason that this matters is because that can impact Some performance things, it can take longer. Um so it's it's just good to know that whenever you change something about your Docker file, um, you might run into this. And With that in mind, it's also good to, whenever you're creating your Docker file, to do things in an order that lets you put the things that are least likely to change at the top, and then things that you might be fiddling with more frequently at the bottom.

So we can actually see this happen. I um I added a requirement to my requirements file. So I didn't actually change my Docker file. I changed the requirements file. But because I'm telling Docker, hey, I want you to copy my requirements file, Docker knows Oh, something about this requirements file changed. And so at the top we can see that in step two it's using a cache. But then we get down to step four where we're we're running our requirements. And we're downloading Django fresh. The last image that we used already had Django. It could have used a cache, but since I added a new library, maybe I added requests or something, it has to download Django from scratch all over again and then keep going. So that's the Docker file. So just to recap, and I

you're probably going to get annoyed with me saying this, but I'll say it again The image is the executable that contains all of the packages for your project. So it's it's the basically the person that you want to turn into. The container is the runtime instance of your image. It's the thing that actually is running your project. So that's you once you've once you've drunk the potion. And the Docker file is the DNA of your project. That's the definition that says this is what I want my project to look like, this is what it needs. So now that we've we've had potions class, we're going to move on to charms. And so let's talk about the different commands that you can use to get Docker to give you some information about your project. So you can tell, you can ask Docker, hey, what images do I have? And the way that you do this is with just the Docker Images command.

And Docker gives you some helpful information. So it gives you the repository that your image is in, the tag that that is, um, that it has. the image ID, when it was created, and also the size of your image. And when you're using Docker in production, the size of that image gets pretty important. Um a few, I think months ago now, Revsys published its its own version of Python that is is smaller than some of the others, and so it makes the images build a lot faster. But that's not something that you need to worry about for your first time out in Docker, but as you get further along in Docker, you'll probably want to explore some of the smaller images. You can also ask Docker to reveal your containers with the with the command Dockercontainer LS. You can also use Docker PS. There's a cheat sheet at the end that I'll link to. You are not expected to memorize all of these.

I certainly haven't So um but you can see here that we, you know, we're Docker is expecting to give us a container ID, the image that container is based on, commands it's running, and some other data, but nothing is there. And the reason that nothing is there is because so far we've built our image, but we haven't actually started running our containers. Containers don't exist until you ask them to. So let's ask our image to give us a container. And we do that with the Docker run command. You run Docker run -p, which stands for publish. 8000 8000, which stands for, hey, take, you know, the container's port 8000 and publish it to my local machines port 8000. So that I can still use localhost in my browser the way that I'm used to, and then the name of the image that you want your container to be based on.

Later on I'll show you an easier way to do this, but if you wanted to start just with Docker, this is how you would run a container Specifically for a Django project. Not all containers will need the ports. For a Django project, you need that port. And we can we can prove to ourselves once our container is running if we open up a new console tab and we ask Docker to tell us about our containers again So then we get the container ID, the image, the commands that it's running, status, some other information about it, and then that name there, you see the names on the bottom right. That's a kind of a nonsense name that Docker gives your container. I didn't name the container for Vent Allen. I don't know what it means. Docker just puts two words together with an underscore, which is, I think, a little fun and whimsical. You can also hop into your container and poke around.

You do that with the Docker exec command. And learning how to exec into your container can be a little bit painful whenever you start to Google it. The people have kind of different strategies. But this is the one that works pretty well for me. So Docker exec -it, the ID of your container, which remember you get by running Dockercontainer LS. So just copy that container ID and then paste it And then shell, and you can actually see that we wind up in the code directory, right? So whenever we set that working directory in our Docker file, We can prove that that Docker is doing that for us. So whenever we enter our container, we are immediately where we expect to be. And we can prove that to ourselves by asking the container to just using a regular bash command. um asking it to list all of its files.

So we see our manage. py file, our requirements file, and our Docker file the way that we expect to We can also go up a level and then list the directories that are there. So we see our code directory is the second one, but we also see all of these other directories. And we don't really need to worry about those right now. You'll get to a point where you might need to worry about them, but for today we don't. It's good to know that they're there though. The container is doing its own thing. It has a bunch of its own things installed. In one of these directories you can actually go into it, go down a few levels and see all of the packages that you've installed. In another one of these directories, you can drill down into it and you can see where your database lives So it's good to know that this is there. We won't go into it today. That's a little bit advanced charms for us.

To get out of your container, you control D. The reason I have this slide is it took me forever to learn this This is a keyboard shortcut that probably everyone in the room knew about me, but just in case there's two of you that didn't, there you go. Um and then we can see though, once we've hopped into the container and hopped out. We never asked our container to stop running, so it's still running. We can ask Docker to tell us the containers that exist, and just because we hopped into a container and hopped out, the container didn't change. It's still chugging along doing what we asked it to do. We can stop our container if we we we don't want it to be running for a little bit. If we want to you know change something or um or test something out, we can stop it with just Docker stop and the container ID And if we're using Docker stop, we can bring it back pretty easily with Docker start.

So it doesn't have to rebuild anything, it doesn't have to retrieve the image. The container is is still there. We had basically just paused it. So stop really means pause. We can kill the container entirely. So if you run Docker kill, this means I want to blow this container away, I never want to see it again. Now remember your image is still there. But once you've run Docker kill, you'll have to run the Docker run command to get your container back. If you kill your container and try to start it again, nothing will happen because it's dead. Okay. Now advanced potions, Docker Compose. I told you that I would show you an easier way to start your container. This is the command that we know about so far, right? Docker

run dash p eight thousand colon eight thousand, the name of our image. And It's not like a huge command, but it's a little bit more complicated than you know manage. py runserver, right? Like it's a lot to remember. You have to remember the port, the name of your image, and that's a little bit annoying. And so then we get to Docker Compose, which is a tool that lets us do things a little bit more easily. It's free When you download Docker, if you're on a Mac , Docker Compose comes with Docker. If you don't have a Mac, then it it is an extra download. But again, it's free, so that's fine. And the biggest thing that Docker Compose gets you is the ability to run more than one container at once. So my favorite feature of Compose is just that it's easier to type. But the second favorite feature is that I can identify several containers, which we'll talk about why that's useful in just a moment.

And finally, I can relate those containers to each other. So I can I can say, you know, if I identify a database container, I can say, I don't want to start my web server until I have my database up. And that's exactly what we'll do. So you can think about Docker Compose as that magical bag that Hermione has in the very last Harry Potter movie. The character Hermione has this this bag whenever they're on the run from the law. And it's it's pretty small, but it holds essentially an infinite amount of stuff. So you'll see her reaching her arm into this tiny bag and it's all the way up to her shoulder. And she's pulling out potions and books and a tent. I mean, all kinds of stuff comes out of this bag. And that's kind of what Docker Compose is. It lets you put all of these containers identify them, define them in this one file, and then you have access to them by just running one command.

You don't have to run several containers individually. So Docker Compose comes with its own file that also goes at the wherever your manage. py is, wherever your Docker file is. It's written in YAML. If you've not written in YAML before, that syntax is pretty well documented. And it's a relatively straightforward syntax. It relies on that nesting that we like so much about Python. So we'll go through this one too. The version is just a version of Compose that you're using. Three is the most recent version right now. So the version that you're using will impact the syntax that Compose is expecting from you. And then you identify your containers. So services is basically another word for containers and Docker-compose

parlants. So I'm identifying a DB, a database container, and also a web container. And this is also one of my other favorite features about Docker Compose, is you can do things like use Postgres without having to download Postgres on your machine. And the reason that this is wonderful is because two or three years ago I tried to download Postgres and get it configured so that I could work on a particular open source project And I couldn't get it to work. Like I I didn't have the right permissions on my machine or something. on this particular version of Python, I can tell Docker Compose, I want to base my database on this image of Postgres that's published. Now, I just have Postgres here, so it's going to automatically get the latest version, but you could add that colon and a specific tag to specify the version of Postgres that you want to use.

And of course, whenever you're making changes about your database, you'll want to update your settings. py in the ways that you're used to to tell Django to expect a different kind of database than the one that it ships with. In your web container, you have to do a little bit more. You have to um have to tell Docker about a few more things so that it will work properly. You can run this command. I usually have mine set up to go ahead and run migrations before it starts my server. There are some pros and cons to that. You probably don't want to have Compose make your migrations But having Compose go ahead and run them can save you a little bit of time. You also create this concept called a code volume A volume is is just a way for Docker to keep track of a particular chunk of data or files between your coding sessions.

And then you can go ahead and publish those ports, right? So this is the same um concept as whenever we were using the Docker run command. We're telling compose I want to take the container's port 8000 and publish it to my machine's port 8000 so that I can access my website in my browser. And then finally, I told you before that we could relate containers to one another. That's what we're doing here. We're saying this web container depends on the database container. So I I don't want my web container to start until my database container is up and running. And a few other things that you can do with Compose is you can add services for like your caching service, like Redis. or um your task service like celery. And if you have, for example, a a celery task that is supposed to run every five minutes and it has a lot of console output, that can get really, really annoying.

It kind of clogs up your console. So one of the really nice things that you can do is you can you can start your project and then you can stop just that celery task, like that celery service. So your your celery task just won't run if you're working on something else, which which is a little bit handy. I like that. That saves you from having to like change your celery beat settings in your settings file or something, you know, which you might forget to unchange later on. And what this means, since you know you noticed here that we have that command line where we're going ahead and running our server, we don't actually need to do that from our Docker file anymore, so we can just get rid of that. And now, instead of having to run Docker run -p 8000 -8000 Hogwarts, we can just run Docker Compose

up. And if if Docker has never seen this image before, it's never built it before, Docker ComposeUp knows to build that image before it starts to run a container. If you need to rebuild your image, you need to make sure that your image gets rebuilt, you can just add the build flag. And whenever you run Docker Compose up, if you already if if you don't already have the image, you'll see all that output where it's creating the image. But if you already have your image, you'll see some output that looks something like this. And so you'll see that we're creating a date a DB service, and we're also creating a web service. And we're seeing that we're attaching that database service to that web service. So Compose is doing what we told it to. It's creating the database service first, then it's creating the web service, and then it's relating them together. And then we can see that it's running the migrations just like we asked it to.

And then it's starting our server. And if we go to localhost on our on our um in our browser, we will have access to our website the way that we're used to. And if we open a new tab and we run that Dockercontainer ls command again, we can actually see that we are definitely running more than one container now. So we see that web um container like we expect, but we also see this new Postgres container that contains our database. And then you can do things like um run management commands using Docker Compose. So you just prefix everything with Docker Compose run and instead of having to have the the container ID like you have to, you know go and like copy that that those weird strings that don't make any sense. You can just specify the name of the service that you defined and compose.

And then after that it's you know dot slash manage dot pie, whatever the way that you're used to And you probably noticed this dash-rm flag. Whenever you run a Docker Compose run command, that will create a new container. that it can run that particular command in. But you don't actually need that container once it's finished doing the task that you ask it to do. So you can tell Docker, hey, you know Run make migrations in a new container, but then just trash that container because I won't need it anymore. And kind of like magic, um Docker will will go ahead and um I saw my my time warning and then I lost my train of thought. Ah. Oh yeah, but so Docker will go ahead and tell your first web container about the changes that were made in the second container,

which is which is pretty handy as well. You can hop into the shell in the same way. Katie taught us about the shell yesterday. Um with with the same technique, Docker compose run web manage. py shell and then you have access to the shell just the way that you're used to. And again you can see that you have that that database. container that's that's getting created and being confirmed before you hop into the shell so that whenever you're trying to access the data in your models, you get those pretty easily. And then you can stop and start containers, um, pretty similarly to the way that you do that with with vanilla docker. You just use Docker Compose instead of Docker. And again, instead of having to get that container ID, that nonsense string, you just use the name of your service. So you can start and stop your webcam containers, which is pretty handy because my experience is that sometimes whenever you make changes in a template

Django kind of likes you to stop and restart the server, which gets a little bit annoying, but it happens sometimes. And so this is a a simpler way than having to copy and paste that container ID to get that to happen. At the end of the day, when you're ready to go to sleep, you run Docker Compose down. Docker will very gracefully remove your containers and shut down its system so that you're ready to go home. And I've created a little sample project for you on GitHub that you should be able to check out, download Docker, download Docker Compose, and just run. There are some instructions there. Um and of course feel free to ask me any questions. I put it together last night at about one in the morning, so if there are any um any errors, please let me know and I would love to correct those. I was I was Not at my freshest at that point. And so I want to close with a quote from Albus Dumbledore.

Albus Dumbledore, the greatest headmaster that Hogwarts has ever known, said: It is the unknown we fear when we look upon Docker and nothing more. And what he meant was that Docker can be a little bit intimidating. Um there's a lot to learn, there's new vocabulary, there's new commands, but at the end of the day, it's a it's a really wonderful tool to use. I am a complete convert to it. It was pretty pretty intimidating to me at first, but now I'm very much in love with Docker. And there's it has so many more capabilities than what I've gone into here. And I've given you just many, many resources and again I will put my um I'll put my slides up these will be clickable so you will know but there there are so many resources available to you some are better than others But that that Docker cheat sheet that's like third from the bottom,

that is just a lifesaver. It has the full list of Docker commands and a pretty easy-to-digest format. Um and I think I've used just about all of my time, so that's all I have for today. Thank you.