What Django Deployment is Really About by James Walters

Speaker 1: Hi there, everyone. Uh thank you for being here. It's the end of a long day at DjangoCon, and all of you are here to cap it all off with a talk on deployment. You are courageous souls. I'm James Walters. Let's talk about what Django deployment is really about. How many of you think that Whiskey is something you order at the bar? Uh how about uh how many of you think a gunacorn is some sort of ballistic horse? A few of you, yeah. How about this? How many of you, is there anyone here who has never deployed a Django project before? Maybe you're a beginner or maybe someone else at your uh organization is the deployment person.

Speaker 1: Okay, a few of us here. Yeah. And now how about the question of the hour? Is Django hard to deploy? Recently, I was looking over the 2023 Wagtail Developer Survey and found it noteworthy that about 23% of respondents said there were not enough examples or documentation about deployment. And Wagtail is just one corner of the Django ecosystem. So I think this is clearly a place where a lot of people have a hard time. Hopefully, we can simplify that some today. Let me begin by telling you what we're not going to talk about. We're not going to talk about the penguin in the room today. I know one of the things a lot of people have trouble with when deploying Django

Speaker 1: is getting acquainted with Linux. And today we're going to limit our scope to just the stuff around running Django and maybe a web server. Your operating system is an implementation detail that's outside of our purview today. What we are going to do today is boil deployment down to a handful of main concerns. Because the problem with deployment, I think, is that we teach people how to build apps. We focus on getting to know Django and using it to build out a project. And then when we get to the end of that, what often happens is we give someone an appointment recipe or a list of steps to follow But the steps that worked yesterday might not work today, or the provider you used might not work for me. Or even if the recipe does work, maybe I got my app online, but I

Speaker 1: didn't really learn anything about deployment. I don't understand any more than I did before about what's going on in the process. And there's a lot going on in this process. There are a lot of moving pieces. To deployment. So I want to break down deployment into four main concerns that you need to think about when you're taking your app from your code editor to the web. The first is static files. Again, in the Wagtail survey, the number one step of deployment that developers struggled with the most was configuring static files and media files. Jacob Kaplan Moss conceded the difficulty here in his PyCon talk from a few years back called Assets in Django Without Losing Your Hair. Now, I know what you're thinking

Speaker 1: as a good Django developer, but static, right? I learned this on the first day. You use the static template tag to link to your static files, and Django figures out the rest, right? So let's take a moment to explain what's going on with static files. These are assets like your images, CSS style sheets, and scripts, and they're called static. Because they aren't dynamically generated. Most of your Django projects pages are built with templates that get filled in with information by your program on demand when users request the page. That never happens with your images or CSS files. They're just files sitting on disk ready to be served. Web servers like Apache or Nginx

Speaker 1: were built for the task of taking files on disk and serving them to users. That's where the web started. On the slide, we have something like an old-fashioned Web 1. 0 website where our user goes to yourwebsite. com, a static HTML file is fetched off the disk by your web server and is served to the user. But we have a Django application because many pages need to be generated on demand with data from the database. We don't have them sitting on disk. A web server can't do that. We need an application to generate those files. So where does that put our static files? Do we want Django to handle them? Since these static files aren't generated by our Django app, we Don't want Django to handle them.

Speaker 1: Your Django program is actually way slower than the web server for this job. So we want the web server to do it How Django deals with that is it stuffs all the static files into one directory. This will be static root in your settings. py. And this makes it easy for the web server to look and find the files it needs to serve. And then we use the static tag in our templates to tell Django that this request needs to be passed along directly to the web server. Now, the reason all this might seem kind of new is because Django has been handling static files for you in development. The Django development server, manage. py run server, has to handle those requests because when you're developing locally, it's the only web server available.

Speaker 1: But when you deploy, you'll have a better web server than that at your disposal, and that server will be quite capable of handling static files on its own. So once you've set a static route to stuff your files in, you can actually stuff them there by running manage. py collect static. And you'll have to do this anytime you change any of your static files. Now, some of you might be thinking, what about white noise? If you're not familiar with white noise, it's a middleware that you can add to Django that intercepts the request in the middle of the cycle. And if the request is for a static file, white noise will interrupt the rest of that slow request cycle and serve up the static file immediately. It's a great option if you're in a situation where you can't directly configure your web server, like maybe your virtual machine is handled by your infrastructure team

Speaker 1: and you can only push code to it. Where white noise really excels is if you're serving your site from a CDN. And in that case, it's beneficial to add HTTP headers for things like caching and compression. And White Noise takes care of all of that for you without having to think about it. So whitenoise really is a great way to solve that problem. But if you're not on platform as a service, I would encourage you to take a look at writing the web server configuration yourself for at least a few reasons. First of all, writing an Nginx or Apache config file isn't that hard. And here's a link to a great real Python article showing you how to do it. Secondly, you made a web app. You should invest some time into understanding some fundamentals of how the underlying web infrastructure that supports your app works.

Speaker 1: After all, a lot of the confusion around deployment stems from the sort of impedance mismatch between the nuts and bolts of web servers and our higher-level web app frameworks. And thirdly, you should learn how to solve this problem using something other than white noise because it won't help you with user-uploaded files. Django calls these media files. This is stuff that people other than you post on your site. Profile pictures, file attachments, etc. The way you set this up is pretty similar to static files. Instead of static root and static URL, you'll use media root and media URL in your settings. And again, White Noise won't help you here. White Noise only deals with static files. So you'll need to know how to set up your deployment situation to find your store of user-uploaded media files.

Speaker 1: If you're in production, what this will most likely look like is configuring a cloud object storage back in, like Amazon S3, for those files to go live in. The rule of thumb is you don't want to store those on the same machine that your app is running on. You don't want to permit people to upload arbitrary data onto your production machine. So a little separation here is hygienic. Okay, so we've addressed static files as one of our deployment concerns that we have to think about solving somehow. The next one is our database. Now some of you might be thinking, but SQLite, right? It's been doing great for me here in development. Why can't I use it in deployment? And that's a question that we'll leave for the database admins among us. Opinion is in fact shifting on this topic a little bit.

Speaker 1: It's possible to configure SQLite in ways that will uh scale quite far. This blog at fly. io is a good introduction to the subject. Simon Willison, of course, is also using SQLite behind his dataset project in some interesting ways. But the point here is not so much what database you use, that's a really big topic, but that your database may well be different in deployment than it is in development. And that's something you have to think about in this process. Generally, if you're using some sort of cloud provider, you'll be connecting to a remote database that's managed for you. And that means you'll have a URL for the database, a port, username, password, this kind of stuff. And all of that will live in your settings. py. SQLite

Speaker 1: is really cool because it keeps everything in a single portable file and that makes it easy to back up. easier to wrap your head around, and since you don't connect to it, you don't have to handle those connection details. Your settings. py is pretty simple, but if you're using a different database in deployment, you'll need to rework your database settings. So here's a couple of examples straight from the Django docs. On the left is what your database is section of settings. py looks like when you run Start Project. You just have the database engine, in this case SQLite 3, and the name of the database. But over on the right, you see an example of what this might look like if you set it up on Postgres. You have the database engine and name, but you also have a host and port to connect with, as well as login credentials.

Speaker 1: Now, not only do you need to configure your new database, but you have to run your migrations as well. Sometimes people get their app stood up and when they start trying to load pages, they get 500 errors. And that's because their database tables have never actually been set up. And so that's easy enough to handle. This is what your Django migrate migrations are for. Just run manage. py migrate while you're connected to the deployment database to run those migrations and set everything up. So again, we haven't gotten into the specifics of databases today. The point is that your database situation in deployment will almost certainly be different and you need to be ready to think about that in the deployment process So, some of you might be ready to get to the bar and have a whiskey, but right now we're talking about whiskey.

Speaker 1: That's a different thing. What on earth is a WISGI server? This one is exciting because your Whiskey server is the thing that actually runs your Django app in deployment. Now some of you hear that and I know what you're thinking. But run server, right? Isn't that how I run my Django app? And in development, you've been using Manage. py runserver to run your app. But as the name would suggest, Manage. py is just a management script. So how do we run our Django app in a production environment? So you'll recall that we talked about having a web server like Nginx installed. This is the piece of software that receives HTTP requests and returns HTTP responses.

Speaker 1: But we have Django apps to generate web pages on demand. So how do we get these two things to work together? The web server usually just handles files off the disk. How do we make it talk to another application? And the answer is WSGI or Web Server Gateway Interface, usually pronounced WISG. The way this works is that when the web server gets a request, it passes it over to a WISGI server like Gunacorn or Waitress or UWISGI. You might have heard these names before. This is what they are. They're WISGI servers. And that whiskey server runs a whiskey callable for your application. How on earth do I make a whiskey callable, you ask? Django already did that for you on the first day when you ran start project.

Speaker 1: It's in the Whiskey. py file. So you set up your Whiskey server and tell it whenever you get a request, run it through this Django app's WISGI callable. And then the WISGI server will run requests through your app. And after generating the appropriate page, it passes that response back to the web server to hand off to your user. Now to configure this that gets into the details of the documentation of your chosen WISGI server. But again, the point here is not to tell you which one to pick, but that you have to pick one. So we've considered static files, we've thought about our database, we've planned out our WISGI server, now we need to spend some time thinking about our web server, or we might not need to worry about that at all. Let's discuss it. So concerning web servers, Apache and Nginx are going to be your two main options.

Speaker 1: Both are great. Configuring either of them is beyond our scope here, but if you don't know which to choose, Nginx tends to be more popular these days. Because it's threaded, so it's it tends to be more efficient. Uh Apache is a venerable old war horse, though. It's a perfectly acceptable option that ran much of the internet for the last two or three decades. So I think it'll be fine for your app. For an example that kind of fits all of the pieces together, RealPython has a great article that involves configuring a Django project to use Gunacorn. through Nginx. So this is Django, a Whiskey server, a web server. Great tutorial applying a lot of what we've already talked about. But the question remains, do you actually need to configure the web server yourself?

Speaker 1: And the answer is it depends on your deployment choice. So you might be running your app on a bare metal server in your home lab or at your company's data center, but for most of us, most of the time these days, we're going to be deploying in the cloud. And the question will be VPS versus PaaS. And so let's let's address what those mean. VPS stands for virtual private server, and this is a virtual machine running in the cloud. It's basically a rented computer. You can get these from lots of providers. A couple of examples are Linode and DigitalOcean. A virtual machine is just that, a virtual machine. So this is like setting up your own server, except the hardware is rented from a cloud provider. So it's a very manual setup process. You have to go

Speaker 1: and configure the entire operating system. But the benefit of that is that if you go down this route, you'll understand all the layers that support your app. The disadvantage is that you're responsible for everything about the server. So if it goes down, you have to reboot it. You have to rotate out log files before they fill up your disk. You have to understand and set security settings, stay on top of updates. etc. So the alternative is platform as a service or PaaS. A PaaS is kind of different. This is a service provided by organizations like Heroku or Python Anywhere. Basically, the provider will abstract away the underlying computer and you just focus on your app. You get your code uploaded, provide some details about hooking up a database, static files, where the script to execute is.

Speaker 1: And you don't have to worry about the server-y stuff, like worrying about updates, rotating system logs, or understanding security settings. You also wouldn't have to configure a web server, the provider would handle that for you. So, do we have a verdict between the two? PaaS usually costs a little bit more than a $5 VPS, but if you don't want to become a part-time system administrator, then it's easily worth it. My personal favorite is Python Anywhere. I think they've done a good job at keeping the control panels focused on the details we've been talking about. You can see in the pictures there the configuration options for telling Python anywhere where your static root is, where your WISGI callable lives. So I think it's a tool that helps you understand this process more rather than just papering

Speaker 1: over it like some services do. They also have a free tier for those just getting started. All right, lastly, I'll point out a couple of tools that are helpful in getting a grip on deployment. First is Django's own built-in deployment checklist. Basically, this is a little tool that scans over your settings file and tries to warn you about things that might not be configured safely when you get deployed. For example, it'll warn you if debug is still set to true. That would leak important information about your app to an adversary. To use it, you just run manage. py check-deploy. And let me show you an example of the sort of thing that you'll get from it. I know this is probably a bit hard to read, but you don't have to read it word for word. I just want to give you an impression of the output that it generates.

Speaker 1: I ran this on just an out-of-the-box fresh project and these are some of the things that it pointed out. First, there's a couple of HTTPS-related settings there. So Django expects you to be encrypted when you're running in production. Then we have a warning about my secret key which needs to be stronger. We have a couple of settings about secure cookies that it wants us to set. Second from the last line there, we have debug is still set to true. And then finally it's letting me know I should configure allowed hosts. So that's a smattering of the sort of stuff that it catches. Go spin up one of your projects and try it out and see what it tells you. Uh a talk on Django deployment really wouldn't be complete these days without mentioning Django simple deploy. Uh this is from Eric Mathis, who wrote the wonderful Python Crash course, and it came out of a need to future-proof the Django section of that very book against the ever-shifting sands of

Speaker 1: of cloud providers. It basically tries to automate the deployment process for some of these platforms and get your app online. Obviously this is great for newcomers, not only because they just finished their project and this helps them get it onto the web expediently. But also because if you have Git, you can then diff all the changes that it made to your code in order to get an idea of the sorts of things you need to change or configure in order to get your app deployed. It's not just targeted at newcomers though. Eric envisions this as being useful to people who want to experiment with different cloud providers. as well as to those cloud providers themselves. It certainly helps simplify getting your customers online if you can say, by the way, you can run Django Simple Deploy and get online with our platform. So very cool project. You should check it out. Eric is right here as a matter of fact.

Speaker 1: So if you want to get in touch with him and find out how you can contribute, I'm sure he'd love that. Currently Fly, platform. sh and Heroku are supported and more are planned, so watch that space. And then finally, I want to mention a project I literally heard about today. It was so relevant, I had to throw in a couple of extra slides about it. This is a project called Django Production. Uh this is from Peter Baumgartner over at Lincoln Loop. He gave a lightning talk on this earlier this afternoon. This project kind of seeks to add some sane defaults for a lot of the problems that you face in deployment. The point he raised is that if Django's philosophy as a web framework is to include batteries that solve the most common problems web developers face, then why doesn't Django

Speaker 1: include any batteries for deployment? So he made one. And like I said, this is very new to me. So I haven't kicked the tires on it yet. But I like the look of it. Some highlights of what it actually does for you are it installs white noise to handle static files. It installs Django and Viron for reading environment variables, something we haven't touched on today, but is often a factor in getting deployed. It installs a package to allow you to run your web server from manage. py. It looks like that defaults to Gunacorn. That might not be a great choice if you're developing on Windows, but I assume it's easy enough to swap that with Waitress or something that would work there. And it will add a health checkpoint, which is important for integrating into system monitoring if you're using that. And then it adds stuff to your settings.

Speaker 1: py file for all of these things right out of the gate. So this ticks a couple of the boxes we've talked about this afternoon. It handles static files and the WISGI server pieces for you, and that's sort of half the battle right there. So this project is worth looking at as well. Uh and that wraps us up. So there's a lot of other things we could talk about. Should you containerize your app with Docker? Should you deploy with a serverless function? Which database is the best database? These are all good next steps to consider. But at a bare minimum, I hope I've shown that these four major concerns, static files, database, whiskey server, and web server, are the places to begin when figuring out how to deploy your Django app. If you can get your head wrapped around these things, I think you'll be able to get your app online and make it to Hello World.

Speaker 1: The rest are details that you'll pick up as you continue to develop and get comfortable putting your work online. So uh I think we have yeah, a couple minutes for any questions, if anybody has any.

Speaker 2: Thank you.

Speaker 3: Thank you for the talk. I'm curious, what do you do you have any thoughts on Docker when it comes to Django deployment?

Speaker 1: Uh so I love the idea of Docker. Docker is one of these things, it's complicated in itself. The great advantage of Docker is it flattens out all the variables of that doesn't work on my machine, right? So you have a containerized environment that's sort of reproducible and isolated from the rest of your system that you can install all the stuff in and it works there. But then Docker might be uh yet another layer of abstraction that you have to grapple with. If you're if you're just starting to I would say you don't necessarily have to worry about learning Docker if that's too much for you, if you're still just trying to grapple with Django

Speaker 1: itself. It's not a bad investment though if you decide, I think I want to go ahead and learn this. you'll also be in good shape for that too. Um I hope the container space gets easier and easier to grapple with Uh over in Linux land, we're moving more into like containerized applications on the desktop. And so maybe containers will get more ubiquitous and easier. But yeah, it's definitely worth considering. It's a very useful technology.

Speaker 4: How do you handle media files with um user-specific permissions?

Speaker 1: With user-specific permissions, so do you mean uh the permission different users have uh access to certain files? Like maybe I I'm the only person with access to the files that I uploaded? sort of situation. Uh that's a great question. So I would uh the best way to do that is you could you would just the the same way that you would do that with any other view is uh you would you would put some Django permissions on it, right? So you would uh if you're using class-based views, you'd have a permissions required mix in and you could only access that view um through uh if you had the the appropriate permissions.

Speaker 1: And that would be a way to implement something that's user-specific, like you know, my user ID has to match uh some metadata about the file that I have in a model somewhere, maybe. As for the question of if you put something on AWS, well by default that's public. Anybody who has the direct link to that uh can just access that. But you can make it private. Django storages. uh I think supports this out of the box uh last time I looked at it uh and is actually kind of intuitive. And it and so I would I would recommend looking there. Django storage has has a way that you can keep file uploads on AWS private, uh, but still surface them in your app. And then you can protect them with Django permissions.

Speaker 2: Yeah, we're out of time, but you can still uh make questions outside. And just a reminder, we have the Group photo happening right now. So if you go straight outside on your left down the stairs, we'll be taking the photo outside. Thank you. That's great

Speaker 1: Thank you.