What is deployment, anyway?

Hi, I'm Katie and this is What is Deployment Anyway? This isn't the talk you're expecting. This talk isn't about the one true way to deploy your Django application. No talk can be that This also isn't a talk about taking your next dot-com idea and your VC funded startup and making it all awesome. That's a different talk. If you're interested in chatting about that stuff, here's my work email. This is more about the first time you want to take your Django application that you've written, a tiny little blog, and share it with a friend. This is more about that sort of smaller concept

use case. This is going to answer the question posed in the title of this talk. What is deployment anyway? Specifically, what is Django Deployment Anyway? And it's going to cover mostly the what. We'll be discussing some of the when and the how, but this is mostly a what talk. This talk is also going to be based on Django 3. 2 and Python 3. 9, the most recent minor versions available at the time of request. recording. If you're joining me from the gear 2032, I'm sorry this talk might just be a little bit out of date. So let's start with Django. As it comes, out of the box.

This is useful if you haven't created a new Django project in a while. It's always refreshing to know where it all starts. So in my terminal, I want to install Django. So I type pip install Django. Pip then goes and collects Django and its main dependencies and installs it for me. Then I want to start a new project, so for that I'll use the Django admin CLI that was just installed, and I'll use the management command star project, which will create a template project for me. I'll call it my project and I'll create it in the current folder. So I have no output from this command, but if I check what folder files being created I can see that I have a manage. py file

and then a my folder a my project folder with some different files in there looks like there's an asgi. py, a settings. py, a urls. py, and a wsgi. py. Huh. We'll get back to those later, I'm sure. So, learning my terminal, I now want to start up Django. So I start by running python managed. pyrun server. And when that runs I get a great big red error message saying that I have unapplied migrations. but Django has helpfully suggested that I run the python manage. py migrate command to apply them. Great, let's quit out that process. and run that command. And it looks like all the migrations have been successfully applied.

That's great. So let's see what's in my folder now. Ah, I see I have a new file. db. sqlit3. Okay, that's great. I should now be able to run my application. So if I run run server again, oh hey, no big red error message. And look, it says that I have a development server running at this URL. Great! The rocket ship of successful installation. I also know that I can access now the Django admin, but I don't have a username or password yet, so let's go back and create those. stop the process, clear my terminal, and then I run the management command create superuser. This command will create a superuser for me that will have access to the Django admin

So I'll use my name. I don't need an email for this. I'll use a super secure secret password twice. And now I have a super user So then if I restart my web server, I can then go to that URL, go to the admin login, enter in my details, and I have an admin. Hooray! Django has a great local development story. We can get our app running really easily, and a lot of this boils down to run server. This command runs our application for us and it also does nice things like automatically restart when we edit our code The Django documentation describes what Run Server does. Seriously, the Django docs are great, and they are very good at explaining what Django does.

If you're newer to Django or web development or tech in general, some of the docs can be hard to parse and understand, but that's okay, that's why I'm here. So let's look at the docs. From the documentation for the RunServer command. It starts a lightweight development web server on the local machine. But if you scroll down a few paragraphs it says in all caps Do not use this server in a production setting. Why? It continues. It has not undergone any security audits or performance tests, and that's how it's going to stay We are in the business of making web frameworks, not web servers. Let me pull that out onto its own slide.

We are in the business of making web frameworks, not web servers. Django is an extremely stable production ready web framework. Django is very good at being a web framework. The fact that it provides a local web server is amazing, but the fact that it calls it out its limitations is even better. So we're gonna need to find another web server. This documentation also brings up some important terms. What is production anyway? And why is it called production? Historically, when computers were fed with punch cards, you could say it was a production to process data. You had to physically move different stacks of these cards around, much like a manufacturing

uh production line. You could also think of it nowadays as a stage production. It is a thing that patrons, attendees, or users come to see. Your production environment is the one you want people to go to. It is your live environment, and it needs to be able to support a crowd. The size of your crowd is going to influence when you start thinking about things like load balancing and caching and scaling, but for right now, it's just us and our friend who want to be able to see this website. Not only is RunServer a development web server, RunServer also serves the pretty, that being the images, CSS, and JavaScript that come with our application. To be able to see a nicely

centrally formatted blue on grey login window, there needs to be CSS to make that happen. These come with Django itself, but if you don't have that set up correctly your website login can look very 1997. I mean it's still usable. Web accessibility for the win But you can tell that something isn't working. Collectively, all of these styling files are known as static. But why is it called static? They're called static because they're unchanging. They came with Django itself and they don't move. They have to be served so that the browser can make use of them, but they don't normally change. For instance If we take a look at the source of that login screen, we can see that we have some CSS

and JavaScript being served from a slash static path. The Django documentation describes how a run server achieves this. In the docs for serving static files during development, there's a bit here, this is not suitable for a production setting. But it's also noted that Runserver will serve these files if you have debugs set to true. Well, we can see that these files are being served. So we should have debug set to true, right? How can we check? Well, this is where we start looking at some of that code that was automatically created when we ran Start Project. If we remember from our file listing, we have a settings. py sitting under our My Project folder. So let's check what's inside that. This one

's quite long and was automatically created for us, but one of the things we see is that these are quick start development settings. Unsuitable for production And these warnings just keep going. We go down the page and we see security warning after security warning These settings all work for a local development environment, but it's completely understandable that all these warnings and stuff Just make it seem so inaccessible to the beginning Django developer who just wants to share their cool blog with their friends This lack of knowledge is because they are a beginning Django developer. They are not a server operations person or a system administrator. It's a completely

different engineering specialization. This is the entire reason for this talk. To demystify deployment. Oh look, there's that debug setting. It is set to true. That makes sense. While we're here, we also saw that there was that db. sqlite3 file that appeared after we ran our migrations. If we scroll down the settings file, we can see that string is being defined here as part of our database's definition. So what this SQL Lite thing to the documentation SQL Lite provides an excellent development alternative for applications that are predominantly read-only or require a smaller installation footprint. Again, that word

development. Django has a great local development story. You saw how quickly we could get our Django site working on our local machine. But to get our site into production is complex. As we've discovered together, you need to provide your own web server, your own database, your own static hosting. And if you aren't familiar with the production grade offerings out there, you'll end up confused. Or worse, using RUMServer in production. Which You should not do. Great work. Oh dear, I've enabled the audience participation section too early. What was that disembodied voice from the back of the room? You had a question? Flask isn't this hard, not a question.

But let's talk about that. Flask is simpler in production. A lot of deployment tutorials you'll see will have a how to deploy Python in production that will often use Flask as the target. Just copy your code and run, it's easy. Well, that's true. You can get an out-of-the-box Hello World application running really easily on many different platforms using Flask. The issue Django has is one of state. Django is a stateful application, full of state. That is, it contains information and data that we really want to keep hold of. The bare bones Django app we just ran comes with the Django admin.

The Django admin requires migrations and requires a database. Therefore Django requires a database. Therefore Django re By and state A Flask application that prints Hello World does not need a database. Yes, you can absolutely deploy Flask with a database and Django without a database. But any application that has state is going to be complex to deploy. Let me share another gem from the Django Docs Earlier in this section on serving static files, there's this link that goes to deploying static files, which links out to a page of strategies for how to deploy static files. But it also has this line. As with all deployment tasks, the devil is in the details. Every production setup will be a bit different.

Every production setup will be a bit different. This line has been in the documentation for over a decade now and it still holds true. I have never seen two production setups that are the same. Most every deployment talk you will see this line saying it depends when telling you the one true way to do deployments. Because it's true, it does depend. But I'm going to use a different line. I am a sysadmin, I'm not your sysadmin. I'm not the one who will be making sure that your Django site stays online. So with any recommendations that I make in this talk, they are just that. Recommendations, based on my own experience It's up to you and your team to understand what you're doing

because you're the ones that are gonna have to fix it when it's broken. Not me. That being said. We need to find these components that we need to upgrade to bring them into production ready values. We need a production grade web server, a production grade database, and production grade static. Django has support for a number of the common options of these, and they will really help us out here. Because they are common options that aren't just Django specific, it means that we can find solutions that will work for the generic use case for all web developers, not just Django developers. This group of people is much larger and there are many more options and they have much more support.

For instance, Flask As I've said before, Flask is simpler in production. This is because you may not need a database or static to host a Flask web app, you just need a web server. And the reason you can get Flask up and running super easily is that hosting a site with a web server is simpler than it's ever been. Let's look at the flask documentation. Yes, I'm showing you the flask documentation at a DjangoCon fight me in the hallway. The flask documentation doesn't need me to do the highlighting here because it's already in bold. Flask's built-in web server is not suitable for production. They also have a list of hosting options. All these link out to various hosting vendors documentation and show you step by step the ways to just copy your code and host it.

This isn't a complete list of course. The host you choose is going to depend on several factors. What platform do you already know? What platform does your team already know how to support? Or what platform are you interested in learning? It's your choice. There are going to be different options for the same platform as well. Some of them are going to provide a complete ready-to-go Python environment for you. You just have to supply the code. Others you're going to have to configure and supply a little bit more. A lot of them will cover basic load balancing for you by auto-scaling, sometimes even scaling to zero, which means that your host isn't running when no one's using it, saving you money They will also provide HTTPS by default, handling

SSL certification for you, one less thing you have to worry about. You can of course do all these things yourself, but if it's just you, your friend and your blog You don't have to do this yourself. Hosting options are always evolving and the best practices change over time Django doesn't list things because they keep on changing. But Django does tell you explicitly where its responsibility stops and where it tells you to go off and have fun storming the castle Many hosting providers out there will have explicit how-to tutorials on how to deploy Django on their platform. I should know I've written a bunch of them. And that will get you the how as of now. We're going to continue with the what

here though. The flask ducks go on to mention this WSGI thing again? We saw that earlier, didn't we? Way back in the beginning with the Django run server docs. Sitting there, there was this line. This server uses the WSGI application object specified by the WSGI underscore application setting. Wait, that was in our settings file, just above databases. We saw wsgi application equals myproject. wsgi. application Wait, we saw that earlier. In our file listing we have a myproject slash wsgipile. So what's in that? This is the entire file and most of it is comments, but we can see from those comments

that this exposes a WSGI callable as a module-level variable named application. Oh look, a documentation link, let's go there. These are the docs that say that Django 's primary deployment platform is WSGI, the Python standard for web servers and applications. Well what's WSGI then? To the Python documentation. This is PEP333, or the Python Enhancement Proposal for the Python Web Server s Web Server Gateway Interface. The documentation abstract shares pretty much what's in the box. This document specifies a proposed standard interface between web servers and the Python web

applications or frameworks to promote web application portability across a number of web servers. This is the portability we're taking advantage of here. All those potential web hosting options from earlier, they either directly provide or allow you to easily provide a WISGI server to host your web application. This is why web hosting is so easy nowadays. You could create a virtual machine and do all this manually, continuing with your firewall and routing configuration. But a lot of these places will say, give us your code, now here is a URL for your website, have fun. They make things so much easier, because this is a problem that all web developers have. And web hosting providers want to make things easy for you so they'll

you'll use them. So this is a service they provide And of course it's not just WSGI now, there is also ASGI or asynchronous server gateway interfaces. you may have noticed an asgi. py file from the project template earlier. For right now, unless you're specifically playing around with async stuff, you don't have to worry about asgi. We'll keep to WSGI. So we've established that you need a WSGI server on your platform of choice. There are many options around there, but the one I commonly use is G Unicorn. It's already been listed on the Flask and Django pages we've seen so far. It is functionally very similar to RunServer's main

function. It serves websites, but more importantly, it's a web server that's actually being security reviewed and is suitable for production. You install it as a Python package, and then you can run the GUNOCunga man specifying your WISGI app. So we can do just that In our terminal, pip install gunicorn, which Pip does, and then we run gunicorn myproject. wizGi call an application And then we run our web server. And look, it's running in the exact same place as web server was. We have the rocket ship of success! But our admin is very 1990s. And we know that this is because of our previous pokering into static issues. Indeed, looking at our terminal, all of our assets aren't found

This is because Runserver handled static for us, which GUInicorn doesn't do. So we can use GUInicorn as our web server, but it doesn't handle the static, so that's still an extra component. But that's okay. we want to choose the best available components out there. As for hosting GUnicorn, how you deploy is going to depend on your host of choice. Check out their documentation. But now that we've got a web server and we've chosen a host, we now need to connect our stateful services, our database and our static. your host will probably have these offerings, starting with the database. Unless you have experience in another database that Django supports, or a team that knows how to just use another database, I suggest using Postgres

Django supports many databases but Postgres has the most support. From the reference documentation, it says Django is and will continue to be a database agnostic web framework. We encourage those writing reusable applications for the Django community to write database agnostic code where practical. However, we recognize that real real-world projects written in Django need not be database agnostic. Django provides support for a number of data types that will only work in PostgreSQL. There is no fundamental reason why A MySQL module doesn't exist, except that Postgres has the richest feature set of supported databases, and so its users have the most to gain.

A lot of people put a lot of work into Django, particularly supporting Postgres. If you would like to get more support for your preferred database, go for it. We would love to have more support for these things. As of right now, the answer is use Postgres. So we've decided that our database is going to be Postgres. Well, how do we get that working? Well we're going to need to edit our settings file for one. In our original settings file, the engine is specifically that SQL Lite3 version, so we're going to need to replace that with Postgres. And then we need to set up our Postgres server on our host of choice. and we'll be provided with information on how to connect to it. There'll probably be a host, maybe a port that can be accessed from, and we'll also be given a database for

the database on that server instance that's ours. And we probably want to make sure that this data is secure. So we'll need to specify a username in P Carl's word, but if I ever see you doing these in plain text in your settings file, I am going to be upset What you should be doing is having these values stored securely. There are many choices for this. Your host may allow you to set encrypted environment variables. Then you can configure your database settings to pull these from your environment with os. inviron. get. One thing I like to do is set an environment file because these are a lot of individual settings to have to keep a track of. The package I use for this is called Django Environ. It's a package that allows you to use the 12-factor methodology to configure your application with environment variables.

What this code snippet does is it'll load your environment and then read the file and augment that, making them all available in a variable called env. But Katie, what is this 12 factor methodology thing? Well The 12 factors are a methodology to building web apps to make them easier to scale, maintain, and deploy. It was developed by the people that made Heroku and talks about all the lessons they learned there. Now this isn't a talk about 12 factor, but I will say that this is the method we're already using to deploy our application, and we're already covering a bunch of the factors. For example, dependencies, a requirements. txt file. Config, literally what we're doing now.

Backing services, this is what our database and static are They're attaching to our service and port binding, that's our 8000 port. If you're using a hosted service, you're already going to get processors, concurrency, deposability, and logging for free. These are the concepts that these providers work from. Instead of giving you an entire machine, they give you part of a virtual machine that you can use. Them implementing these factors help them scale. Of the ones that we haven't covered, you can think about using GitHub to store your code base, using your host build process to deploy your site from GitHub. and then replicate that setup for your development environment, your testing environment, your production environment.

And then your admin process, that's just your migrate command. There are many great talks that step into this methodology more, both in general and for Django. But we should give back to implementing Django Environ. So, I create an end file and I put my values in there, including other things like my secret key that I should not have in plain text in my settings file. I then install Django Environ, which I can then use in my settings file. After a little bit of code at the top of my file based on that snippet from the documentation, I can then change my code to use env to pull in my settings. These will pull from either the environment variables I have. or a file if I want. That file is going to be extremely useful for local development.

Check out the Django Environ documentation for more ways that you can use this package. The database URL feature is quite neat. So we need Postgres for feature coverage and Django and Viron for configuration portability. And since we have a database, we need to actually apply the migrations to it. That's the admin process we mentioned earlier, running manage. py migrate. Later you're going to have to consider about when you'll run these migrations and if you want to automatically make them or do any customizations. Marcus Holterman is a subject matter expert on this. Check out some of his talks and blog posts for more information. But back to the what. We've got our database sorted, we've got our web server sorted, and now for static.

Again, your host of choice is going to have a backing server solution for this. It'll be an object store of some sort These are some of the oldest parts of cloud computing and are exceptionally robust. There is also a help package for Django to help integrating these options into your application. Django Storages is a collection of these custom storage backends. These hook pretty deep into Django and allow you to do all sorts of things with whatever static host you use. The documentation will guide you through the config you need for each type. And you'll end up adding things at the end of your settings file somewhere around where you have the static URL setting. Now we have a place to put our static, but what do we put in there?

How do we collect our static? We know we're missing this step as our admin looked very 1. 0. But you see that's another one of those hour words. The command you're gonna need is called collect static. But how to do this is a completely separate talk in and of itself. A good example of this talk is Jacob Kaplan Moss's Assets in Django Talk. He goes through different strategies for Static, including a deeper dive into what Collect Static does. So there we have it. Use GUnicorn as a web server on your host of choice. Use a hosted Postgres database with Django Environ for settings management, applying said migrations with migrate. and use hosted static with Django Storages applying with collect static. This is my recommendation of what deployment is in 2021.

People in 2032, that disclaimer is also for you Things may have changed. Giving this talk is going to be an artifact of its time. Please check the Django docs for more up to date recommendations. Try creating a project from the template and see what it has in the comments. I've shown you how to do this. Just repeat this process. There are a lot of things about the how and the when that I just didn't have time to cover today. There is a lot in this space and it's absolutely okay if you don't know about all of them. Each one of these are its own talk, worst case its own conference. The specialization of operations engineering is complex, opinionated, and ever

evolving. But what we've done is just give you a taste about what Django needs to run, given you some help parsing the base documentation. and understood some of the concepts and suggestions it's making and given new helpers to help you deploy. Thank you so much for watching. I am Gleasant, G-L-A-S-N-T on all the platforms no matter what those are in 2032. And if your CTO wants to talk to me about highly scalable serverless applications and Django hosting, here's my work email Have a brilliant rest of your day.