You Don't Need Containers to Run Django in Production with Eduardo Felipe Castegnaro

Well, hello everyone, I'm Eduardo. Uh and this talk is a little bit weird, okay? And I want to start saying that I don't actually hate containers or Docker or Kubernetes. So this is not a hate talk. This is actually more about the idea of needing versus wanting something, right? So There's nothing wrong with you wanting to have Docker on your small team or using Kubernetes form to manage like three pods. That's fine. But you don't need it And this is what this talk is about, what you actually need. And for people that are starting up, there are two talks from last year's PyCon that I really enjoyed and that motivated me to

start this talk. The first one is from Katie McLaughlin called What's Deployment Anyway from last year's Django Kong. It's online, it's amazing. And then On for from Itamar, we had a talk last year called Zero to Production Ready , a best best practices processes for Docker Packaging. And this talk is a 15-minute talk on packaging a single application. And he had like 120 slides. And it's thought, whoa, that's too much. For someone that's starting up, you don't actually need that. So I want to talk about when do you need containers, right? Because sometimes you do need them

And I want to talk when you don't need containers, which I believe is most of the time. What is expected from a production environment? I mean, why why do we have a difference between development and production? And what is a web server and how to choose one? There are a bunch of options for Django, and we're gonna go through a few of them. Um what is a reverse proxy and why you should have one and how to keep running everything all the time And last but not least, if you're not doing Docker and Kubernetes, what deployment actually looks like for a production environment, right? So The main thing here is when do you need containers?

Well, I believe that you need containers when your company has programmers in the thousands. Right? You are on one of those companies that are on hyper scale, hyper growth. You are doubling your team size every three months. And Things are chaos and containers do help you manage those those scenarios, that chaos. And You have users in the millions with unpredictable traffic. You have a huge spike out of nowhere, and you need to scale up really quickly. And last but not least, you have the valuation in the billions. So you can just pay for people exclusively to work on your infrastructure and Docker

and Kubernetes and manage all of those things. What about companies that have just a handful of programmers and maybe users in the thousands? So when don't you need containers? And my guess is pretty much the rest of the time, you actually don't need containers to run 99 % of the projects out there. Mainly because Only 1% of companies are actually these hyper-scale, hyper-growth companies. And there are lots of little companies doing just fine. So I want to do a little background on me and and who I am. And I work for one of those small companies and we have a SaaS platform that does digital advertising.

It's called OnSign TV, and we have been running it for over 10 years. We have steady growth, profitability, but there's just eight of us running the back-end team So we run our whole huge infrastructure on seven virtual machines. It's like do we really need containers for that case? Is that gonna help us But when you go online and search for this topic specifically, it feels like you should be ashamed of not using them, right? If you go On the hacker news, you think like you are the dinosaur in the tarpit just waiting for extinction to come for you, right? It's like Oh my god, I am not using Docker.

I should just go and hide. And a lot of that is container FOMO, right? Your fear of missing out on something great. But it's important to understand that containers Have a lot of of things attached to it. It's not just one issue So the advocates that they they talk about containers, one of the things they talk about is reproducibility, right? So how Are you going to write your code on your VM and deploy it to production and you're gonna have if you're running writing it on macOS and deploying to Linux, you're gonna have A bunch of issues regarding those two the differences between those two OS, but in my personal experience, in 10 years running on

Science TV, that happened exactly once We have a single issue that was caused by the fact that the file system on macOS is case insensitive and the file system on Linux is case sensitive that happened once in 10 years and was fixed in like five minutes. So do we really need this whole reproducibility argument to bring us down with a lot of complexity And when we are writing Django code, we are writing code at such a high level that most of those differences will be ironed out by the platform. So Django will do a great job, Python will do a great job, we don't have to worry. But let's say you don't want to hear me, right? You just you want

containers. So When you get containers, you get everything associated with it. This, this , and I'm not sure if you can read it because it's tiny, tiny text But the first thing written on top is overwhelmed? Don't worry, we have a guide So they have, it's called the Cloud Native Landscape Initiative, and they have a map of everything related to containers and deploy and being cloud scale. And it's just Huge. You can't possibly fandom one single person understanding all of that. And In an abstract way, we can think that containers is just a way to package your software so it can be executed.

It's like compiling your code before you run it, but on a larger scale. But the thing is, once you have a container, you can you have to actually run it. And you have you built your image, you have to run it. And if you Created a bunch of images of a bunch of different things, they need to talk to each other. So you will discover that you need a orchestration, something like Kubernetes. And then you need So much more, like so much more. And I think that that orchestration is a great thing and we can get so much from it, something like zero downtime deployment. But you notice that

those things can be can be achievable on your own without having all that complexity, and those things won't protect you from yourself Most of the downtimes we've had in the past were not caused by how we deploy the code. It was caused by someone not thinking that adding a column to a really big table will literally lock that table forever if the the column has a default value. That is if you're using Docker in Kubernetes or if you're deploying it yourself, the issue is the same. So I want to also talk about people with Kubernetes, they also talk about monitoring and the orchestrator keeping tabs on everything running.

And mostly things just work, right? If you create your software right, you won't expect a seg file out of nowhere. And if you do Every single system that has ever existed has a process manager. You don't need something that has like 20 million lines of code to just manage processes. And so this talk is for the small teams. It's for people doing self-hosting. It's for people that are like doing it themselves. And The thing is, about uh self-hosting is nothing more than you get a real machine or you get a virtual machine that has an OS.

And you just use it. That's it. And there are a bunch of uh companies out there, they are called platforms as a service, such as Heroku or Elastic Beanstalk And they provide things on top of those VMs. And in a way, Kubernetes was created to avoid vendor lock-in from those platforms. But You also have to understand that Kubernetes was created for harpers scale companies. And so they have much, much more than just avoiding vendor lock-in. And one thing that people usually don't talk about, and I do think it's unbelievable. I I talked to a friend of mine that have a little a startup in Brazil

and he was like, oh it's great, everything is fine, but we are spending like six hundred thousand dollars on AWS a month. And you're like, what's your revenue? Oh, we are like $100 ,000. We are running out of credit. And that's a real thing. That happens to companies that just outspend themselves by doing those integrations And then when you you have to fix that way later in your development life in your product lifecycle, you have already run out of cash. So We are talking about self-hosting not only as a way of understanding what you're doing, but also as a cost saving mechanism. So What do we want in production that we don't want in

development? And if you've been writing Django for any amount of time you know about run server, right? It's the the full way of the of running a python application And you think, okay, I've running my code with run server, everything's working fine, and I gotta put it put it somewhere, other people are gonna access it. Well, that environment is inherently different from your machine because the internet is the wild, wild west. And every single one is malicious. There's no good people on the internet. So in production, you need a secure environment. You need some protection against malicious

clients and you don't have To you don't want to wake up in the middle of the night because someone crashed your site. And you also want to be able to fix things easily. So seamless updates are Something really good for a production environment. So this is what you want. How can you achieve that? Well, the first thing you need is a web server. And there are a bunch of alternatives from from the Django community. And A web server is something that will run your Python code. We we are making that distinction here because we also have a piece of very important infrastructure

that's called uh web server sometimes but it's we are talking about here as a reverse proxy it's an http server that will get all traffic from the web sanitize it somewhat and then forward it to our Django code And we want a process monitor, a way of making sure that our application is running in case of reboots or in case uh of seg faults. And the last thing we want is a way to deploy our updates without much of a fuss. And you see, I think that one thing that it's important for everyone to understand is that Scalability, it's a great issue to have.

Most of the time you won't need to be scalable. Most of the time, getting a bigger virtual machine is a hundred times easier than getting two smaller ones. So that's just a fact of introduce introducing on your platform things like Different code versions of your code running at the same time and dealing with all that complexity. So if you go the way of Kubernetes , You have those two for free. I mean for free as in you already have Kubernetes, right? So but you still need to choose the first two ones And that's important because uh Kubernetes has the concept of an ingress controller, of something that will receive external HTTP traffic.

And it's still up to you to configure that, right? So you're not getting it for free. And A web server is something that they just don't understand. They talk about a container and a container exposing an a port they will talk HTTP. So that's up to you. We're gonna start with what we need, regardless of Kubernetes or not, and then we move on to everything else. So Choosing a web server. Like I said, you have your code, you're doing uh you're doing run server, it's working fine, you're gonna push it to production, and then you check They run server documentation to understand which option should I turn on to run this in production?

And you see this very nice warning where they literally scream at you that you should not use this in a production setting. It has not gone through security audits or performance tests. A they don't care. Everything that's within parentheses, like those four lines, is just we don't care So you have to choose it on your own. Python has a great Django has a great guide for that. But the guide doesn't actually help you choose. The guide just says what's available. So what should you choose? Well it depends. And that's why the guide doesn't actually tell you something, right? So every site its its own unique snowflake. It's up to you, the developer, to understand what you're doing.

But right now you have before you even think about web service you need to make a decision. You need to choose are you going the Whiskey way or the ASGI way. So the WSGI or the ASGI way. And We, as a community, have been evolving and developing ASGI for the past five years, and it is great, but there's always a button. And we're gonna talk about it. So the WSGI is the standard way, right? It was created almost 20 years ago, even before Django. At the time, every single framework did its own thing and there was no interoperability.

The development landscape was really fragmented. And WSGI unified that and allowed allowed a standard way of running Python web apps. And in those olden days, that was probably through mod wsgi in Apache. Right. But that was 20 years ago and the web has changed so much since then. I I mean we now have WebSockets We make a lot of HTTP requests in our code ourselves because we're talking to external APIs all the time. So things have become more asynchronous in by nature And nothing of that is supported by WSGI.

So about five years ago, we had a new protocol Called ASGI. It's not still a standard, but I believe that Andrew, who is here in the conference, might be proposing a path one day, one can hope. And he created that based on the idea that we need a new protocol to support asynchronous things. And He he started that by creating a project called Django Channels that was supposed to be the future of Django until it wasn't Sorry about that. But that that was a great, great exercise because many of those ideas are coming back to Django.

So from Django three point oh we got ASGI support and we get we got uh In the latest beta, in the latest version 4. 1, we gained support for our sync queries and new updates, which is amazing, but still no support for transactions, which is a bummer. So We still can't use everything we want in ASGI because it is like I said five years old and we as a community which I love we are slow people And I just love that because I don't have time in my small team to update Python every three weeks. Right? I need things to move at a slower pace

And the beautiful way of doing this is the way the Django is doing because you can mix and match as async and sync in your views if you're using a SGI You do have to pay a small penalty to penalty for that. We're gonna talk about it and we're gonna see it. But you can mix and match them. And having said all that, I think that ASGI still has a long way to go. So it still is only five years, but it shows. And we're going to venture now in the land of WSGI in the world, world, world of WSGI, starting with G Unicorn.

Who here hasn't heard of G Unicorn? Okay, so we got what hi. So you're gonna uh hear a lot about it today And G Unicorn is like hardcore old stuff, right? It it comes from a time where you fork for each worker It's like memory that doesn't exist. You just fork, fork, fork. And then it but that the the whole thing about forking one once for for every worker is that you have a great way of scaling if you scale your VM. And GUnicorn became the de facto way of running Django today.

And like every old software, which is something that I love, it has so much flexibility. Right, it is it it it it has like 85 configuration settings you don't have to use most of them It also takes configurations from environment variables, framework settings, configuration file, and environment variable called deunicore command args that is just to simulate a command line argument and command line and you can mix and match all those because this software It has passed every single test there is for integration and it's easy to integrate wherever you want.

And one of the great things about it is you can have multiple worker classes, right? So a worker class in DUNicore parlance is something that will actually serve your code. And in the old days, I'd say t about five, no, about seven years ago, before ASGI, that meant Uh that meant that we had async workers that could hack into Python's code and monkey patch everything so k Python could just pretend to be a sync all the time. They called greenlets or green threads and we don't do that anymore, thankfully. And

but still G Unicorn is a great great software And let's talk a little little bit about performance. And I'm gonna say that performance is not important, but I'm gonna show you why it's not important. So the thing is, the default worker of GUnicorn is called a sync worker. One request at a time. So there are no persistent connections. Every request will block the worker until it is complete. It is also really fast. So if you have let's say this this test we're doing here with a hundred clients, right, you can get 50% of your requests delivered in two

milliseconds and the the tail goes slightly up and if you have a hundred percent of your requests delivered in let four milliseconds or less That is just a plain request going through Django, working its way through the stack, and then being replied being delivered to this do to the client Now, if you scale up the number of clients, you will start getting a little performance degradation, right? So Especially on the tail end. You can see here that you most of your requests will be fine, will be under six milliseconds, but then things really tail up. And if you get 2000 second 2000 clients, that

that scaling up of of the the time it takes to answer 100% of your requests. Well that goes up, but you see that there's some linearity here, right? It doesn't scale linearly. It's it's more like a slow slope. None of the requests failed. It was able to have to answer 25,000 requests per second. In a single 16-core VM, it had 33 workers, no request failed. We just had to wait a little bit longer. And that is using the sync worker class. And let me tell you something. That previous slide was the best case scenario, right?

So you could not get faster than that. I have removed, I'll publish the code later, but I have removed everything that I could from Django and was just replying to a single request. So why do we have that? Because you need to understand that performance is more about the architecture of your code than about the web server you choose. And the thing is, with the default sync you are going to default sync uh worker From GUICO. The number of simultaneous requests is going to be limited by the number of workers because each worker can only answer one request at a time. And here comes the thing. If you do that, if your

little Python code is using, is connecting to an external API. Then you are limited by that external API. You just brought that requirement right into your code. And until you get one of those replies, you cannot reply it yourself. So you're just blocked. And there's another thing here that is because every work worker is its own distinct process, you will scale linearly with my workers in memory usage. For our product, it's about 20 hundred uh two hundred megabytes per worker, usually, but there is a fixed cost that you're gonna pay per workers.

And so you you can't really just think about adding more workers to overcome the limitation of only answering one request. And getting a huge VM with a bunch of CPU is not going to do you any good because you're still fixed in how many requests you can take. At once. But the great thing about GUnicorn is it's really easy to configure, right? You just need three configurations and you're good to go. And the first one that is the W is the number of workers, then you put in your application, and then you just set The temp work directory, which is really important if you're running on a cloud environment.

If you're not doing that today, please go and fix your code. And The the last is the socket you want to bind to. And one of the worker temp directory is important because one of the ways the GUICorn talks to itself. To make sure that a worker is running is writing to a file. And if you're doing that on the cloud, that file is probably somewhere else. So you're doing a network request Every request you have just to talk to the file system, it can lock things up. So, okay, that is the way of the G Unicorn. It is the old way. It exists, it has existed forever. But let's talk about the new stuff, right? So the new stuff which is ASGI You have Daphne.

Daphne is the OG ASGI server. It actually it was started in 2015, like I said Four Django channels and it actually created the ASGI standard itself and it was the first implementation. And definitely is based on twisted. Twisted here is for the old schools because twisted used to be all the rage in web development for Python in like 2003 , 2004. Everyone was getting twisted. And The thing is, Twisted is a great foundation for network development. It is a sync by default, so we are coming full circle of starting with a sync.

And then coming back to sync and then doing a sync again. And definitely, it is one of the few Frameworks that actually supports HTTP2 by itself. So you don't have to to configure something else to have HTTP2 support. It's still in active development. It released version 4. 1 like 10 days ago, I believe. It's like less than two weeks ago. And The thing is, it's a great piece of software, but it's also five years old, so it doesn't have a lot of knobs in terms of what you can configure, and sometimes you do need those knobs Now there's a new contender

on the on the ASGI space. It's called UV Corn. And I love those names. Like love, love, love. So UVCorn is a fast ASGI server. It has a focus on speed. How much faster than Daphne? Around 40%. And it uses a different loop called UV loop. It's a looping implement uh signaling eventing loop implementation. And that's why it's called UV corn. And one of the most important things about it is that it has a way to limit concurrency. Because remember on the first part where we talked about ASGI and we talked about How WSGI actually has a limit of based on the number of workers you have of how many requests you can answer at a single time?

Well ASGI doesn't have that limit, but you are still limited with how much memory you have. You're still limited in what machine is running your con your code, right? So having a way of limiting concurrency is really great because you don't let things balloon out of control. And the last thing about it that it's kinda kinda curious, and you're gonna see uh in a couple of new other topics is that it can be run under G unicorn as a worker. So G unicorn is like The it is like Emacs, it can do everything, right? You can start your own web browser on it And so how are we gonna take a look, quick look on G

Unicorn? And you see its performance is pretty much the same as G unicorn Things start scaling up a bit as the number of clients progresses. So still no errors, but you get a huge latency spike When you have more than it can handle. And that is based on a bunch of reasons, but it's also not very important, and I'm going to tell you why. So, because that scenario on that previous slide is different from the unicorn. That is the worst case scenario for UVCorn. It is not doing anything, I think, so you're just paying the penalty.

You're not actually using it for anything. So it can only get better. From G unicorn can only get worse. So it's very important to understand that distinction. And the thing is, we don't have the a maximum number of requests that we can handle with ASGI. You can set it, but you don't have to. And by doing that, you also bring in another problem that memory usage is less predictable. Memory usage can balloon out of control if you don't Take that down. And how can you take that down? Well, you can take that down by using a reverse proxy. A reverse proxy is also something that can limit the number of concurrent requests to your backend

And you just all all of those servers we talked about here today, from Daphne to To UVCorn, to G Unicorn, they all recommend having a reverse proxy. And a reverse proxy can also do SSL termination, which is adding SSL support, HTTPS support for your website. And one thing that that always makes me happy is that things like a reverse proxy, they are updated really slowly with new features and really fast with security issues. Nginx, if probably if there's a security issue on SSL, is what it was discovered on Nginx, right? So it's what everyone uses. So, one thing that it's important to understand regarding a reverse proxy is that not only

serves your code, it also serves static files. And static files is something that has changed a lot over the past few years, mostly because of HTTP2 and JavaScript modules. And the seven years ago I would be standing here and saying, you should bundle all your all your JavaScript in a single file. You should bundle all your CSS in a single file. Not anymore. That it's no longer required for you to have a good performance. But you should stat expire ahead this far in the future to catch it. And You should use manifest static static file storage because that is one thing that will keep your site working. Every time you change a line in your JavaScript

When you do a collect start uh collect static command, Django will actually calculate the hash of the file and prepend up actually append the hash to the name of the file. And this means that you can serve those files with an expire far in the future, like one year. And you won't be having any bugs relating to a client using an old version of your CSS or an old version of your JavaScript. It's it's great and you should you guys should use it. So Now we talk about Nginx, which is the ruling reverse proxy. It is the most used and most recommended of all of them. It is also

Incredibly fast. It is as fast as to be as to not be a part of your thought process when choosing something, right? Because I try to I I really try to find a way of hammering it down and thinking, oh, it's gonna use a lot of resources. So I could saturate a 10 gigabit network link with an eight-core VM and I still had 55 % of CPU left. So it it is a non -issue. That's why you should have it. It is really you you can get 20 million requests per minute on an eight-core VM serving static files. It's ridiculous. That's why people like Netflix still use Nginx

So the thing about Nginx is it is kind of hard to configure. It is arcane. Its syntax was written by someone without, I don't know, a degree in in language and language design. So it is kind of complex. And it doesn't do things that we consider table stakes today like HTTPS. So here is a sample configuration of Nginx. And it this is just to serve static files. And then to forward it to GUnicorn or UVCorn. Well, it's it's actually there's a little bit more. It wasn't it didn't fit in a single slide.

And today we have newer alternatives that are easier. Let me just compare that to this whole configuration, which offers HTTPS support for CADI. Caddy is HTTPI out of the box, it has great default values, and you can literally configure it to be a reverse proxy in five lines. It can't get simpler than that. I mean, there's nothing that you can remove because it's all there. So if you're starting today, maybe give give Katie a go. Don't sleep on it just because everyone uses Nginx. Like I said. Caddy is plenty fast for you and it's much much easier to configure. So Now we're gonna talk about keeping everything running all the time.

And keeping everything running here there used to be a lot of motion in this area ten years ago. You had a lot of process managers. You had from native ones from the system like system vnit or upstart. Then you had supervisor D and a bunch of other monitors, but they all converged to a single one called system D. System D looks it is really complex and looks scary. And I guess the best thing about system D is that everyone uses it. So you it's really easy to find good examples, really easy to find good tutorials, really easy to find good documentation, because

once it's everywhere, then you don't have a lot of of You have nowhere to run. And with system D, you can this is pretty much a complete example. So all you have to really do is You have to set what is going to run with this exact start line, which is actually your G Unicorn process. And then a way to reload it. And this is what's really interesting. Because if you do everything right, by the time we're done, you're going to be able to reload without losing a single uh single request. It's going to be easy and it's going to be free.

As long as you line all your ducks in a row. And so How I like like I said, how are you going to deploy your code doing all that? Well If you're using Docker, you're going to build an image and upload it to an image, to an image archive, and then you will deploy it, use Kubernetes will deploy it and will just rotate every single instance of that image. And to a different container. If you're doing it yourself, my recommended way would be to get the apply code. How many people here have done this in the past? Git deploy code is not hard to do at all.

It is quite easy. So when you're doing git deploy If you can SSH to a VM, you can get deployed to it. Simple. What are the permission permission models? Do I need an IM account? No. You actually need an SSH key. And if you can go you you can actually deploy to it. This is I'm sorry. Oh there you go Before you deploy to a VM, you actually need to configure two things on your git branch on that VM. The first one is You need to to set deny current branch to false because you're deploying to what's already there and git by default denies it.

So you're setting it to false but git config deny current branch false And then you need to put a hook. And that hook is gonna call a script that will actually do the deploying for you. And you think oh my god now I have to write a script. I didn't want to write a script. So I'm going back to Docker. Well Docker file is a script, right? So if you do one you can do another. It's not that different And it doesn't have to be really complex. You just override the files, install dependencies, collect your static files, reload your code, and that's it. So How's that how does that look like? Well, looks like this. This is a complete example So you activate your environment, you enter your folder, you install your requirements, you migrate your database, you collect your static files, and you reload your unicorn

If you did everything that I set up to now, if you did a manifest static storage for your static files and you use G Unicorn and you used System D, which is controlled by the system CTL command right there. If you did all that, you got zero downtime deployments for free. Like I said, the same kind of free that you're actually going to get with Docker because most of the time the down most of the time the downtime is going to be on your code. Trust me. I've seen that So let's wrap things up. And in conclusion, what I want you to take away from this talk, okay? The thing I wanted to talk about is

Don't be afraid of doing things yourself, right? You can choose the tools that work for you. And if you're a small team, you should prefer a stable technology stack. If something has an LTS version is a great signal that that software is for real. If you're doing a new version every six months and completely change changing your API, a small team doesn't have time for that. And last but not least, performance is less important than architecture. When deciding between WSGI or ASGI, it's more important to look at your own code and see: are you going to do a lot of requests?

Are you going to do a lot of a sync code? Then use S ASGI. And don't feel ashamed of just using WSGI. I mean, alone sign TV. All our screens are connected to our service. We have over 50,000 simultaneous WebSockets connections, and we use Go for that. Did I offend anyone? I'm sorry, but don't use Python for that. So don't be afraid of mix and matching what you get. Because the end result is what matters. Thank you very much.