Type UWSGI; Press Enter; What Happens? by Philip James

Speaker 1: Thank you very much. Welcome to Type You WizGee. Press enter what happens. If this is the talk you're here for, congratulations. If not, you're totally free to leave. And in fact, to give you that opportunity, I'm gonna ask you all to do three things for me real quick. First thing I'm gonna ask everybody to do, it's late in the day. Everybody stand up real quick. We're just gonna get we're gonna get ready for the talk. Please place your hand over your heart. Take a step to the left Fantastic. Now you can say that if no matter else what happens, the talk was uplifting, heartwarming, and moving. So let's say you've built a Django app, you're ready to share it with the world, you tell your friends about it, and they think, hey, this thing is really great, but when you deploy it, when you put it out for the world, you shouldn't be using Django

Speaker 1: run server, you should be using eWisGee. USGI, you say. What's UWISGI? And you're a clever developer, so you go and look at the USGI project documentation, and it says things about being a proxy, it says things about uh taking care of protocols and managing processes that you can apparently write plugins in C, C<unk>, and Objective C. And this seems very informative, but it doesn't answer the fundamental questions you have about what is UWISGI, what's it doing, and why should you use it And those are the questions we're going to try to answer today. How does UWISGI handle processes from the operating system level? How does UWISGI handle networking from the operating system level? And then why should you use UWISGI? And to tell you that, I'm going to tell you about like why I use UWISG.

Speaker 1: My name is Philip James. If you were at DjangoCon last year, you might remember my talk, Frog and Toad, learn about Django Security. If you've been to other PyCons, you may have seen another talk of mine type Python press enter what happens. And in that talk, I went through the What it was on the box, what happens from the operating system level when you type Python and press enter? You may be aware that if you type Python at Python and press enter, you get the prompt at the terminal, but what is the operating system doing under the hood? to make that happen. And so in that talk, we talked about how there are these things called processes, and a process that you might use to start Python is a process called bash. Bash starts the Python process by making a copy of itself

Speaker 1: and then changing that copy to the Python process through a technique known as forking and execing. And that process gets access to these things called file descriptors, which are the mechanisms that are used by the Python process for input and output, for you to communicate with Python and for Python to communicate with you. And that's the basis for our starting point here of how does UWISG handle processes? And to talk about that, we're going to talk about an app that I created a while back. I couldn't find it anywhere else on the internet. I was really surprised, but cats serve cats as a service. If you are in need of a cat as a service in your life, there's a GitHub link that you can go check out. But the app is pretty simple. You go

Speaker 1: to cat serve and you get a cat. Now, I did some digging because some people told me to use UWISGI when deploying this, and so I found a Uizgi incantation. that would let me run this app from my server. Type UWSGI, press enter, what happens? And what happens if you do this incantation for UWISGI? And actually, quick show of hands, how many people have used UWISGI before ever in any context? Okay, of those people who have their hands raised, how many people knew that knew that you could run UWISGI from the command line, just like blank command line options? Okay, not as many. And that was a realization for many of us as well, is that UWISGI, like many things on Unix systems, is a command.

Speaker 1: You can run it the way that you would run it outside of a daemon. And so if you type UWISG and press enter with these incantation with this particular options, you get this a bunch of output. We've condensed some of it, and the line that I really want to point out here is that line at the bottom. Spawned UWISGY worker one. And it has this thing called a PID. Now you might remember from our previous slide that Bash and Python had PIDs. Those are the process IDs. And this line is UWISG telling you, hey, I have spawned one worker, one thing that is going to run your app , and there's the process ID where you can find it. And so what that looks like, if we were to diagram it a bit, is there is this UWISGI process that you have started. UWISGI forks itself and execs

Speaker 1: your Python app to make this running Python process running your app. And when a request comes in, now uh that worker is busy and it's serving your request. And that's great. But we wanted to add an extra special feature to this web app. You know, we've been hearing a lot about machine learning. Um we really want to be able to serve you the best cat, not just any cat, but the best cat specific Specifically tailored for you. So I put a bunch of uh tracking pixels all across the internet to collect data on what was going to be the best cat for you. And we put this behind this beta feature. And we we tried really hard, but we couldn't get The time to load the beta best cat for you under 10 seconds.

Speaker 1: We think we can maybe do better in the future, but right now the page takes 10 seconds. And that's a bit of a problem because as we see here If I'm loading the beta, which you can see, and then I try to load the normal homepage, well the homepage used to load really fast and now it's not. And in fact, if we wait the full 10 seconds, we'll see that it's only after the beta page loads, with this highly tailored cat for you, that the normal home page cat loads. And that's really not what we want. You know, it we don't want the beta feature to be basically making the rest of our app worse. And so what we need to do is have you WISGI start more than one process. For Django apps especially,

Speaker 1: if you want to be able to serve multiple requests at the same time, you need to be s uh having Django run with multiple processes. And so here we've added this dash P2 to the end of the command that we're using to run UWISGI. And UWISGI tells us that it spawned not just one, but two UWISGI workers with two different process IDs. And so if we go back to our beta, we'll see that we can have the beta loading in one window and the homepage now loads independently. Hooray! We've fixed our app again. Going back to the diagram, the way this works is when UWISGI forks and execs, it forks and execs two processes. And so while one is busy serving the beta page, the other one can be serving our home page, and our users still get a pretty good experience

Speaker 1: So that is basically the core of how UWISGI, and realistically, any other Python web app server like GUnicorn or some of the mod libraries for Nginx or uh Apache are going to handle this. They're going to fork multiple processes and ha route requests to each of those different processes. But let's talk about what I mean when I say routing requests to those different processes. So if we go back to that command, I'm passing in HTTP 8000, and that's telling you WISGI, I want you to run this application on that port. And so if we look a bit closer, we're gonna see look at the middle section of output now, where we see some kind of relevant information that USGI is telling us

Speaker 1: It's telling us that it's bound 8000 uh FD4, it's spawned HTTP 1 on a master process 1220, there's some other FDs there. And if you remember from talking about processes at the beginning we talked about file descriptors being how input and output is connected to Unix processes. And you WisGee is giving us this hint here. That these FDs are the file descriptors to these processes that are being passed around and bound to let USG talk to the internet. So let's talk a little bit more about that. Let's say you want to connect to a remote server. And the remote server here is going to be a box. And the way you connect to a remote server is through ports. The ports here are represented as telephone jacks because a way that you can think about ports on your operating system is that they are like telephone lines that are waiting for things to connect to them.

Speaker 1: And so you want to SSH into this box. You're probably going to use port 22. If you've SSH'd into a box ever before, you've probably used port 22. If you've Used Git, you've probably used port 22 because you've probably used it in uh SSH mode. And so the first thing that uh SSH is going to do is it's going to connect this phone line Which is our metaphor for a socket. It's going to connect a socket to port 22, and then it's going to use that socket to communicate through the port to the outside world. Now UWISGI is doing a similar thing on port 8000, where it's connecting this phone line, this socket, to port 8000 and then using that to communicate and accept connections coming in from outside.

Speaker 1: There's a little bit more complexity. Both the socket and port functions are happening in a space called the kernel. The kernel is the the core of the operating system. It's where all of the interfaces Interface with hardware happens and it turns out if you want to do networking, you have to interface with hardware at some point. And the processes like SSHD, which is running SSH, and UWISGI, which are running your server, are happening in user land And there's a mechanism for user land processes to talk to the kernel, and that's through syscalls. Syscalls are these special functions written in C and normally accessed in C that can tie right into the kernel. Let's go a bit deeper. Let's say you're on your computer, you're in Chrome, and you want to connect to your Whiskey server on port 8000. Well you know that port

Speaker 1: 8000 is going to be accessed by the kernel. Uh and the first and if UWISGI wants to bind itself, wants to connect to port 8000 so it can receive connections. First of all, the first thing you would notice is if you were to make this connection from Chrome to port 8000 before any of the connections have been done, you would get a connection refused error, right? Because there's no port exposed. Uh the kernel doesn't know what to do with this, so it just tosses it back at you. And so UWISGI needs to issue some syscalls to kind of get things set up so that it can accept requests from your browser. The first thing it's going to do is issue the socket syscall to create that telephone line, to create that socket. And it's going to be can passed that socket as a file descriptor. Because if you're familiar with the Unix

Speaker 1: philosophy where everything needs to be some sort of file or at least file-like object. UWISGI needs this file descriptor handle to the socket that it's just created. UWISGI then calls bind. And what bind is going to do is going to tell the kernel, hey, I've created this socket. Please connect this socket to the port that I specify. So it's connected to port 8000. And then UWISGI is going to call listen. What listen is going to do is it's going to tell the kernel, okay, everything's set up, ready to go. Please expose this port 8000 to the outside world so that I can start accepting connections. Now it's at this point that UWISGI is going to do its fork and execing to create the worker processes. And what's interesting about forking and execing is when you fork and exec a process, you get the same file descriptor references that the parent process had.

Speaker 1: So both the USGI master process and both worker processes have the file descriptor access to the socket that was created for port 8000. And these two worker processes are going to call another special assist call called e-poll wait, which is effectively these worker processes telling the kernel, hey, we're both waiting for a telephone call. We're waiting by the phone. As soon as somebody calls the phone, we're ready to pick it up. So now you can make your request from Chrome against port 8000. Uh the phone rings. Uh and both processes try to pick it up at the same time. They both have access to the file descriptor, and they're both going to reach for the phone. But they can't both answer the phone.

Speaker 1: There one of them is going to win first. They're both gonna try to call accept to both try to pick up that phone at the same time, but only one's going to win And the one that wins is going to get a new socket, which represents the connection from port eight that from Chrome through port 8000 to the Python worker process. And it's now going to be handling the request while the other Python process issues the e pull wait syscall again letting the kernel know hey so the other guy won the first time but if a new request comes in I'm totally ready for it. And that was a very long, complicated diagram, but that's basically the core of it. Through a combination of syscalls and sockets, uh UWISG sets up a connection and then passes that

Speaker 1: a reference to that connection. to its worker processes, its child worker processes, and those child worker processes accept the connection to handle the request. So now we get into, now that we've talked about how UWISGI handles processes and we've covered a bit about how UWISGI handles networking for handling requests, why would you use UWIG? And the first couple things that I'm going to talk about are not really specific to UWSGI. They're kind of applicable to any Python application server. Uh but a couple are going to be specific to UWSGI and I'll call those out. So the first thing is code reloading. This is something that the Django run server really can't do. If you are trying to Deploy a new version of your code.

Speaker 1: You've got code running on a service and you want to deploy a new version. You want to be able to deploy that version without shutting down your entire server and bringing it or shutting down the application server and bringing it all the way back up So when UWISGI detects that there is a new code to be run, when you tell it, hey, I would like to change out the code on the workers, it sends a signal to its worker processes. The hangup signal, SIG HUP, telling the workers, hey, as soon as you're done, please exit and reload yourself And so in this example, we've got one worker that's busy. Both workers get the SIG hups. One isn't busy, and so the one that isn't busy shuts down immediately. The one that is busy kind of gets a little notification Indicated by the power symbol, hey, when you're done processing your current request, you should shut down because

Speaker 1: you need to be reloaded. It completes its request. it notices that, oh, I should refresh myself. It also shuts down. Now notice throughout this entire process, you whiskey held onto that socket that was connected to the outside world. So you whiskey didn't never dropped a connection More connections might have been added to the queue during that time, but UWISGI was always maintaining its presence the outside world, so from the user's point of view, it may have been a little slower, but they never got the connection refused. So that's reason number one, you might use UWSGI, but that's pretty common. Reason number two, tunability. Earlier we noticed that when we were running our beta feature, we needed more than one process. In order to serve our application well and not have any users be blocked on our beta feature.

Speaker 1: But why two processes? Why not 20? Why not 200? And the answer is UWISGI will let you spawn as many worker processes as you reasonably want, but there's going to be a right number of worker processes for your system. In my testing, a Python application worker takes about 200 megabytes of RAM. If your machine has about 2 gigabytes of RAM because you're running a small server, maybe the right number for you is 10 or 8 if you want to give some wiggle room. But the important note here is that UWSGI lets you tune basically every parameter you can think of to match what your system needs. Again, this one we think UWISG makes it easier, but it's also something you can do with G Unicorn and a bunch of the others. Uh security.

Speaker 1: There's a very so overall the Django team does an incredible job with security. In any part of the framework that you would use on a regular basis that actually kind of run um composes the application. They are very quick to respond to security holes. They release patches quickly. But there is one area of the Django code base where they have said quite explicitly they will not do any sort of security audit or do any sort of security patches, and that is run server. Directly from the docs, run server is not gonna through security audits, and that's how it's gonna stay. And that might not seem like a huge deal. But just as a small example, this is uh the headers from an HTTP request, right?

Speaker 1: Uh if you're familiar with this, this probably makes a lot of sense. If you're not familiar with this. Brief overview, you are getting the root page over HTTP 1. 1 on host catserve. io, which unfortunately we don't own, but you're welcome to create it and then put our code up on it, and that would be great. And this seems pretty straightforward, but what if you get a request like this? What if there is a layer of your system that interprets the first header, but not the second header? Or vice versa? Now, it could be that everything here is fine, but the point to make is that the Django team has not thought about this case for run server, and the USGI team has. And so security is one of those things that whenever any somebody says it, everyone in the room, including those people who work in security, feels slightly inadequate.

Speaker 1: It's more that you should be focusing your effort on using tools that have been vetted by people who care about the security of their layer. And the Django team has enough to worry about, and so run server has not gone through security audits and that's how it's gonna stay. Use a different application server. Again, pretty common, not necessarily specific to you, WISGI. Another thing that ha USG or other application servers have in its favor is config files. I showed this because the title of the talk is type USG press enter what happens. But many of you who use UWISGI on a daily regular basis are probably more used to seeing a format like this that you have in a UWSGI conf file that you can check into Git, you can pass around as text files, you don't have to remember the magical incantation.

Speaker 1: on your server. You can just point, you can just run UWISGI pointing it at this conf file, and it will do exactly what you want to do captured. And that's great because you get all the advantages of just having a text captain. configuration file that you can again check into Git, you can do code review on, you can uh iterate on over time, and you never have to worry about remembering the correct options. That is a super powerful thing that I think anybody here who's done sysadmining appreciates. Again, not necessarily specific to USGI. So now let's talk about some things that are super specific to UWISGI. UWISGI comes with so many features and so many modes of operation that you can probably replace large parts of your existing infrastructure with UWISG and you may not even know it.

Speaker 1: For example USG comes with a static file server. If you want your workers to die after a certain number of requests because you think you have a memory leak and you can't be bothered to fix the memory leak right now, you can set a max requests per worker on the worker and it'll just restart after a certain number of requests. Uazy comes with a queuing system. UWISG comes with HTTP support, HTTPS support, and HTTP2 support right out of the box. It comes with the Hari Kiri mode, so if a single worker grows to having too much memory, it'll just kill itself and start a new worker. Again, if you've got a memory leak Why debug it today when you can debug it six months from now and just have UWISG take care of it for you? If you want to prove how well UWISGI is doing, you can use USGTOP. If you want to see exactly where that memory league is coming from eventually, you can use memory port.

Speaker 1: UWISG supports async. And so when I look at this list, a thing I think to myself is, with all these things that UWISGI can do, Do I still need a separate queuing or a static file serving system? Do I still need Celery? Do I still need Redis? What else can you do? Uh so why you WISGY, code reloading and tunability are pretty common, uh, security config files, and just so many features. If you haven't dove into the UWSGI docs yet, highly recommended you will discover things that you did not know UWISGI could do. And so that's about it. Uh special thanks to Unbit, the guy who created UWISGY, who reviewed this talk

Speaker 1: for me, and my co-creator, the person who helped me write it. He was incredibly helpful at answering questions. If you have any questions, I will answer them now. We've got about five minutes. My name is Philip James. I am definitely so I uh do a whole bunch of things. I've worked in Django and Python for about 10 years. I am definitely available for consulting, especially on security or performance. So if your company is having problems that you think Could be solved by someone who's given talks on security or performance at DjangoCons. Definitely come talk to me. That's my email address. That is my Twitter handle. Feel free to tweet at me. It is my birthday, you're also welcome to buy me a drink at the bar later tonight. Uh and with that, I will turn it over to I think we have a couple minutes for questions.

Speaker 2: Thanks for the talk. Uh every single tutorial I've seen on like how to get Django up and running on the internet says you make uh you d use you WISGI and put it behind Nginx Um why is that, especially if UWISGI can do static file serving? That was, I thought, the whole point of using Nginx.

Speaker 1: That's a great question. The reality is that, well You would have to ask the specific tutorial author why they made that decision. as their kind of frontline server and so more people trust it in terms of being that like termination point and being the kind of the heavy hitter in terms of taking the most load. That being said, I actually do think for most small to medium applications you could totally 100% get away with just running UWISGI, but there are some things, especially around like certain special proxy configurations. that Nginx is going to handle better.

Speaker 1: And that was kind of our my point with the like list of features is if you hit a point if if you think all of your use cases can be covered by eWISGI. Use UWISG. A thing that USGI can't do quite as well is if you need to be um Actually that's not true. It's got plugins. I was gonna say there's some languages that USG doesn't handle quite as well, but it has plugins for most languages. Uh people trust Nginx more. think is a big part of that. But for small domino applications, I personally wouldn't have any fear about using UWISGI just out in the wild. And I think a lot of I think people do, they just don't talk about it as much. Hi Katie.

Speaker 3: Hi. Is Micro Whiskey the same as U Whiskey but ASCII?

Speaker 1: Uh micro has two syllables, you has one syllable. In a talk you're really stressed for time. And so I chose to cut about a hundred syllables-ish out of my talk by calling it UWSGI instead of micro whiskey. But thanks for the question.

Speaker 4: So you've talked about a lot of awesome things about UWISGI. What is the worst thing about it?

Speaker 1: There's so much there that trying to actually parse what the docs are telling you you whiskey can do is really tricky. The docs are definitely written from the perspective of someone who is a web engineering professional and could be far more beginner friendly especially for exposing some of these cooler advanced features.

Speaker 5: Happy birthday Philip. My question is um so in terms of the process that happens with the communication with a kernel where the socket is open and the port is exposed. How is it different from the run server? What does the run server do under the hood except for the fact that it doesn't can't have multiple workers

Speaker 1: So when I was creating this talk, we I was definitely under the impression that run server didn't have multiple workers, and then it turned out that run server sometime in when no one's looking added multiple workers So RunServer actually does do a lot of the same dance that UWISGI does, but there are things like in that list of features that I described that RunServer can't or won't do. And Realistically, um the like run server is a hundred percent fine for development, but I trust the team when they actively say please, please, please Please do not use this in production because they they're that's never going to be a priority for them. And I think that I've seen them like actively not like meanly, but like actively reject packages for making it more production ready because that's not the goal of Run Server

Speaker 1: But it's doing most of the same dance of like spawn some processes and then have them wait on a socket because it does do multiple processes now.

Speaker 6: All right, thank you.

Speaker 1: Thank you very much.