Using Django as a Micro-Framework: Hacking on the HTTP handlers.. by Carlton Gibson

My I'm slightly jet lagged. My body is telling me that it's 11 o'clock at night, which is when I go to bed. So slides. Yay! Hello. Right. Thank you, Katie, for the introduction. As she said, I'm Carlton Gibson. I'm your friendly Jenga fellow. I'm at Carlton Gibson on GitHub and Twitter. Follow me there. Um using Django as a micro framework. Um so what are we going to do? We are going to look at the core HTTP handlers of the framework. This is how at bottom Django handles web requests. So Django is a web framework. Handling HTTP requests is what it does if it doesn't do anything else.

So, this is the heart of the framework that we're going to look at. On the way through, we're going to look at the middleware Whatever they might be. We'll come back to that. And I'm gonna show you how you might configure Jang Django as a as a micro in a micro framework style. By that I mean in a kind of single file application Now I have the emphasis on the mites there, right? You are never going to do this. Well, you might. But I struggle to think of a case where the extra structure that a proper Django project gives you isn't of a benefit. However, the point isn't that you're going to start configuring Django in a micro framework style. The point is to get under the hood a little bit and to open the box.

People very rightly treat Django as a bit of a black box. That's it does its thing and we don't look inside and well, you know, that's the whole idea. But I think it leads to thoughts that perhaps Django is a little bit more complicated than it really is. I sometimes think see folks making aside as if they think Django is somehow a little bit clunky. Right. By looking inside, I want to show you that oh no, it's not that complicated, right? It's not clunky at all. And then finally, never to miss the opportunity to beat one's own drum, once you see that perhaps it's not that complicated after all, you might start to think that you could contribute to Django. Right? Perhaps you might contribute yourself.

That those 1200 open accepted tickets that Maris and I have our eye on, well, we might be able to get those down a little bit more. Okay? Right, if we get through all of that, I'm going to consider it a success. So what is a micro framework? Well to be honest, I have no idea at all. Maybe it's just something that you say to get on the front page of Hacker News. Thank you. One thought is that it might be Lego bricks that you put together yourself versus the batteries included that Django gives you that you get with Django. I don't know. Well, but one thing that I think there is to microframeworks is that they have really great hello worlds. Okay? So let's look at a few examples. Now, before I show them to you, the disclaimer. Everything I put up I'm putting up because I respect it deeply, not because I hate it.

Okay. Just because it's d just because it's not Django doesn't mean it it's not great. I'm I know you don't think that I think that, but it's tech, so I have to say Okay, here's the first one. This is um an Express app. Express is a Node. js web framework in JavaScript. Okay, so this is the Hello World off the front of the um Node. js website. And so first of all we require Express and then we define an app. We instantiate an app there, we define what port we're going to serve it on. Then we um add a handler to that ha handles get requests. to the homepage there. And then we have a function. And in my day that would have been an anomalous function, but now JavaScript had these has these nice cool arrow functions.

All that is is like a lambda. in Python. So in the on the left hand side of the little arrow there, it takes a request and a response. And all you do in the function body is write the request body that you want. That's your handler. And then at the bottom we conserve it. And we'd run that with node HelloExpress. So that's the first example. Let's do one from Python world. Here's Flask. Okay, we import some stuff at the top there from Flask. We instantiate the Flask app. Again, we root, we get a handle and we root it to the home page there. And Here we're just accessing something from the request to show you how you access the request in Flask because we're not in we're not injecting it as a parameter to the function, we're pulling it from the enclosing scope.

You run that with Rut Flask Run or with Gunnycorn or whatever. The final example is from Starlet, which is Tom Christie, who's the creator of Django Rest framework, his ASGI microframework. So it's an asynchronous microframework. Here again we just import some stuff at the top and then we create an app. The starlet app. Again, with the with the app in hand, we root a handler to the homepage there. And the handler, it's an async def function there, it's a coroutine. We pass in a request, and this time we're returning a JSON response. You know, we could return an HTTP response, but this is what's on the Starlet homepage. At the bottom there, we can s use Uvicorm, which is an ASGI

protocol server, which to run the application. Okay, let me just have a sip of water. I'm slightly worried that I might end up talking so quick too quick, but Russell's after me, so I don't think it's going to be a problem. Okay. Now these are all solving exactly the same problem. They're all turning requests into responses. You have a view which takes a request and returns a response. Okay This is the web problem. This is exactly what we have in Django. So here we have the minimal Django view. It's exactly the same. We We import the response object, we define a view, we pass in the request, that will be a Django request, and we return the HTTP response.

Okay. The difference with Django is that the other the other examples, they kind of wear the app bit on their sleeve. It's really easy to see where that app bit is. But with Django, we have to go and look for it. So here's our Hello Django project. We started this with Start Project, with Start app. We had added the app to install app, all the usual stuff, right? Now we don't usually go and look, but if we wanted to find the the app, we have to go and look in here, which is in the a a file called whiskey. py. Now given the talks this morning, you might be like Whizgy Schmizgy, we w we we're all ASGI now. Well we could do we could have an ASGI dot pi. It would be exactly the same. Most of Nothing's changed, right? As Andrew said, we it's bolted on the same. Everything I say applies just as much to ASGI as it does to WISGI, but we're going to focus on WISGI here.

So let's have a look inside the whiskey. py file. Okay, now this is the whole thing. How does that come out for size? Oh it's not too bad. Don't worry. Don't try and read it. But it is the whole file. WISGI config for Hello Django project exposes the Whiskey callable as a module level variable named application. The Whiskey callable Now, WISGI is the Python standard for um protocol servers to talk to applications. Okay, so if as long as you satisfy Whiskey, the whisk a Whiskey server like Garnicorn or or um You WISGI would be able to talk to your application. So the the Whiskey callable is a callable that takes an in

a Whiskey environment, environment, and a start response callable, and it returns a response Right now this is easier to see than it is to talk about, but just remember that and we'll come back to it in a minute. And the great thing with WISGI is that because it defines a contract, a WISGI server can talk to your Python app if you satisfy it. So let's so if we go back to whiskey. py, this was the file that this is our this is where Django keeps our app, the key bit 's right at the bottom. And here it is simplified so you can see it. Let me have another sip of water. Right, the key bit is this get Whiskey application. um function. All it does, all this whiskey. py file does is to assign

the application object this get whiskey application function. So let's dig down there. We have to dig a few layers, but we'll come back up. But don't worry. So this getWISGI application function is just a two-line wrapper. It calls Django. setup and then it returns a thing called whiz uh an instance of a thing called Whiskey handler I don't want to talk about Django setup. If you've ever wanted to use the Django ORM in a script outside of the normal Django project, then you've probably used it. It configures the settings and the logging and it sets up the ORM If you don't call it, you have to deconfigure any settings that you're going to access yourself, and you can't use the ORM. Okay? But if we look into Whiskey Handler

, No , that's a bit that don't try and read that either. But I'm putting it up because that's the whole class. We'll dig down into the in in the bits of the. It's 23 lines. And the point of showing you the 23 lines is that there's not much there. I don't want you to think it's super complicated. Just the only thing to note here is that Whiskey handler is a subclass of base handler. Just remember that. Then let's put up the call method there and with just the main bit showing. I'll simplify it very slightly. Okay? Now this is exactly the whiskey callable that we were looking for. Okay, it takes an environment whiskey environment and a start response callable and it returns a response. Okay The first thing it does there is it it converts the WISGI environment into a Django

requ request object. Okay, the request object that you see in your views. The ones that you're used to with all the, you know, request. get, request meta, request or request meta, request. get, request dot post, all of those things. Okay. Where the comment is, well, that's it's doing some other whiskey stuff that I don't really want to talk about. It's just a few lines and um We we call the start response callback with the response status code, so 200 if it was okay, 400 if you made an error, 500 if we made an error. and the headers from the request and and we call that in where the comment is and then we return the response. Okay So that's three of the four lines and that must mean that all the magic is in this get response line

where it says response is equal to self get response Where we pass in the request. So we are going to dig down into that get response method and we're going to see what goes on. But in order to do that, we need to go for a little tour via a thing called middleware. So what on earth are middleware? A middleware is a callable that takes a request and returns a response. Just like a view. Okay? That's what our view does. It takes a request and returns a response. That's the web problem. And the idea is that because they have the same interface, middlewares can wrap views. We're passing a request to our view. We want a response back.

But equally, if there's a middleware wrapped around our through a view with the same interface, we could pass we could just as well pass our request to that and wait for the response to come back from that. It's a bit like a decorator. You know? We still pass in a response, we still want back a re uh we still pass in a respon a request, we still want back a response. Okay? So let's look at a very simple example. Simple middleware. This is from the Django docs. Okay. We initialize our middleware with whatever view or middleware we're wrapping, and then it's a callable which does what it wants Call uh like whatever ac action it wants to take, then it calls whatever it was wrapped that it wrapped to get the response back, and then it can do a bit more if it likes, and then it can return the response.

Now that's the structure of a middleware. Okay? There are some other hooks which I don't want to talk about. You can add extra methods to middlewares that let them do other things. Process view, process exception, process template response. Process exception is kind of op kind of obvious. And that's if you don't want Django's default exception handling to play. So you can intercept Django's default exception handling and return a custom exception. Django has this thing called template response. So if if your view returns one of those, then process template response lets you process the template response. Um you can change the template, say, or you could adjust the context or whatever before it's finally rendered.

Process view, well, that's a little bit more A little bit more weird. I guess the best way I can think to describe it is it's like a middleware saying, hey, I'm gonna defer to any other middleware and give them the option to return a response. But if none of those other middleware return a response, then I want another go at this just before the view is executed. Okay, that's the best. Uh it's not perfect, but it'll do. Anyway, I don't want to talk about any of these. Let's go back to our simple middleware. Here's the init method. Okay? It just takes in a get response thing. Now that might that's that could be a view or a middleware, except it can't be a view. It can't really be a view. Why not? Well, it would work for our Django view.

This is exactly what we need. A single view that takes a request and returns a response. But normally we don't just have a single view. Normally we have multiple views. Right? So here we've got two views. We've got hello, which returns the response hello, and goodbye, which returns the response goodbye. And we route those to different URLs. So we need to know which view. To pass the request to. And then in Django, views don't normally just take a request. Okay, whilst in the simplest case we can think of a Django view as just taking a request. More generally, one takes a request plus a possible number of positional arguments plus a possible number of keyword arguments. So here

we have a halo view which takes a name and we route that using this path routing Path URL pattern where we have hello plus the name parameter, and so when the when that is called, it needs not just the request, but the the name keyword argument as well. So for these two problems, um Django provides the base handler. Now you remember that that whiskey handler, our application, our whiskey application was a subclass of base handle. Okay? And base handler has this private get response method. It's underscore get

response method. So WISGI handler has it too because it's a sub class. Now I'm not going to show you the source code for get response, but it's 62 lines including the doc string. Okay. And that what it does is resolve and call the view, then apply the view. Those those exceptions The process exception, process view, process template, response hooks, it applies those and then it returns the response. And this the key bit there is this method is everything that happens inside the request response. Middleware. It's the bottom of the framework. This is the bit where Django takes the request, examines it for the keyword arguments that you need, works out which view it wants to call, and calls it. Okay? So it if that's the bottom, we can start heading back up.

When we instantiate our simple middleware, we don't pass a view, we pass it Base handlers underscore get response method. Or we pass it another middleware, because middleware can be nested. This is the default middleware from our project, the middleware setting. And you've all seen this, right? In your projects. You've all added a middleware to this, probably at the top or at the bottom, or very specifically after a certain named middleware. They need to be in the right order because they're called from top to bottom. So for instance, authentication middleware, which is Somewhere in there, can't even see it.

Too too up from the bottom. That's the middleware that sets the request. user property on your request. So when you're in your view and you go request. user That only works because request. middleware was in play. But request. middleware requires the session to be in place. So it has to come after session. middleware in the list. So this is kind of This is the middleware onion, right? Each middleware gets the request. It can do its thing. It can return a response if it likes. Or it can call the next middleware down. And when that's done its thing, it will get the response back. And then it can do something else. And it can return the response. It can either return the response that it got, or it can go, no, I don't like that response.

Here's a new one. Okay? Right, so you'll remember we had a whiskey. py file that created our WSGI application by calling getWISGI application. Okay. Get def yeah. Get WISGI application just calls Django set dot setup and then instantiates a Whiskey handler which is a Which is our WSGI callable callable, which is our application. Okay? And our when we instantiate our whiskey handler before our whiskey server calls it, And before we get we can call get response, right? We have this load middleware call.

And all load middleware does is it goes through the settings, Anya. And that that's at the very heart at the very heart of which is the the underscore get response handler. And it instantiates those Into a chain, into a kind of nested object. Get response plus the first middleware, plus the second middleware, plus the third middleware until it's done them all. And then that becomes the ultimate handler Which is called when we dispatch it. Okay? So with the middleware chain um chain loaded, we can look at get response We had our WISGI callable, remember, that maps the whiz the WISGI environment to a Django request and then calls get response. It does some WISGI other stu other WISGI stuff and then it returns the response. So finally we look into

get response. Here again we take the request object We get the response from the middleware chain. We pass in the request. The middleware chain is that nested structure of middleware with the get with the underscore get response callable at the middle, middle of it That gets the response out. Response closable objects append. Well, we add in the request. So part of the WISGI um contract is that the WISGI server will call close on your response object if it's got a close method. And that is particularly relevant for things like file responses, where you want to, when the WISGI server's finished with it, it wants to close, it wants to clean up. So we just add the request there so that

the request can be cleaned up if there's anything going on. Going on there. Then if it's a 400 error, or more than a 400, so if it's a 400 or a 500, we'll log the response and then we'll return it. And that's it. Okay? Base handler, I can't show you the whole class, but these are the three methods that we've talked about. Load middleware that populates the middleware chain, so sets up the application Get res underscore get response, which is this wrapper around the Django views, which knows how to root a URL to a particular view function and to get the keyword arguments. And then get response, which is what we actually call per request. Um to do it. Now there are a couple of wiggles that I haven't talked about.

There's atomic requests where you can wrap in the entire request response cycle in a DB transaction. Okay There's exception handling which I haven't talked about. What happens if your view or any one of your middlewares raises an exception? Well you need to catch that and you know do a nice error page or whatever. Right, but those extra bits, they don't take up any space. The whole of base handler is just is less than 150 lines long. Right? So they can't, there isn't much room for the stuff I haven't shown you the the stuff I haven't talked about right and so to business how do you fit a Django applied application onto a single sharp slide

Like this. This is our Django micro example. Here we just import some stuff at the top and then we call settings configure. And we just call that with the one setting that you really can't get rid of. And well, you could, you'd have, but you'd have to do subclassing and custom code and all the rest of it. You have to tell the settings where the root URL conf is. Okay, so in this case it's in this set very same file because we've only got one file. Then there's our few function. This is the same Hello Well view function that we used to. And then we just route that view function to the home page, like the same as all all the rest. And then We instantiate our application, which is the Whiskey callable, which is the WSGI handler, the Whiskey handler.

And that's it. That's how you fit Django onto a singles Right. Right. There the base again There's there's nothing to it, right? We take in a request, we pass it to a view that's wrapped in an onion of middleware. That's it. There's virtually nothing to it Importantly, it's the same virtually nothing to it that all the frameworks out there are doing, and they're doing it in more or less the same way because we're all handling the same problem, the web problem. Okay, we're turning requests into responses. The base handler is 150 lines long. The whiskey handler is another 20. I put it up. In its entirety earlier on, right? There's really not much to it.

There's no room for it to be clunky. Okay? So that's the the kind of essence of it. What to remember? You probably don't want to put Django in a single file. Nobody uses the Express hello world. There's a Express Starter command line tool which gives you a full project with places for your static files and all the rest. Everyone uses that. Same as start project in Django. You want the extra structure that it gives you. Okay? Don't, you know, unless you're having fun, do what you like then. You do want to use and love middleware, right? They're not that scary.

They're just a callable that has an opportunity to do some stuff before the next one down and it has an opportunity to do some stuff after the next one down and you can use them to do all sorts of wonderful things. CSFR CSRF protection. the security checks for headers that Django provides, sessions, authentication, all of these are out of the box. They're middleware. Right? Middleware is one of Django's kind of secret sources. It's it's give it's where mo a lot of the power comes from. Okay, there's the ORM, but that's the same thing. Don't make a middleware that um makes 200 database queries on every request. But investigate them, play with them, and win. Okay, and then hopefully

you've seen the core of the framework, right? You've seen it's not that complex. So maybe you'll come and play and start contributing too. Okay. I'm Carlton Gibson. I'm your friendly Django fellow. At Carlton Gibson on GitHub and Twitter. I've got a podcast, Django chat. Every week listen to that. I'm here for the rest of the conference and I'm here for the sprints on Thursdays and Friday. Please come and talk to me. I'd love to meet you. I'd love to come and say hi. All right. Thank you.