Taking Channels Async

Speaker 1: Next we have Andrew Godwin who is here presenting Taking It Channels Async. Andrew was a member of the Django Core team and has been working with Django since two thousand seven on projects such as South, migrations and, of course, channels He works at Eventbrite as a senior software engineer. He will not be taking questions, so if you have a question that you have for Andrew, please take that once his talk is over.

Speaker 2: Thank um thank you, Lacey. So yes, I am here to talk to you about channels. Uh and don't worry, this is not the same as uh yesterday's talk by Jacobo. Um this is a different kind of talk about some of the history of channels, some of the reasoning behind the virgin change, and some of the things it might mean for Django in the future. But first, a little bit about myself. Um oh, and once again this is not working. There we go. So as uh Lacey said, I'm a Django core team member, I work at Eventbrite, and I have this bad habit of Doing things like network programming for fun. Now, it's not a particularly fun activity. I don't recommend it as a side hobby, but it's a thing I do nonetheless. And this started in 2015 on a project that back then was called Django on Air. This was the very first code name for what channels became.

Speaker 2: I was sitting down thinking, I should make a thing that isn't migrations. It had been quite a few years at this point. And I wanted to look at WebSockets and real-time stuff. Eventually that became The thing we know today are called channels. And over the time, channels 0. 1, which was released in 2015, became channels 1. 0, released in 2017. This first version of channels had quite a few things going for it, and there were a few particular design reasons in its development that ended up making it not so great. The key problem it had was because of the time of its inception, remember this 2015, Python 3 is not such a big thing, I wanted to make it Python 2. 7 compatible. This of course means you can't use things like async IO, you can't use things that are in modern Python, you're restricted to things from quite a few years ago.

Speaker 2: I ended up using Twisted for the web server, because Twisted is way well understood, it's very well known, and running Django synchronously. If you want an idea of how this looks, this is a rough architectural diagram of how channels used to run in version one. You had two different processes. a web server, which was called Daphne and runs in Twisted, then a Redis middle layer, which transported messages to and from the web server to separate Django worker processes. If you used channels 1. 0, you would have been familiar with having to run a server and then separately run workers. And if the worker wasn't running, there were some weird error cases, it didn't work super well. And in general, it just had too many moving pieces. One of the problems with the design is it was really hobbled,

Speaker 2: sort of undermined by the fact it had to support Python 2. This of course not only is having too many moving pieces meant no support for async. io. And ultimately the end result of this design and all the moving pieces and the complexity meant it was very easy to shoot yourself in the foot. Um I my personal belief of Django is it's designed to give you a safe environment to experiment in and understand what you're doing. And ultimately channels one didn't really have that. It was experimental but not perfect. And ultimately I think the design of what I did was wrong. This is something you have to admit on stage like this. And so it has to sort of sit down last year and and look at everything again. And in the light of twenty

Speaker 2: seventeen Python 3 being a supported thing, Django 2 at that point had been released and only supported Python 3. I re-examined what it meant for channels to be. And then this year I released channels 2. 0. Channels 2 is a pretty substantial rewrite of the channels 1 codebase. The most important thing is it is async. io native. It has that built-in support for running async code using the things that Python 3 gives you. Things like async def and proper awaits. It also supports both synchronous threads and asynchronous coroutines. And even better, because of all this built-in support in Python, because we can do things in the same process it meant deploying it became a lot easier. Rather than having those two separate processes, you now just have, like you do with WSGI, a single web

Speaker 2: server process inside of which lives Django. And so that's sort of the the overview of the rewrite. Um it was about 75 % of a rewrite, I would say. It wasn't particularly Rip everything out and start again, but over the course of months I took every file, examined it, kept what I could, and threw away what I couldn't And a lot of it had to be thrown away for one key reason, and that is that I had to start making Django partially async. Now if you're not familiar with Python's async support, one of the main problems you have is that asynchronous functions and synchronous functions have a different calling interface. You can't mix and match them. It's very hard to make a library that supports both of them. And of course Django

Speaker 2: has been around for quite a long time. It has a lot of wonderful synchronous parts. Here is a very basic diagram of some parts of Django in a layer stack. It's not quite as simple as that, but you get the idea. And all of this is synchronous And given that I wanted to make things asynchronous, I had to sit there and work out what to rewrite. In channels one, all this was running in a synchronous worker process. I could keep it, it kept working as it was In channels two, because it's shared in the same process underneath an asynchronous event loop, you have to make sure that things yield to each other. It's called cooperative multitasking. You have to be cooperative. And so one of the things Channels 2 does is introduce a second

Speaker 2: almost parallel kind of stack for doing a lot of this stuff. You can see that everything in that box on the left-hand side there, that remains synchronous, even under channels. Things like the ORM and the old Django views. They're kept as is for backwards compatibility reasons pretty much on top of that. The other things, the ASCII routing, the ASGI middleware, I'll explain what ASGI means later. Those all exist in a pure async world. And for that reason, they're basically ground-up rewrites of a lot of Django. Things like the author middleware in uh channels, for example, it does give you a user object like the Django author middleware does, but a lot of the s inside of it is written in an entirely different way

Speaker 2: to try and be asynchronous. And this is one of the things. Like I had to try and make Django async native most of the way through its stack. Again, it's not all the way through, and I'll talk about a little bit why and plans for that near the end of the talk. But the first thing I want to look at is how I did that. And one of the things you may have seen in the talk yesterday about channels was these two functions, sync to async and async to sync. These are the two functions took me about two months to write and perfect. Uh they are incredibly tricky and the name as it suggests tells you that one of them turns synchronous functions into awaitable asynchronous functions, the other one turns awaitable

Speaker 2: asynchronous functions into synchronous functions. And each of them have their role to play in the process. Let's start by looking at sync async. This is the easier one of the two. So why do we need this? Well, as you've seen, Django has a lot of synchronous code. The RRM is a very good example. I'm not the sort of person who can sit down and rewrite the RRM in a year. That's not a project I can do. And so we have to run that kind of code in threads. If you run synchronous code on the main event loop, because it's cooperative multitasking, you're going to sit there and block the event loop all the time the synchronous code is running. It's not an error, which is unfortunate. It just sort of sits there and slows down. Actually really hard to work out what's happening

Speaker 2: And so you have to take all your synchronous code, put it off the event loop and put it to the side into a thread and let it sit there in the thread idling and doing its stuff. Python has pretty reasonable support for this. It gets better in version 3. 7, but in version 3. 5 it looks a little bit like this. You get your event loop, which is the top line there. You make a feature to run in the executor. There's a thing called a thread pool executor in Python, which, as the name suggests, runs things in threads. and then use literature await the future. When you do the await call, the event loop pauses that coroutine, starts the thread up, and then when the thread finishes, it notices and resumes your coroutine where it left off.

Speaker 2: As I said, this is good for not just calling the RM, but things like rendering templates. Django's template system can do blocking requests. You can call the RM inside it, you can call other parts of Django inside it. And also handing off to Django views. Channels is designed to live alongside your existing Django view code and your existing middleware. And so we don't want to disrupt that. We want to keep that view code in a happy synchronous world where you don't have to sit there and rewrite it all to be async to even get started with channels. Let's turn to the more difficult task, which is async to sync Now, as I said before, async code runs on an event loop. And generally in Python, your main thread, the thread that your program starts in, is where you run your event loop.

Speaker 2: And often what you'll do is you'll be in a synchronous thread and you'll want to call an asynchronous piece of code. You may have seen in yesterday's presentation about channels, all the different methods on channel layers are all asynchronous. You can't send to a group without it being an asynchronous call. And so if you're in a random managed. py script, if you're in the RM, if you're anywhere in Django that's sort of old synchronous code, you want to call this async code, but from a synchronous context. Now if you remember the previous code was pretty easyable, easy to read, well formatted. Uh this is the short version uh of async to sync. As you can see it's not particularly readable. Uh you're not meant to be able to see it from there. But the essence what it does

Speaker 2: is it makes a thing called a future. A future is a thing in in Python and many other languages that says, hey, I'm going to go and do a thing and when I finish, I'm going to give you the result. So it makes a future. And then it has to jump from the thread it's in to the main thread because Python async is not thread safe in a wonderful mind-bending way. And then once it's on the main thread, has to then go and find the main event loop, get the make a coroutine, get the future from the other thread it just left. tie that thread into the other coroutine and then go when you finish unblock this thread and sort of put its hands up like this. This is where most of the pain came from during development. It is tricky to do. It's still not perfect,

Speaker 2: but it's a long way from where it was. And the key thing this does for all the difficulty in development is it lets us provide async native APIs. I can write things like the channel layers once and then let Django code use them wherever it is. This solves one of the problems I had with channels one and the reason I didn't do async there, I didn't want to write everything twice. I didn't want to have to have the channel layer synchronous and the channel layer asynchronous. Even for things like middleware, I didn't want to have to have the synchronous user middleware and the asynchronous user middleware for four channels as well as the Django one as well. And this is one of the things where channels philosophy in general is that both synchronous and asynchronous code are useful.

Speaker 2: My goal is not to make everyone write async code. It is more difficult, more complicated, and harder to debug. My goal is to let you write it when you want to, and then let you use synchronous code the rest of the time. And so it's very important to have that cross-compatibility. where you can can call the other world, the synchronous world, and vice versa, whenever you want to And not just let to let you use Django's code and channel's code wherever you want, but to have your own code written that way. If you want to write a reusable application that uses async code With these kind of functions, you can ship it in the knowledge that people can call it from anywhere. They're not going to be locked out of it because they're on synchronous Kajango.

Speaker 2: This is an unfortunate part of Python's design for async. There's many, many articles online about the trade-offs that Python made. They were made many years ago as well. But the key thing is Generally, you'll find async APIs and libraries are entirely separate to synchronous ones. For example, the Redis library that we use for the channel layer in channels 2 is AIO Redis. It's an async IO only version of the library. You can't call it from a synchronous context easily. If I was going to ship a version that supported both They have to import two different Redis libraries, write two separate code paths, one for synchronous, one for asynchronous, and somehow manage the state so that If you listened on a synchronous thread and then waited on another asynchronous thread that they talk to each other properly.

Speaker 2: This is why we have as compatibility layers If you want to see more about this particular problem, about separate interfaces and how to understand them, I have a blog post on the topic you can go and read It's not particularly long, but it covers a lot more than I can in this talk about the different worlds and how you want to manage and think about taking your code and running it in both an asynchronous context and in a synchronous context. But let's go back to that Django diagram. And there is one key feature on this diagram that may be a little bit small and has eluded you. On the bottom right-hand corner, uh right here in fact, there is a small arrow. That arrow on both the small version and the big version of the diagram

Speaker 2: is representing the incoming request from outside. Now in the nice old Python we all know and love, that is WSGI. It's A very well understood standard. It's been around for many, many years, and pretty much every single framework, application, server in the Python ecosystem supports it But of course it has one problem. It's synchronous. And this is not a thing where I can say, oh, we're just going to wrap channels in async to sync and put it underneath WSGI Generally when you do asynchronous stuff, you have to make it asynchronous from the outside in to make things efficient.

Speaker 2: So you have to make the routing and the handling and the middleware asynchronous, but maybe the ORM is still synchronous. If you don't do it that way, you end up having to block a whole Python thread. or a whole Python process just waiting for one asynchronous program to run. And so the problem we really faced is how do we make WSGI async? Now the easy answer is you put an A in it. And of course that makes it async. And this is not quite what I did, but it is the essence of what happened. Over the past few years I have been trying to develop a standard called ASGI, which is like WSGI but for asynchronous situations. And one of the key things with WSGI

Speaker 2: is that it is simplistic. It has a design that's easy to understand. You make a function, the function gets given two arguments, an environment, which is Basically, here's the request in the path and stuff, and a callable that says call this when you want to start your response. So you take the envron, you send the headers with the second callable, and then you just return data. It's simple, it's easy to understand. There was an ASGI 1 to match with channels 1. It was, shall we say, overcomplicated by half. And so as part of the channels 2 redesign, I sat down and looked at ASGI as well, with the goal of how do we make a standard that not just Django can use, but the wider Python world can use as well.

Speaker 2: This is the essence of what I ended up with. An ASGI application is a little bit more complex than a WSGI one, but not by much. So you have basically A thing that you call, usually a class, with a scope. The scope is like the M for. It has the path, it has the method in HTTP. It tells you all the information about the connection. And then when you've got your instance of the application, it then calls the coroutine section, the async def in this example. And then sets it off as hey, here's the thing to receive data, here's the thing to send data, do what you want. One of the nice things about this is because it just gives you a coroutine. You don't have to listen straight away. What channels actually does is before it listens on the socket for incoming requests, it first

Speaker 2: spins up the Redis channel layer and makes a name for itself and listens on its own socket too And so what channels consumers do is listen on both the receive callable from the server and also on Redis, which means that no matter what you're sending and where the events are coming from you're still gonna have that same coroutine and that same instance handling them. This is one of the things you heard Jacobo talk about yesterday, where you can store things on self because it's a single instance. This one class is sitting there in a tight loop working out what's happening. The other thing with ASGI that actually came from the first DjangoCon Europe I ever attended, back in Prague in 2009, is a concept called Turtles All the Way Down. This is something I I think I can credit to Simon Willison

Speaker 2: or maybe Adrian, I forget who it was. But the idea is that in Django as it stands now The middleware and the views and the URL routing all have different interfaces. They're all separate things. You can't layer them in different ways. You can't have URL routing depend on something from m a middleware, for example. And I didn't really want this. One of the things I've wanted in Django for a long time is to have those pieces have a very similar interface. And again, one of the problems with WSGI was it didn't really give you that flexibility Whereas what we can do in channels is we can say, oh no, we have more flexibility. In channels you have a URL router is just an ASGI application that takes other applications.

Speaker 2: Everything from the middleware to the consumers to the routing, it's all just an ASGI application. That doesn't just mean it's swappable in Django, it means that you can put other non-Django things above and below it. One of the things I'm trying to encourage that we missed out in WSGI because of its slightly late adoption is the idea that you can swap pieces around of frameworks. What if I want a bit of Flask in my Django app? In channels, if and when we get Flask to have support for this, then you can do that kind of stuff. And then the question really becomes, well, what does this mean for Django? I have, you know, stood here and talked about adding things to channels and how you make Django more async. But all of this is very much external.

Speaker 2: Channels is a Django project, but it is still not in the Django core repository. And so the question really becomes, well, will it become that? Should it become that? And this really comes down to one question, which is how much can we make async? As I said earlier, the whole idea is that you want to make things asynchronous from the outside in. You want to progressively turn more and more parts of Django into an asynchronous capable part of the system And because we have those functions to let us go between asynchronous and synchronous worlds, what we can do is we can in theory slowly replace Django piece by piece. progressively making more and more of it asynchronous, but still keeping full backwards compatibility, still keeping the old views and everything that we still use today.

Speaker 2: And there's one particular problem with this vision, and that is the RRM. Now, as you heard from Casey, the ORM is a very complex beast. It's many different parts inside it. I've been doing Django for over a decade. I still don't understand all of it. And the idea of making asynchronous initially seemed very difficult. A lot of it is built around these synchronous database bindings and synchronous ways of writing queries. And At this similar talk at PyCon a few weeks ago, I was pretty much sure that I couldn't do this. But talking to people since then, having the idea I'm now pretty sure we could do this. There is again a way we can slowly make things asynchronous.

Speaker 2: The key thing here with a project as big as Django is that we can't do it all at once. There's no way we can take Django, flip everything asynchronous and ship it in one go. Not only is that a huge amount of code and maintenance work, it's going to result in a huge number of bugs. And so any plan we have to have has to be iterative, has to go in small steps. Let us release it over the course of many releases and have people feedback how it works. And with the RRM in particular, I am now convinced we can slowly take the RRM and make it more and more asynchronous. This is still a thing I need to write up and present to the rest of the core team and the Django developer community at large. But this is one of the things and the final pieces for saying maybe we can start making Django itself asynchronous.

Speaker 2: Maybe you can bring parts of channels back into the core if it makes sense. And then this gets to a bigger picture as well. There's really a question here of, you know, I'm trying to replace WSGI with an ASGI standard. At PyCon we had a meeting of different a few different frameworks and servers, generally agreed that there was a need for this and we should do it. But there still is a question here of like You don't just replace a standard. There's the old joke that you go, oh well we have five standards and none of the standards are any good. So we're going to invent a new standard. And Narrator, there are now six standards. Like that's the problem we face here. Fragmentation is terrible. One of the things I really had to focus on with ASGI is making sure that it is backwards compatible.

Speaker 2: There is a thing that we actually ship with channels that lets you run WSGI applications as ASGI applications. And that's part of it, but the other part of it is is there a need? We can sure we can do the technical work, but that's only part of the equation. Part of running a framework is making sure that you or you're using That your users want your changes to. Migrations, when I worked in it, is a feature pretty much everyone needed, even if they didn't know they had to have it. WebSockets is much less of that thing. WebSockets are niche. They're not a super important part of many people's web development process. Long polling is a bit more important, but it's still much more niche. And so there's a real question here, do we

Speaker 2: need to replace it? And one of the things that first appeared to people was like, well, you should Obviously take it and write a PEP and go straight to the Python development mailing list and then have them approve it with a big rubber stamp. That's not really how things work. One of the things in the history of WSGI is it came about after the fact. When I started doing Django, WSGI wasn't a standard. It came about in those first few years because of the arising need. and the work between things at the time like cherry pie and Django and Turbo Gears, they were all trying to work on a similar solution. One of the things I think is to have a standard working, you have to have multiple servers implementing it and multiple frameworks implementing it as well.

Speaker 2: Now I'm very grateful that um not just Daphne, which is the channel sort of built-in shipped with server, but also Django REST Frameworks uh Uvicorn also now has full ASGI support. So that kind of makes me much happier on the server front. There's also apparently we work in Twisted, add twisted support for this natively. This then leaves having multiple frameworks. Django, sure. Django is a big framework. We are not the only framework in Python. We have to be friendly and play with everyone else. And so talking to people like Pyramid and Flask and other frameworks to try and make this important to try and make this a agreed upon is important as well. And this is ongoing and a slow process, but in general there's pretty good buy-in. People seem to

Speaker 2: These days believe there's a need for having not just WebSockets, but like I want to run HTTP things that last a long time, like slow requests or long polls, or any other number of things that aren't WebSockets or HTTP. And then you get the final real question here, which is, well, sure, we can have Django bit async, we can have an async version of WSGI, we can have the RMB async. Do we want that? Asynchronous code is difficult. It is, and trust me, I've been doing it for three or four years now, a massive pain to debug. There are so many more error cases and edge cases to handle. You can make things that silently slow things down but don't fail.

Speaker 2: There's a magical environment variable in Python you can turn called Python async. Oh, this coroutine is too slow, or you didn't await properly here. But the end The end result is it's still difficult. You can still miss an await, and your code apparently works perfectly, but in reality is only working in a single coroutine at once. And so this is why the channel's philosophy is to let you write both. Both synchronous and asynchronous are important, and any plan I think we should outline for the future should have both. I want to keep the ability to have that nice, easy, synchronous interface for most of the work you do. And then when you need it, this is Django's strength. When you need to go deeper, when you need more complexity, Django

Speaker 2: can then open up and let you into that deeper section and say, okay, you asked for it. There's now like you have much more control over the request. You have asynchronous code now, and it's there. And ultimately the big question here is, you know, what is Django? Channels is a project done by a much smaller number of people than core Django. It's me and a few other contributors. And Comparing it to Django is sometimes difficult. Like, should Django be doing this? What is Django's goal? Is there a like we don't have a mission statement or things like corporate things like that? And ultimately the question is like, should we be taking Django and adding this stuff into it? Should we even be preparing Django for this new asynchronous wave? Yeah, it's a lot of work. We're gonna have to s

Speaker 2: if we're gonna make the ORM asynchronous, sit down, do a lot of nasty testing and rewriting and thread pooling and all the things like that. I personally believe that's the case, but my word is not the word of the Django core team, and I think it's one of the discussions to have. Ultimately, Django is not just us, but all of you as well. And so I encourage you to, you know, come and talk to me and other core team members, come and talk at the sprints about this stuff. And like I really want to know about what you think Django is. where we should be going and if these kind of changes are important to you. And with that, thank you very much.