Django Through the Years with Andrew Godwin
Published November 3, 2022
This video features Andrew Godwin at DjangoCon Europe 2020 in Online.
DjangoCon Europe 2020 (Virtual)
September 19, 2020 - 17h10 (GMT+1)
"How To Break Django: With Async" by Andrew Godwin
Curious how to put your code into a deadlock? Want infinite loops, but in parallel? Or just want a new, exciting kind of data corruption? We'll look at these and other ways that asynchronous code can make you write some spectacularly nasty code, as well as the ways Django tries to save you from these terrible fates.
Note: Q&A not available due to technical problems.
Django’s async views can run alongside synchronous views under WSGI and ASGI, but asynchronous programming introduces concurrency hazards that ordinary synchronous code avoids. Andrew Godwin shows how missing `await`s, synchronous database calls, unordered side effects, race conditions, incorrect sleeps, thread-bound connections, and misplaced transactions can cause silent slowdowns, data corruption, deadlocks, or operations that never run. He explains Django and Python’s guardrails, including synchronous-operation errors, async debug mode, coroutine warnings, and safer thread-sensitive adapters, then recommends writing and testing code synchronously first and converting only measured bottlenecks to async.
Summarised automatically from the transcript.
Automatically transcribed, so expect mistakes in names and technical terms.
Hello everyone, and I hope you're having a good Django Con so far. I am very excited to be here to talk to you about the wonderful myriad number of ways you can break async with Django. So let's get going. First of all, let's talk a bit about me. I'm Andrew, if you've not met or seen me before. I am a Reasonably long time Django developer. I've been working on Django since about 2008-ish. I've worked on things like South and Django migrations and now the async stuff. I've been around a lot of places, I've been doing Django for quite a while, and I still love this community and everything it brings. So thank you all for being who you are. And crucially, uh, I am unfortunately not in Europe. Um, I'm doing this remotely uh from the wonderful world of Denver, Colorado.
Um it is wonderfully beautiful here. Um we've been blessed with a pretty nice uh set of days here in autumn. It snowed very recently, which is a odd thing in September, that's the way things go. But yeah, I am sad not to be there with all of you in person, but hopefully we will all meet again in person very soon, if not next year But with that done, let's talk about the good news. And the good news is async views are in Django right now. You can go use them, you can just put async death with your views, it will all intermingle and work perfectly. I'm not going to go into too much detail here. You can see more about this in my other talks I've given in various places. But the nice thing is you can just write views that are async def alongside normal def views that are synchronous. Thanks to some wonderful uh interior rewriting of Django's request flow.
Uh both of those are servers in both modes If you're running in WSGI mode, then it will just work and serve synchronous views like normal with no performance impact and it will serve asynchronous views in their own like async mini event loop It's not as good for performance. You can't do long polling and stuff. But you still can still do parallelization and things like that. And if you're in uh WS, sorry, if you're in ASGI mode. Then of course you can run both synchronous and asynchronous in that mode in the same way and the synchronous ones wrapped in their own thread. So it's there It's good to use. Please go use it. I encourage you to have a go with it. And then as you'll see in the rest of the talk, I encourage you to go about it in a certain way. So let's talk about that. So first of all, I want to be upfront here. Synchronous programming is really quite safe, as safe as programming can be
anyway, right? But like it's all in order, you can understand how instructions execute one after the other. It's a mental model we're all familiar with, at least if you started out doing procedural programming. If you did functional, I mean, A, thank you, and also I apologize. And B, I hope you still know how procedural stuff works. I can't do monads very well. But the point aside, like we all understand how synchronous programming works. Asynchronous programming, on the other hand, is difficult. Now it's not impossible, certainly. But there's a lot more sort of subtleties to it. And this is true not just of asynchronous programming, but all concurrent programming in general. Be that asynchronous models like we have in Python, be that threading, be it multiple processes, be it across a network. These are all concurrency problems and you'll see versions of all the same problems in every single one of those kind of system designs, but it is harder than writing code all by itself.
If you've ever worked on say like microservices or distributed systems, you know this all too well. And in many ways, asynchronous programming is inviting those same problems into your code base where normally things were safer And so I want to kind of talk about the kind of things you can see when writing async code and certainly the things that I have run across personally. Now I learn best by example, so these are all a set of examples of Mistakes I have made at one point or another throughout the years. I do encourage you to try these at home, but do not try them in production. You want to try them away from having actual users where you can experiment and learn from them. That's my recommendation. So with that done, let's go and look at what we can do in Django with async. And I went, you know, I led off here with like great async views are in, we can put async def in, fantastic.
There we go, there's an async view. I've taken an existing view I had, I put async in front of it. Let's go, let's hit run. This isn't going to work. Now, there's a couple of reasons it's not going to work The first one is, and the main problem is it actually would have worked in some ways. If Django hadn't stopped you doing this, this is perfectly valid Python code. It will run and compile and execute. And what it will do is it will start an asynchronous view in its own coroutine. It will then get to this . get line here where the book is where they're trying to fetch the book. And then it will synchronously go talk to the database and block the whole append while it's doing so for maybe up to a couple of seconds and then come back and return. Now, the way asynchronous programming works in Python is it's called what's called cooperative multitasking.
You have to cooperate and tell Python, hey, here is a point where you can pause my execution and go and run other things. It's still single-threaded on single core. The GIL is still there So if you don't have an await, that kind of like co-operative thing in your code, it can't return. So this would actually just run and it would run and block the event loop for 50, 100, 200, maybe 500 milliseconds. And that's really bad because you would take this code, you'd run it, it would pass unit tests, it would work in your browser, you'd push it to production, and it would work. But it wouldn't be very efficient. In fact, it's probably less efficient than the old synchronous code. because you're blocking the event loop, it isn't like that, it's not designed for that. And so your launch is like, why that's weird. Why are we getting like weird slowdown? What's going on?
The code looks and runs fine. It passes tests. These are the worst kinds of failures. They're called silent failures in my personal vocabulary. I do everything I can to hunt them down and stop them. This is no exception. Now, first of all, let's talk about how you're meant to do this, which is like this. Our friend the await keyword has turned up right there. Very important. It tells the function that you can well the coroutine that you can pause there and hand control back to Python But crucially as well, we also have a different version of GET here, you see right there. The way Python's works, and this is, you can read more about this in my um Django con AU talk about the ORM and trying to make it async. Like this isn't real code yet. We haven't got an async ORM. This is an example. But like we have to have different
callables for synchronous and asynchronous modes. So we can't just have.get work with async. We have to have to have.get async. And so that's kind of the difference here, the two things you can see. But there's also one more thing, right? So like This is fine, this will work once there's an async RM. Again, it's not in yet, and this is kind of a demonstration of what it would look like. Now, what happens though if you do this previous one? Well thankfully. I kept doing this and it kept basically screwing up all my tests. And so what Django has it has this exception. If you try and do anything in Django with database right now, because all database access is still currently synchronous in Django. you will get this error. This is a synchronous only operation exception. It basically tells you you have tried to call a piece of Django that is synchronous.
probably the RRM from an asynchronous context. You shouldn't be doing this. You should be wrapping that code in async to async cool. We'll see that later. And that'll be a much safer way of doing it. Now this is what I call a guardrail. There's many of these as you'll see that we can add and do have in Django as we go along. But this turns a silent failure or you know, a silent performance slowdown into an explicit failure. And in my opinion, that's the way to go about things. This way, there's no way you can write that first pie c piece of code and have it pass tests. Jang will be like, no, what are you doing what are you doing? This is a synchronous operation. Stop it. That's kind of the idea. Now there's another kind of mistake you can make. This is the same code as we saw here on this previous slide, but there's one thing missing, and it's that of wait. What happens if you miss awaits? Well there's two things.
First of all, any side effects of it don't happen. Unless it's a get, it's not too problematic. Imagine it was a create though. If that was a create, it wouldn't actually run. I wouldn't make anything. So that's the first problem. Secondly, because asynchronous callables return a coroutine, basically await is looking for a thing to consume, which is a coroutine. If you don't have an await there, you just have a cogene object, because in Python everything's an object. And that cogene object in this example becomes book. You pass it to the template and you're like And then somebody's like, I can't do dot name of a coroutine, what are you doing? And then you get more exceptions. So thankfully this isn't this isn't really a silent exception in this case. It does become a problem as we'll see later when you have just a side effect and no variable assignment and you can't detect it. But thankfully Python's got a guardrail for you. We'll look at that in a bit.
So, in this particular case, at least, the most basic example that I made so many times, Django's got your back. Now, this is a principle we'll try and return to, like Django's goal, in my opinion, is to be a safety net to give you guardrails that when you are comfortable you can remove one by one, but by default they're there in place. Basically preventing you from doing bad stuff And that's just one of the examples. I think the main example that many people run into when they first try to do async code, like, oh yeah, let's try it. And they're like, what is this exception? That's what it is. You need to do things a different way. And the reason you do them a different way, it's dangerous if you do it the first way and you wouldn't notice the worst case. So here's another example. Um I have often said that the key thing about doing things in async world is parallelization, running two things at once or three things or four things at once.
And here's a perfect example. I have a situation where I have a view that makes a user account. It has two things. It writes a user row. That takes a network call to Postgres that takes time and needs to be async. It also sends an email that takes SMTP and could be async as well. Both these things happen asynchronously, great, let's do them at once and have them run in parallel. Fantastic. What could go wrong? Well, again, this is one of those subtle things where like it's not quite the same as you do it in sort of normal synchronous mode There is no ordering guarantee. So if you're not familiar, this asyncio. wait here, that's the primitive function in async IO that takes a set of things and runs them in parallel and then returns when they're all done. And obviously you can pass it a whole number of things, but there is literally no ordering guarantee.
You could run one thing first and then the other thing, both in parallel. Or crucially, you could run one thing and then the other thing might not happen because the process has crashed. So imagine in this case there are two different failure modes. In this case, I could have my create user account function run and write a row and then the process crashes before it sends an email, or I could have the send email function run and send an email and then it crashes before it writes a user account to the database. And that's two different failure modes. Now, maybe you're okay with both of those, but I prefer having less failure modes that I know. And so in my opinion, when you're doing things like this with side effects that kind of have an implicit dependency, you want to do something more like this, where like in my opinion it's much better to have like only one failure mode and that failure mode is we made the account but
didn't send you an email By explicitly ordering them, we can do that. We can have it, okay, account made, then if it crashes, it crashes. That's bad. Hopefully it doesn't happen very often. And then emails sent. Much better way around. Now of course this is only a problem for things with side effects. If you're in parallel just querying, getting data and then returning a result from it, that's very safe to do in parallel. But if you're not doing those things, if you're doing side effects. Be very cautious of doing this in parallel. It's very tempting. You can get amazing speed-ups from doing all queries in parallel. Like think if you had like a view with 20 queries and you did all 20 queries in parallel, you could probably get down like a tenth of the time But you've got to be aware of the extra layers of failure you're adding. Imagine there wasn't just two of these, there was like five different things. The number of combinations of how they can now go wrong and what can run and what can't if they're all paralyzed
is enormous So be very careful of that. And unfortunately in this case, there's not a good way to prevent against it. We can't detect it. This is a thing called a race condition. It's a classic thing in concurrency You can read so much about it on the internet. I couldn't do it justice here. But essentially these are just a fact of life with concurrent programming, asynchronous being part of that You can't really prevent them apart from structuring your code well. It's kind of a path you have to tread yourself. You can do things to help prevent them. There are some things you can do to aid yourself, but in the end, it's kind of par the territory In some ways the direct trade-off of when you ask for performance, this is what you get. This is what you know computers and processes and kernels handle as well. Like those things run asynchronously in some sense and they
they have handle plenty of race conditions So let's change tracks a bit though and talk about database access again. Now database access, as I said, there is no async ORM in Django 3. 1. Unfortunately, I'm sorry, it takes time So instead, if you want to talk to that database, you can't do it in an asynchronous thread. So you need to do it in asynchronous thread. And what you do is this. You can write your database code in a function, a synchronous function, and wrap it in the sync to async decorate. And what that does is it takes any synchronous callable and turns it into an asynchronous awaitable. So with this code as written here, I can actually await create user account and it will correctly make a sub thread, run the code there so it doesn't block the main thread. And then when the results come back, wakes up the coroutine and returns it the result and does exception propagation and stuff properly.
It's all done there for you. We ship this as part of Django these days. There's a reason we recommend you use it. It's much easier than anything else. However, it's not quite perfect. So imagine I did this. Like I've written code, I've got stuff that let's say again does two different things. It does the classic example of validates a user account name and then writes a user account row. And because I want to reuse my validation logic on say my sign-up form till I have like a little tick or a cross, I've made the validation logic a separate function But of course, obviously I am a media moderately okay developer. I know that, hey, if I don't do check username exists and then write user inside a transaction I can have another raise condition where, well, two requests come in from the same username, both check it doesn't exist, both go, yep, not written database, and then both write it in and they clash.
So that's why we had transactions, right? And so I'd be like, okay, great, I know this, I'm gonna write this code right here. And again, to the naive Python runtime, this code is perfectly fine. Because transaction will find the connection on its thread, it will set transaction up on it, and it might be stuff to run in it. Now here's the problem. I just told you that sync to async runs things in different threads, and it does And transactions and connections in Django are threadbound. So all this code actually does, if those are synchronous pieces of code, is it sets up a transaction on the async thread. doesn't use it at all, makes a brand new synchronous thread, and then in that synchronous thread it runs things outside of a transaction. And again, this is silent failure.
This won't actually fail. It probably will pass all of your unit tests, most likely, until you get those two requests just perfectly tied to hit each other and go around where the transactions protecting and bam you've got data corruption. And again, this is silent failure. It's really annoying. Now, again, Django could have detect that, hey, you shouldn't be using transactions in a synchronous In an asynchronous environment, right? That we can do. But this is more a problem for when we do have an asynchrono RM. Because what if you do genuinely write this code and we support async transactions, what do we do? Do we error? Do we try and port the transaction over to the other thread? Like do we silently fail? Well not do that, obviously it's terrible. But like these are kind of the problems with asynchronous design and like especially context managers not kind of going
and wrapping the code below them as you think they should do And in my opinion, not having transactions is one of the scariest failure modes because it is really a thing that's very hard to test. Like honestly, try writing a test for it. It's almost impossible to do it unless you perfectly write the code sequentially. And certainly having code as a single opaque block is pretty difficult. But at the same time, it doesn't really happen a lot in production when it does, it's really bad. Like your foreign keys stop pouring to the right place, you get dependency errors, or like things just get messed up. So I want to avoid this at all costs. We're going to look deeply into how to solve this in the proper async ORM. And again, if you want to learn more about that, I opine a bit on this particular topic in my DjangoCon AU talk this year But it's still kind of a hairy problem by itself.
So let's go away from databases now and talk about fetching. Fetching URLs in parallel. Classic example of a thing you do with async. Let's say I have a hundred URLs, I want to fetch all of them and see which ones are still alive. Because you know, good URLs don't change, but many URLs die over time. So code like this is what I might write. And it basically, you know, has a nice uh asyncode. wait. So you've seen this wait before. We're feeding it a list comprehension with all the URLs. And it's going to take all those coroutines, so there's a hundred of them, it's going to run them all in parallel, efficiently at once, and then collect all the results and then return it to us. And as you see at the top there we have a dictionary that's counting the number of things that are alive and dead. Now if you look closely at the inside of our fetch site method or function , You can see
if you know it look there's a bug. And that bug is that we fetch the value of alive, then We do their client get HTTP get there's a wa and await though there crucially and then we write back the value of a live plus one. Now you can imagine that like if There are many things running at once, which of course there are by design. Many of them can fetch a live and they'll all be zero. And then all go and do their request and then basically the first one fetches alive, does his request then suspends, second one then goes in, fetches live suspends and so on. When they all come back, they've kept the value of alive in the local memory area, but it's wrong. They've all got the wrong number. Again, this is a classic race condition. What we've done here is
we have a piece of code that isn't atomic where it should be. Now here's the fun thing In threading, this is very hard to solve. But in Asynki we can just do this. We can switch that await fetch the await and the fetch for alive basically in place. And what this means now is the alive is next to where it's written. It's kind of a single block. And with async IO, things are atomic between a weights. So if there's an await and a second await, everything between those two is going to run atomically. Nothing can barge in. Remember, it's cooperative multitasking. If you're not going to await, nothing can come in and stop you. So in asyncode, this is perfectly valid code and we safe. In threading, however, this is not safe. Threads can interrupt wherever they want. And they can just jump in and go, hey, nope, this line, we're gonna stop.
So crucially they can interrupt between reading await and writing await. And there's nothing you can do about it. So in the threading world, you need a lock or something here, but in AsyncO, thankfully, we have the nice property that between awaits, things are atomic. And this is one of the nice things about the trade-offs of like, you know, there are good and bad things about all async mechanisms. Await-based ones are one of them, but this is one of the good things. With things like threading or even with like GEvent, you don't know if a certain function is going to context switch. In fact, with threading, anything can. With gevent, who knows if somewhere in the function there's a request that's going to context switch you away. And so in those two, you have to be much more defensive about your atomic code. Whereas with an await-based language like not only Python but also Node does this for example,
you have that kind of built-in atomicity where like, hey, like as long as I don't await, I can do stuff and not and assume I'm not interrupted. And it's really nice Now here's another example of where that kind of comes into play and where it's really important. Now here is again it's a contrived example, right? But it's the idea of like I'm gonna do a long fetch and I have a second code team that's gonna wait for that to finish and then notify me. Now, this is interesting because, I mean, obviously you probably wouldn't write it like this, but let's say we did, right? This code will either run perfectly or Be stuck in an infinite loop forever. And guess what? You can't tell which one until you run it because it's kind of non-deterministic. But obviously, even a chance infinite looping is bad. What happens here? Well, crucially Remember I said that you have to have an await to give up control?
There is no await in this notify function. What that means is it never gives up control. So let's say we submit both functions, uh right down here, you can see, um, to run in parallel, and the event loop chooses to run notify first. It enters notify, it sees that ready is false, and it enters the loop Now, as all good loops in Python that could be potentially long running, we haven't just done a pass, because that would make it loop, busy loop, very, very badly, we put a sleep in there. So the telepreter can sleep and there's time to do other things. In threading this works perfectly. This does not work in Asyncoro because you're not waiting. And so what this does is this literally sits there and locks up the event loop and stays in Notify, just sits in that loop forever. Nothing else can run, ready will never be turned on and such a deadlock there waiting for something to happen.
And bam, you have an infinite loop, even though it looks like you wrote a good loop So how do you fix it? Well you might just do this. And this is better, but it's not fixed. Because uh remember a weight needs an asynchronous version. There's asynchronous versions of everything. It's true for the RM, it's also true for sleep. The default time. sleep function when you call it does its sleep. So it's not too much of a problem here. Imagine this was a five second sleep. And then what would happen is we'd start this function, we'd enter the loop, and then the way it evaluates is it right it evaluates to the right of the await first. So it's going to go, okay, I'm going to sleep for five seconds and then return none, and then it will await on none. So you'll get an exception saying that you can't await none, but only after you
block the event loop of five seconds So the real solution is this. And this you see has an async I. io compatible version of sleep that when you call it doesn't do anything and it returns an awaitable. And when you await that awaitable, that's when it sleeps. So you can see like this is it's not super subtle, but it's very easy to look at code like this and miss that you should take the two steps to get to code like this. And this is kind of partially the way Python's designed. This is the kind of the bad side of the await mechanic. The good side is the atomicity, the bad side is A, you have to remember to add a weight, and B, you kind of need a different version of stuff because a weight is a separate keyword that has an expression rather than being a way of calling a function. There's no like asynchronous call type in Python.
It's merely you call and it returns a coroutine that makes it an asynchronous function. So that's kind of the subtlety there. Finally, I want to talk about one other thing that I kind of screwed up on, and that is sync to async. As I said earlier in the talk, this is a very useful thing that you can wrap around database code and it runs it in a thread. And you saw earlier the transaction wasn't in the synchronous threads, we didn't have any side effects. Great. So obviously if I was doing stuff that Ran on the same connection like this code here, where the code is kind of like, oh, I'm gonna set up some like transaction level stuff on the connection first and then run a query on it. Um, this might be how you'd write it. This in current Django will not work. And there's a very subtle reason why.
So I said sync to async does run things in threads, in a sub thread, and it does, but it doesn't run them in the same sub thread It can run them in different threads. And remember, transactions connections are thread bound in Django. And so if these two functions here run in different threads. they're not talking to the same connection. Like the first one's gonna set up stuff on a connection, the second one's not even gonna use, and then even worse, a third thing might come along and get put on that first thread, because there's a just big pool of threads it's reusing And then it's like, oh, this connection set up and it's the wrong setup and things fail. Now this was me uh having a bit of a mistake. I kind of went for speed and flexibility over safety. And so guess what? It's fixed.
At least it's mostly fixed. In the most recent ASGIREF commits, we have changed this behavior so that things always run on the same thread. If you want to, you can turn it off for performance reasons. You put thread sensitive equals false, so they're running different threads. But by default now, they will all run on the same thread. They will all share the same connection. Middleware and the view code within the same thread. All manner of weird bugs that I have seen and I have honestly been trying to fix are now fixed by that one small change. And yeah, it runs a little bit slower, but it's not that much and it's worth it for the safety. And if you want to turn it off and you know what you're doing, go ahead. You can just turn it off. And that's just one example of like how do we defend against stuff like this. You've seen just a small subset of the examples of how you can break asynchronous code. There are so many more.
And how do you mount a defense against a thing that's like fundamentally just based on the language paradigm of doing? Right? It's as you've seen like you can miss a single await keyword and screw up your entire code base. Like if I have a function that makes users and I miss an await on the function that makes users, the view will run and return yes, user made. But I never awaited it. It's not actually gonna run. So silent failure is a really common thing. It's really tricky to build around that. The first thing I recommend is this thing called Python async. io debug. Now if Django has guardrails, this is Python's guardrail. Basically, if you turn this on, a whole number of debugging features for async apps turn on in async. io. There are quite a few things that these top two here are the most crucial to me. First of all, any coroutine
that runs for too long will be flagged to you and be like, hey, this coroutine ran for too long. Why is that important? It means it didn't give up control. A good coroutine runs for maybe a few milliseconds and it waits while it does a long thing. If your coroutine runs for a long time, and by default the long time is 100 milliseconds then it probably means you're actually calling a synchronous thing by mistake. Maybe you're using a library and somewhere d from the library is a HTTP call you didn't notice This will detect that and go, hey, when your coroutine ran, it took one and a half seconds before it yielded control back to us for the next to wait. That's not, that's too long, something's going on. So this lets you find the cases where you're actually using synchronous code where you shouldn't be, that's blocking. The other thing is the inverse. It can detect unawaited coroutines.
This detects the case where let's say you have a side effect function written in an asynchronous style that you want to run, but you forget to put the the await in front of it. So if you if you just have like, oh yeah, create user and no await, what happens is Python will make the coding object ready to run and then it will just vanish into the local memory space and never get run and get garbage collected. When this happens with this mode turned on, Python's like, oh , you made a coaching, they related it. What are you doing? This is bad. And so both of these aren't perfect. They're not going to like outline the area in your code that's wrong, but they're going to give you hints that something's not right. Like, okay, like I know that if a thing runs for too long, there is a blocking synchronous course somewhere And I know that if a code seems unawaited, I'm missing an await somewhere. And with that information
you can go do some of your own research and work out exactly what the problem is. Hopefully this will improve over time or get code analysis tools, but it's a good start Again, Jang has other things like synchronous only operation, I call these guardrails. These are things where when we can definitely detect you're not doing a thing right, we'll raise this exception. There are a very few number of cases where you do want to try and call the RM from asynchronous code, especially when it's like a one-off function or something like that, but generally you don't want to do it, and that's where a guardrail makes sense. Now of course these all have options to turn them off. That's very important to us. It is totally up to you what you want to do. By default, they all come turned on. Like all of Django's other safety features, like the security middleware and so on and so forth. That comes by default.
And remember, this is all because asynchronous programming is by its nature hard. It's not that like Django or Python are screwed up here, like concurrent programming in general and async programming in particular are just difficult to think about. You can basically screw up in so many more ways than you can in a synchronous context that we just have to defend against all of this. My personal way of doing this is to write code synchronously first. That lets me understand how it flows, get a mental model of it, then I can write a good test suite. Then only when I've done that and I know that that code is a performance bottleneck, then I will go and refactor it to make it AC. This is what I recommend to you, that you do as your mechanism, because you may find that things you think will be performance bottlenecks aren't, and vice versa. Because you don't want to write your whole thing in async from the beginning.
It's going to be a massive nightmare and more work than you want. You want to have the ability to like lift up, oh like this one view. we're gonna make it async. And that's one of the reasons Django supports a hybrid mode, right? Like you can just make one or two views async and the rest synchronous. Django will just deal with it for you with all the right safety around it and there you go. Bob's your uncle. You can ha have both worlds for the price of one. That's kind of the point. And that's kind of Django's contract here. We are here to do our part where we can. Django's job is to give you a safe framework that you can quickly and safely develop code in, and then when it comes time to make it bigger you have the room to carve out like hey this section I want to trade my safety for speed or this section I want to replace what Django does That's what we're here for. And that philosophy, as it were, continues into how we're trying to design Django async.
It's really important to me personally. We continue this. is I can just hammer on a keyboard for 20 minutes and make a site and not worry about having giant security holes in it or like giant performance holes. That continues here. But that's kind of our job. And so I'm telling you we will try and continue that and make async the best it can be But of course that is in the framework of async as a whole. It's not just a Django problem, it's not just a Python problem, it's an everything problem. Node is looking at this, RAS is looking at this, both of them have an await pattern as well, for example Many other languages have tried this over the years. Like there's so much more to be done in terms of making good, safe, concurrent programming. We're just at the foot of the mountain of what we can achieve. And so I hope in five to ten years I'll be standing in a similar position being like, it's great, you can just write async now and it's all perfectly safe and like you can't screw up and all the race conditions are detected by default.
But Unfortunately, it's going to be a little bit of work to get anywhere near there. And we may ne we may never get there, but my hope is one day that async programming will be a lot harder to make mistakes in and really worth writing stuff in from the get-go rather than doing synchronous first. But until then, I uh hope you have fun trying Django's new async stuff. Um I hope you have enjoyed learning from the ways you can you can make things go wrong. I do encourage you to try this at home. Like Download Django 3. 1, try and break it. Just don't do it in production, please. And if you're interested in helping out with async stuff, please come on to the Django forum. We have an async subforum where we discuss stuff and designs You can see me talking about transactions in middleware there, for example. But yeah, until then, I hope you have a lovely afternoon. I hope you've enjoyed DjangoCon
Europe and I hopefully see you somewhere around the world in person at some point soon. But until then, I'll see you next time.
Yes. Django can run `async def` and normal synchronous views together in both WSGI and ASGI modes, though WSGI runs async views in a less capable per-request event loop and ASGI wraps synchronous views in threads.
Discussed at 0:49The asynchronous operation returns a coroutine object rather than running, so side effects such as creating a record may never happen. If the object is passed onward, it can also cause type or attribute errors; Python’s async debug mode can flag unawaited coroutines.
Discussed at 7:46Be cautious: parallel side effects have no ordering guarantee, so a crash can leave one operation completed and another undone. If one action depends on another, run them explicitly in order; parallelizing independent reads is safer.
Discussed at 10:05Put the database work in a synchronous function and wrap it with `sync_to_async`, then await the wrapper. This runs the blocking database operation in a thread instead of blocking the async event loop.
Discussed at 12:28Use the asynchronous `asyncio.sleep` and await it inside the loop. Calling synchronous `time.sleep` blocks the event loop, and merely adding `await` to its return value does not fix the blocking operation.
Discussed at 21:42Thread-bound Django connections can break if related synchronous functions run on different worker threads. The default thread-sensitive behavior now keeps such calls on the same thread; `thread_sensitive=False` is available only when you understand the safety and performance trade-off.
Discussed at 24:06Enable asyncio debug mode. It reports coroutines that run too long without yielding, which often indicates blocking synchronous work, and it warns when coroutine objects are created but never awaited.
Discussed at 25:38Write and test the code synchronously first, then convert only the views or sections that are proven performance bottlenecks. Django’s hybrid support lets you keep most of the project synchronous while selectively making parts async.
Discussed at 27:57Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.
Published June 13, 2025
Published June 13, 2025
Published June 13, 2025
Published June 13, 2025
Published June 13, 2025
Published June 13, 2025