Django schema migrations and deployments minus the misery

Speaker 1: All right, hello everyone. Thanks a lot for coming and welcome to Vigo. I hope that your fride was quick and easy. Mine was a bit uh long, but uh hopefully I'm here So I'm Antonis. I uh work for a company called Platform. s8s, um where we create uh a platform as a service. Uh we're an infrastructure provider, we have also a booth. So if you want to hear more about what we do and what is AppSan and how you can uh have an easier life deploying other applications and especially Django applications, feel free to to reach out, although we'll see a few things in the in the workshop Personally, I'm a Django developer. I've been coding Django since 2012, 2013, something like that. I remember I have a t-shirt that says uh uh

Speaker 1: now migrating. It was I think Django 1. 6 before that We were uh depending on other tools for migrations and now we have uh a better life with uh Django migrations embedded. So let's hope that uh we can make our lives a bit easier and see how we can do this with uh And when we have early updates, when we have you know more interesting deployment setups. So let's get it started. By the way, if somebody uh uh can see the stream and they have any issues let me know otherwise uh we will continue uh because we're also streaming some people uh the workshop so what are we going to talk about today um So we're going to to get ready. This is what we do right now. We're going to talk about general deployments. I'm sure that most of us in here have deployed at least once

Speaker 1: our application, but we're going to go back to the basics. We're going to talk about you know What are the building blocks? How do we do this? And all those things. And then you know we will take a bit of our happiness and we were going to talk about static asset management. Which is great, but uh you know it has a lot of work sometimes and maybe we can do this easier. We are going to talk about schema migrations, which is amazing as we talked about because uh you know Everything from our models goes directly to our database, but what happens during deployment and even worse, what happens if we need to roll back or if we want to if we sorry need to run uh the application um in multiple regions at the same time And uh we're going to talk about the silences and then at the end we're going to

Speaker 1: see a few uh a few examples and potentially solutions and uh we can have a discussion. So let's get it started and I think uh understanding the Django deployments and you know the building blocks is maybe the most uh the most important thing. Uh Usually our setup, it can be what we do locally, it can be something that we have set up on our own server, or it might be something more interesting that we have set up in a cloud or through a provider like AppSan is uh is comprised from those things. Uh on top we have a proxy. It can be your favorite server like NTMX or you know your CAD server or whatever. Then we have your app servers, we have a CAS Hopefully, or uh usually any database. That's like the minimal thing that you need to run a Zoom

Speaker 1: application. Of course, we can have things like uh a cellular worker or you know other interesting stuff uh alongside that, but uh you know this is like the minimum vial product the minimum viable product, the MVP of a Django application. So for the proxy, um Are you using something different? I think that most people will start from those things. So if you have not heard of those before, that's fine. Nginx is my personal favorite, but again, that's uh you know uh up to your taste CADI has been very common lately in traffic. If you're more into the containers ecosystem, maybe you've seen traffic a bit more. So the idea of the uh of the proxy server is that's going to handle things like your TLS. So you have an XTS connection coming, you need to

Speaker 1: stop the TLS there, you know, negotiate it with the browser or the application that's talking to you And then from that proxy you're going to continue to your uh to your application. You're going to do things like uh the um domain names, so depending on if you're pointing the www dot whatever uh your server is or uh other things. Uh you're going to redirects from non-ww www or you know reverse and all those things. Um For the application server, we're going to, I mean that used to be the case. It's not the case anymore, but you know, those were the uh the most common servers. Uh whose here is deploying using uh Unicorn, for example. Okay, I see like maybe 50% I would say

Speaker 1: anybody deploying with UVCorn? A few hands, nice. So you have your async code in there with sync to async and uh reverse. Uh you're very, very brave. uh for doing that. Any other servers? Is anybody here deploying with a different server than those? No. So okay, I got uh quite a good of uh of uh the idea here so The application server is the server that's actually running your Python application. So when the request comes in, if the request needs to be served by an API code or you know if you set a view in Django, it's going to be served by one of those two servers. And those two servers have a view a few uh good things and bad things. You know, everybody complains that Python is low, Python has you know the global interpreter log and all those things.

Speaker 1: We know that we love Python, so let's see how how we're going to do this a bit better. So usually those uh those servers uh are going to to serve one request at a time. And the way that we're going to scale those servers is by having multiple processes running one uh next to the other. This is not necessarily true for the ASCII interface, which is the asynchronous interface uh of uh of Python Because finally we can have the cool things of VentLeaks and the other things that you know other ecosystems had in the past. But the idea is that we have process-based management of those servers. For the cast, uh just the examples, not nothing serious here. I mean it depends on uh what you uh prefer

Speaker 1: doing. But uh Redis and Memcast used to be one of the uh favorites. Uh my personal one is Redis. Usually you will see my favorite thing uh or two on the top. Uh and memcast has been around for a while. I personally prefer Redis because uh When it comes to Redis, you can do other things as well. So Redis is not only forecasting, it supports strings, it supports queues, supports other things. So sometimes you want to use the same tool for multiple purposes within your application And then for database, Postgres and MySQL, like uh uh the most uh uh common ones. Um I've started with MySQL, uh I've been continuing with uh Postgres, and of course you can have things like MongoDB or other things, although these are not uh directly supported in Django.

Speaker 1: Also, another thing that uh many people uh use lately is uh SQLite. You might use it is uh for your local deployments or you might use SQLite for um uh not local deployments local development and you might use HP Lite as well for uh production environments we've seen many uh people doing that lately It has a good thing that uh it's very fast, very easy to deploy, to manage to backup and everything. But uh on the other hand, if you have uh multiple servers and applications, it's not uh it's not as easy So now that we have the basics, let's start talking about a few of the things that you are going to stumble upon when you deploy your first application.

Speaker 1: So let's say that you uh develop your application locally, you do monet spy one server, everything runs perfectly, you know you have your static assets and all those things. But uh now you deploy. The the first time that you're going to deploy your application using the unicorn or uh a similar server, you're going to see a very nice broken uh screen So you say, okay, what's happening here? The thing is that uh Django is doing a lot of nice things when you develop locally, like ceremony or static assets, but when you go to production You're not going to waste those uh application server resources to serve SAT assets. As we talked about, uh Python application servers can serve one request at a time in quotes because that's not necessarily true with uh with the ASB

Speaker 1: interface but still this is very resource intensive and that's very good when you serve Python code Because when you say Python code you want to make sure that you uh you get away with uh with Gil and all those things. But when it comes to a simple CSA or JavaScript file That's not really the case because let's say that I I load a Python application that has 20 different static assets like uh five images, two JavaScript files, CSS files, you know, all those things I do not want to waste all my uh application server resources just to to serve those static assets. So by default, Django is not going to serve those static assets for you and you have to find an alternative way Um like any suggestions for tooling around this? No? Nobody?

Speaker 1: What are you using? You're not serving static assets in production?

Speaker 2: Nginx.

Speaker 1: Nginx. So uh here comes you know the proxy. You remember the in the beginning we talked about the proxy in front of your application. So you can configure your application so that static assets like routes like/static, which is the most uh common one, are going to be served by your proxy or whatever you have in there, and then the rest of the Python code is going to be served by your server. But then you have to make sure that your proxy has all the static files. So what are you doing there? Maybe replicating the static assets to all servers or building static assets collecting static sorry to all servers. Maybe that's a solution. Another common solution is to use something like white noise. Anybody has used white noise in the past A few hands, nice. So white noise

Speaker 1: is a um a library in Python that's going to serve your static assets and it's going to make sure that your static assets are very well cast and everything. That's very good, but still your static assets are going to be served now by your Python application. So in order to get past that, you will need to put your proxy again in front of you. And that proxy is going to cast those sadic assets in the ads, or maybe you have a CDN and then the CDN is going to cast those sad assets And the third solution would be to use uh something like an external CDN. So you have uh somewhere external outside of your application where you push your SAT cassets And then you serve them from there. So for example, you can have things like uh an S3 bucket or you know uh those things where your static assets are going to be served by there and then your application is going to continue being served by

Speaker 1: uh your uh Python code. So how are we going to to mix all those things now that we talked about you know in general into more specific things in um uh in Django So, first of all, the static assets can support multiple storages and multiple finders. So that's two very nice tools that we can use to better handle our application. So, first of all, storages Storages are uh the way of Django of storing those static assets after it has uh post-process them. So Django is using actually I should have put those in reverse. So first we have the finders, which are the um the modules let's say that find the static asset inside the application. The default way of doing that is going through its

Speaker 1: Django application. So let's say that I have three Django applications. My Users application and my uh sightings as we will see in the in the hands-on example later on. And then so Django is going to go to those applications, go to the static directory, so that's the the predefined one And in there it's going to find all the JavaScript images, uh CSS files, and so on and so forth. After finding those, it's going to use one of the storages in order to store somewhere those files so that we can serve them later on. So in the case of uh the nginx server that we talked about in the beginning, we are going to store the static assessing directory that's going to be available by uh by the nginx or whatever proxy we have in front That's very easy and very straightforward.

Speaker 1: But what happens if we have multiple servers? Then maybe we are going to do this on every server. That's one solution. Or the second solution is to use a storage that's external to the application. So we're going to push the static assets uh outside. The other solution is to use white noise, or maybe there are other packages that I'm not aware of that use that do the same thing, which is serve the static assets, has them So there is a space and storage called the uh hast static file storage or something like that. Oops That's going to uh let's say that you have a file called main. js. It's going to has it so and rename it to main. has. js. And that has going to be saved in a JSON file that's going to do all the mapping.

Speaker 1: And then be able to share those files with permanent cast, like cast it until the end of uh uh Python 3. Let's hope that's uh not going to be uh very close to us. That 's uh the static file sorted. So the default one is just copying files from your applications to a directory or an external that you uh that you want. And the manifest static file storage is going to hust them and store them. As we said, the manifest static file storage is the one that is going to allow us to cast those files indefinitely Because otherwise, let's say that I'm seeing the application now. I have a CSS file saying that I want everything to be

Speaker 1: black and white because that's my current theme. I get the request from design. I want to make sure that I include this nice yellow color that is now introduced in a brand. I'm updating the file, it's being collected, it's being served, but maybe some clients have uh cast locally the old file, so until this actually propagates to all the customers and all users uh I'm going to to be yelled at by uh marketing or by design and of course we do not want to be yelled at. So having a manifest static file SORAD is another nice thing that we that we put here, but but it presents as an issue when it comes to uh deploying applications in production. And having multiple instances of the application running. We're going to see those in a bit.

Speaker 1: Wrapping up, uh, I want to uh to make sure that we have you know all the initial um Initial bits and pieces in place so we can uh uh get our hands dirty. We have schema migrations. So schema migrations are what? Are um files that describe a change in state. So let's say that I have a database with a table that has three fields. I go to my model, I add a new field, and that field goes and creates a new schema migration. So that as soon as I apply the schema migration, my database state will go into from the previous schema to the new schema. That's it. It's very simple Also it supports rolling back, so it supports the reverse operation of going back to having four fields, back to having three fields. That's a very good thing because uh when capture

Speaker 1: Django, uh at least personally, I have forgot how to make sure that my database has you know all the schema that it needs to have because uh for all those years I've been you know just writing models, writing migrations and Everything happens automatically. The bad thing is that I've forgotten how to write SQL. Sorry about that. I Google a lot and thankfully we have things like Copalette, OpenAI, and other AI tools that write very good SQL code. Those migrations also have orders, dependencies, so it's really really easy and really handy to do them because I can make sure that uh they're applied in the correct order. Dino keeps track of what has been applied, what hasn't been applied, and all those things. So in general, they are great. But what happens as I deploy the application

Speaker 1: and maybe multiple versions of the application start uh running at the same time? Then I need to plan for that and this is uh the second thing that we are going to talk about today. Now Going back to uh to the basics, we we talked about two very important things and two very nice things that Django uh presents to us, which is schema migrations and static file storads, but uh we said that maybe uh we have challenges when horizontally scaling those applications Why this is a challenge? This is a challenge because there might be more than one version of the application running at any given point in time. This sounds very easy and very simple, but it's a very, very big uh trap that we might fall in. The

Speaker 1: reason for that is that you know applications have been become more flexible. We would not have you know like a single server that we deploy the application and then we do uh minor deployments, but we have multiple servers, maybe we Uh we do what we call rolling updates where we deploy the application to a new server and then slowly and gradually uh decommission the old one and so on and so forth. So this uh presents challenges. So for static assets Uh we talked about uh how to serve static uh assets without depleting app server workers. Uh but it's not as easy because we talked about you know a single Nginx server in front It's not easy if you have multiple servers or if we uh we do this rolling update. Uh how do we serve the correct cast

Speaker 1: or how do we serve you know let's say that uh we have a a web page and the web page is going to do five requests. And uh just for this example, let's imagine that we do not have HTTP2 that you know makes sure that we have a single connection for everything and all those things. And let's say that for some strange reason Uh some static assets are going to be shared by application A and some static assets are going to be shared by application B. Sorry, by server A and server B In that case, if server A and server B are in the same version, that's good. No problem here. But what happens if, for some strange reason, because of a proxy failure or whatever, the request Ends up in a wrong server. In that case, let's say that we have a new static asset with a new has, and we try to request it from an old server,

Speaker 1: we're going to get back a 404 and a broken page. And same happens with the reverse. So managing all those application versions has become has introduced a new problem. So we solved a lot of problems, a lot of problems like you know zero time done deployments and all those things. But we have thankfully introduced a new one. As developers we are lucky because we have a new problem to solve. So this is what we're going to try to solve today. The second one that we're going to solve today is how to apply schema migrations At which point in time did you apply the schema migrations? Do you apply them before deploying? After deploying? Between deployments, if you know it's like a rolling one This is another big question because if you add the remove fields, I mean if you change like the default value, I suppose that you can live with that

Speaker 1: because it's a default value. Potentially you already have code that can handle this change But what happens in additions, deletions, or you know, changes in uh the way that uh, for example, we have uniqueness in the application For all those things, we need to make sure that we first of all plan for that. So our code is not as easy. We have to make sure that you know those migrations happen in a specific order and maybe through multiple deployments. That's might be that might be another strategy And the other thing is what happens with rollback. So let's say that we have a deployment, it failed, or maybe we we we have a bug that we did not intend to and we want to go back to the previous version. With the application, that's very easy. Just go back to the previous version. Most of the systems that you we use at where have this ability and potentially we have already baked this ability inside our system or maybe we have you know like a commit that's a reverb commit

Speaker 1: that's going to to be deployed and then uh have the the old version of the code. But if this code included migrations, then things start to be a bit more interesting. Because running back, okay, it's very easy. It's as a command migrate application name to the previous version. That's how Django does it. But is it as easy? Personally, in my production experience, I've seen many times where I was very, very, very, very confident about being able to go back to the previous version and you can guess what happened there next. It wasn't as as easy. So um If you want to follow along, this is a repository. It's under my GitHub account, so github. com slash a calipet slash bigfoot.

Speaker 1: This is the application that we're going to use. Um and uh also potentially we're going to to install uh the AppSan platform CLI. I'm going to deploy it to AppSan as well for the sake of the demo, but uh this is something that you can uh to add it later on if you want to. or on your own time it's not necessary for uh for now uh so Let's see how we can uh plan for those things and see how we can make things better. So back in here, let's see Okay. All right. So let's start with the application

Speaker 1: Which is a very simple one. Actually what I have done here is the size okay for you at the back or should I make it a bit more big? Okay, cool. So I have a very simple application which has um One Django application cited, applications and applications, which has a model file with a few models. So nothing very serious here. Users, sightings, so the application just for uh for context. is an application that uh people can report sightings of big fruit. Uh I've seen I've seen it. It's somewhere up there, so we're going to find together

Speaker 1: And then we're going to add comments to those and you know a simple like a simple uh application uh using Django This application is completely identical with the old one. The reason that I have two applications here is that it's not as easy to showcase problems that will happen when we change versions So instead of uh trying to be fast enough to to cast these three deployments, I'm going to deploy two applications, like the old one and the new one, at all times so we can see all those problems in real time So the old one is going to be the one that we are going to start working on, and then the new one is going to be the one that we are going to start adding features into it. So let's see what I have here Let's make sure I have cleaned up everything.

Speaker 1: Nothing fancy here, I'm just uh using Docker locally to start those two applications, and this means that I have uh So nothing fancy here. I have one application on the left, which is I think the old one, and one application on the right, which is the new one. I have opened the two ports. Um and as we talked about, we talk about Bigfoot. If I go to Admin Provide that I remember what user had created, uh I should have be able to add new side

Speaker 1: in. So let's I'm going to create a new super user I'm going to go to sightings. Another new one demo. By the way, I love how Django is uh providing that admin. That's one of my like when I talk with friends, it's my uh go

Speaker 1: to thing. Try Django, it has admin out of the box, you don't have to care about the thing this thing. And your customers or you know your users will be happy about it. So that's the thing. Uh I saw it too. That's the the comment Of course, I'm not putting the correct data in here, but the idea is that we can have it like this. So as you see, we have a siding here. And we have a setting here. So both applications have the same data. They they they look into the same database because that's what uh we're going to to see here And everything is fine. So now I need to start changing things into the new application so that we can start seeing all those problems and how they arise So

Speaker 1: let's read the user as is, we don't really care about the user, but deciding, okay, we have a title, description, owner, etc. Let's start with a simple uh with a simple sentence. So in here, I would like to uh make sure that the title is unique, for example. Because we want to make sure that sightings are not duplicated or we've not have spam from users trying to input the same thing. So, what I'm going to do here is update uh the model in here And then go to new and then do something like uh make migrations. So make migration is going to create a new migration, as we talked about, which is going to have these chains Which is very interesting.

Speaker 1: This is how Django adds these operations. It says that this is an alter field operation. So we are changing a field in the database. And the new field, the new data for the field is this. So very simple. Now we are in the case where we we have the new application, it's not uh deployed yet, and we have the old application which is running uh And actually let me verify which is the old one. So the old one is import 8000. So here is the old application, it's running. Um nothing happened here, nothing changed in the database. I can still go and add new things. Uh and of course. I'm not going to add a new duplicate siding because that means that uh this will fail afterwards So this is the beginning of the application.

Speaker 1: Uh the beginning of the deployment, I can go here and start the deployment. The deployment process typically would be that I deploy a new person of the application and before requests Start coming into the new version of the application, I'm going to run uh my migrations because I want as soon as the application is up and running, all the database schema to be uh ready for me to start accepting requests because maybe my code depends on those requests. So I'm going to do migrate and now we're going to see the application, uh old version of the application running and the new version of the application running at the same time. Nothing fancy here. So my migration was applied to the database, but the old version of the application can still work.

Speaker 1: Why? Because what I changed was not a very big scheme update. It was not something that uh um an each plan for it. That's very easy and handy, so I can be happy about it. Let's see that I continue. updating my uh my application. Now a new request comes in from uh from legal in compliance and says you know We need to get GDPR uh approval of the people adding new uh sightings. Of course, we're in Europe we love it, but sometimes we have to click a lot of you know checkboxes before we come and use applications So we go here, we do something like GDPR ZDRK.

Speaker 1: And then I'm going to use a Boolean field. Which by default should be false because I have not taken any uh uh acceptance from existing uh sightings And uh let's leave it like this. So, what will happen here is that the new version of the application will go to add a new field to the database, it's going to have a new default value. Probably this is not a big issue for me because the old version of the application does not know about this field, so maybe I will be fine. So let's try this in that. Let's see the migration as well. Where is the migration? D DPR. Here it is. So

Speaker 1: nothing fancy here. Another Django operation which is called add field. So I'm adding a new field to the database. And I'm using this model, this field, and this is the actual field that's going to be populated in the database. Perfect. I'm also migrating because let's say that we start deployment and okay everything is okay Now a friend of mine comes in and says you know I saw Bigfoot yesterday so I have to go to the admin. Okay, that's fine. Let's go and add a new siding Um they tell me that okay I so uh sighting in we go.

Speaker 1: This option to add the DPR compliance because I'm still on the old uh version of the application. Uh I will not add any comments for now. I'm just going to add this to the database. And here the problem starts happening. So what happened here is that I have a field in the database that says It cannot be null. And the old version of the application tried to save a new row in the a new row in the database without adding this field with an explicit value. So why did that happen? Because Django when it has defaults it can have like a Python function and other things so the default cannot be in the database. With an asterisk, we are going to talk about it in a bit. But by default, it cannot add things in the in the database as defaults, and this means that the default value actually happens when the model is going to be saved in the database.

Speaker 1: So because the old version of the application does not have this new field, everybody trying to save a new uh sighting during a deployment that lands to the old uh version of the application is going to get a 500. And the 500 errors are not something that we want to have in the application So let's go back and see how we can potentially solve this. Any uh clues or any ideas on how we're going to do this No, maybe we can leave 500, maybe it's for Laminando too, it's fine. But apart from that, let's say that it's uh not something that we can accept So from the error methods I'm getting that uh we cannot have null values in that column.

Speaker 1: So what I'm going to do here He's going to uh to make sure that this can be null If I remember correctly, I think that's the uh the field in the database. So what I'm seeing here, so instead of doing the migration like I did before, just with the default value, I'm going to allow an alt value so that I do not get this press So I'm going to make migrations again, migrate again. So new version of the application is coming out. And let's see if we can make this work. So it did work and we're great. Or are we? If we go back to the database now and let me check what's the best uh what I can do is the Django

Speaker 1: cell. I'm going to do from Bigfoot Models import hoops um sighting Let me remember the the private key So this is the setting that we just ended in the database. And if I go to the GDPR okay I don't get back in value because this value is actually none. So what happens here is in the database I might have a few sidings with false, which are the old ones

Speaker 1: A few sightings with null, which are the ones that created after the deployment, but before the old container was decommissioned, and you know the correct ones that happened afterwards. So I have three values in the database instead of one So maybe I solve my my problem for now, but I still have invalid data in my database. This might be okay, so we can make sure that our um Our newer version code handles this correctly and handles uh null in the same way that it handles false, but this is a very specific case and maybe we have a field with uh more interesting data in it So I don't think that this solution is the best one. What we would do in this case would be to go and update do something like Sightings

Speaker 1: Objects update with D D P R okay We are actually going to filter with D DPR OK equals none and update everything with dbrk equals false. So we're going to deploy a new version with a new migration, which is going to be a code migration that's going to run this command. So after two deployments, I'm going to have the correct state in a database and maybe in between my New version code is going to handle correctly null and false values alike. So now if I do the same thing. I get false because that was a null value and now it's a false value.

Speaker 1: So okay, we solved our problem, but maybe the solution is not great. One of the things that have been finally introduced, and I'm very happy about this in Django 5, is that now you can have db defaults So instead of doing that, let's go back to the initial uh the initial case. So for that I'm going to roll back the database migrations to uh to the first states which was this one and I'm going to migrate big food To here. So what happens now is that my code is now back to the to the old version of the application. Sorry, my code, my my database, so that I can still uh

Speaker 1: do changes So, what I'm going to do here is going to add the b default. The b default means that the default value is not good is not going to be assigned by the Python code, but it's going to be assigned by By the database itself. So it's like a hook that happens there. And in that case, I'm going to do I'm going to also delete the migrations because I don't need them anymore. I don't like this, I'd not like this. And I'm going to make migrations again. So now in theory nothing changed I can still go here

Speaker 1: another new new sighting. Let's try and do this again. So a new sighting. I'm very clever with descriptions This is a title. I think that AI would be better than me in citing those values. Let's go, let's go, let's go I will assign a better score to this one for no particular reason. And now this was set to the database without an issue. But again We need to go back and see if actually this was what we wanted to do. So again we said something from the old version. The old version has nothing to do with ZDPR. So let's see what happens in the database in here. Now I'm going to get uh the one with uh primary key equals four.

Speaker 1: And DDPRK, it's going to be false. So now we actually manage to do what we want in the database. We've been going back and forth a lot with uh you know migrations and all those things, but uh what I wanted to uh uh to keep up from this is that even though we have migrations, even though our migrations are great, we need to make sure that our migrations are forwards and backwards compatible. So we want to make sure that We do not create migrations but actually break the code that's going to come into the future or migrations that going to be break the code that existed in the past. So we need to make to be very thoughtful of those and maybe and especially excuse me when it comes to uh user-facing code. Of course there are other tactics that we can follow.

Speaker 1: For example, we can add everything in the database but then expose the APIs So we we do this again in two steps. One step is doing the migrations, putting the code, putting the models, putting everything, and then exposing those APIs or those views to the user. So In that case, we do not mess with those models, but it's not always possible. We can have models like this one that existed before, and then we just want to append uh another new field into there The second thing is about static assets. So now for static assets, we are going to see a few things and we are going to do them together. But unfortunately, I could not think of a good way of serving requests between those two applications from the same domain. So for a few of those things, you are going to take my word for it that things work

Speaker 1: this way. So for that uh we are going to go here and uh do collect collect subject So let's do other composer run. I'm sorry exec alt doesn't really matter. I'm going to do collect static And let's add more verbosity so that we can uh we can do this. So, what happened here is that Django went to all our applications, found all our static assets, and then created this app static uh directory and actually we can go and see this so we went to the old one which is here

Speaker 1: we went to static And then you see here that we collected all the static assets from doing the application. So now we're using the default static assets collector, so that's good. Which means that we don't have any extra files, we don't have any you know special cases, but the problem is that if I for example go and change um Not these. Let's go to the CSS. Let's go to the to the background color. I want to change the background color to I don't know RZB Let's hope that the pilot will help me help me here.

Speaker 1: Oh, it has a color picture. Okay, I mean a lot. Let's use this color. Okay, so I'm changing the button color. Into the old application Of course it's not working because I'm checking out the wrong file No , no, no. Okay, now it's working. So I actually stumbled upon the same problem that I was going to talk to you about, which was that I had cast.

Speaker 1: The old CSS file. My browser said, okay, you know, this is the same file. Maybe I will not care about it, I will not try to contact it. Also, maybe it used a header that checked against the size of the file But the size did not change because the characters were exactly the same, exactly the colors, so the characters were the same. And it's not it was not going to pull uh the file from the server So the problem here is that uh the old application uh and the new application might have differences, but those differences might take some time, and that time is not dependent on us because the browser might have its own policy It might follow cache headers, might not follow cache headers, and you know there are many systems in between your server and your application, and maybe your users will continue seeing the dark version of this instead of the new orange version of this

Speaker 1: So the first solution for this is go and change the uh the static slot to the host slot because um actually I need to Let's see. I need to uh change the has storage so that we can see what happens. Because here, as you can see, all the CSS files, for example, the app CSS. Always has the same the same name, not in here of course. In the static directory that has collected this. As you see, it's still called app. css So now I'm going to go to the settings file and find static storage.

Speaker 1: Django hast. Let's copy and paste because that's what we're good at. Do you remember the five slots? Okay.

Speaker 1: Always have a backup branch if you do not remember everything by heart. Here we go. So what I'm going to do here is collect SAT again. We see the correct value With fixing this I love CSS.

Speaker 1: Let's remove source maps one moment and we will be ready to go because Django is trying to find those as well. So what happens here is that In the root directory, in the build uh file, you see now that I have those uh app. has. css. So finally now I'm lucky enough and uh if I change something the browser will remember the file name. So if the file changes, the file name will change so that the application will always load the correct file. And how does uh Django solve this? Let's see if we can for oh okay, we were lucky. Um so Django is doing uh is creating this file which says that you know

Speaker 1: this file which I found through static file storage Is renamed using this has at the end. So Django is using this file so that it can do this mapping. And whenever you load an application, you have you know app. css in your uh in your templates, but Django is going to replace this With the file that it finds in here. Everything is great in here and amazing, but now let's say that I go to the new app CSS This is it and now the background color uh instead of orange I'm going to uh how did I make the color open? Okay, let's do something lazy. So

Speaker 1: in the in the new application I do not like uh orange so I change things to to red So I'm going to do collect static, but now I'm going to run collect static on the new application to see the differences So I do that, which is great. And uh if you if we go here and we go to the static application and then we go to app. css, we see that we have We have this here for the old one. App CSS on the new one.

Speaker 1: We have to use the static file storage on the new application It's very hard if you have two applications with the same code and remembering that you have to do this all the time. So let's run static assets. Uh collection again. Perfect. It worked this time. So if I go to the new application into the statics director static directory Come on dear. Here it is. I see that I have a new app. css. The problem is that the problem and the solution at the same time is that the old one and the new one have a different has That's great because that's what we wanted in the first place. If the app CSS changes, we want to change the file name so that it's not cast and so on and so forth. But what happens now if I try to load

Speaker 1: The new CSS file from the old server. The old server will not have this CSS file, and this will break my application again because uh this file is going to be a 404 in order to uh To test that we're going to go to app. css in HTML files So I'm going to to base HTML in the old app. And I'm going to touch it a bit. Okay. Just let me do this. What I'm going to do here is does not exist.

Speaker 1: css I do not remember the has, but it doesn't really matter. What I'm going to see here is that whenever I'm going to load the old application, the users are going to see something that's broken. So what we need here is a solution that's going to work across applications and at least to my knowledge the only solution to do that is to host the files externally. So one solution would be DS3 Okay, stores them somewhere in the back elsewhere. The other solution would be to somehow store those applications uh and send them by a proxy The good thing about proxies is not only that they can you know do all those management and stuff and serve files for us, but they also have their own cast. And for example, in the Absam case, and now I'm going to switch to here, we have this nice um

Speaker 1: bracketing here. So here we say to To the absent proxy. I want you to serve files. Okay, that's fine. I want you to serve them from the static directory. That's fine as well. So I don't have to push files in external location so that they are served But we also say that you know by default I want to keep those files for one hour. What does that mean that you know when we have this rolling update, hopefully our application is you know uh Visited often enough that the old version of the old CSS file will be already loaded and cast into the proxy. So the next time that's going to be requested is going to be uh Requested by the correct application. The other thing is that this static directory can be a mount, so we can have

Speaker 1: sales storage between old and new versions of the application. And we can make sure that you know both old and new are served from the same directory and are always available there. So that's one of the solutions. Another solution would be to use uh the the white noise uh middleware so that we can serve files directly from the Python application but then cast them. That's very handy if the proxy and the application server are in different machines. That's also another setup that we uh we see often enough. In our case that's not necessary because uh the proxy and the application server have access to the same local directory for uh serving the static assets And um what else did I want you to say here? And of course, we still have

Speaker 1: one very specific issue, and this issue is for New servers trying to serve sorry old servers trying to serve the new CSS files before the new CSS files were served So of course that's a very very specific case, but we still have you know a small percentage of uh uh cases where we cannot serve this nicely And potentially this can be only solved in the proxy level. So in the proxy level, we should say that you know every new connection goes to the new server because new is always better. We know that And in that case, we can make sure that we will never get in a in a case where we're going to try to load the file which is not hosted in there

Speaker 1: What I see, we're almost at time where so I'm going to go back to uh the slide so And potentially get a few questions if you have any questions, you know, comments or whatever else you need. And yeah, thanks a lot for staying until here. But the bottom line is we love Django. Django is amazing. I'm with you on that. We have very nice tooling in Django, but when our applications have to grow, we have to make sure that we uh do all those things, you know. Um carefully and we make sure that uh there are no users that are going to to have crosses just because we were lazy or we did not want to plan in advance when we add migrations or other stuff So this is it.

Speaker 1: Uh any questions? No questions? Cool. So thank you very much. Enjoy the conference. If you want to learn more about AppSun and how you can deploy your applications and uh focus only on the code we are outside and uh also if you want to have these nice bracelets pass by our booth and on Friday we have a an after party on the rooftop so it's going to be amazing uh just uh pass by and uh we'll be happy to Uh how drinks and sad about Dango. Enjoy.