Optimizing Django deployments with a Continuous Observability Strategy.

Um so name is G Guillaume, obviously French, as you can hear, and I work for UpSign Platform Message. So this is where I live. Any idea where it is? It's in the US obviously. Houston, thank you. But sometimes it also looks like a lot of my application. I've got all those users coming in in a single bottleneck, and I'm not sure what to do with them. So Thanks to Pendy and showing all the testing. That 10-minute talk is gonna more focus on how you can test performance and how you can basically build testing regression on them. So the Houston answer to that problem has been for basically 50 years, just add more lanes

Uh I live on the catty freeway which has 28 lanes in both ways, uh which is just a mess and it doesn't solve issues Like you should not throw metal at your performance issues. Adding servers may be fine at first, but you need to focus on actually optimizing your application and making sure those lanes are a lot more fluid. The main two reasons to not throw hardware at the problem is basically obviously cost because you're paying for those resources, but also all the carbon footprint that you're generating through those resources. So our goal is to make our apps the most efficient as possible to make sure that the user experience is good, our costs are low, and our environmental impact is actually

also lower. So, two different issues. Can we detect and fix issues in production quickly? Sometime a patch, a new feature, a bug fix is gonna be rolled out, and you're gonna see an unexpected spike. In traffic and maybe lowered performance. So how can we detect those quickly and fix those? The second one is that once we fix them How do we actually create a performance budget for our whole application and then set up some testing to be able to make sure that any new release of that application actually follows the budget and is within the budget that we've set up? So yeah, if we look at Houston, it's such a giant mess. And that could be your application. You've got workers, queues, you've got your main backend, maybe you've got different clients coming in.

Eight semesters. You don't actually know what creates all those issues. You can just see on the world map that there are a lot of problems. And that's what we call in application performance monitoring tools basically traces. You can track transaction and you see that That person coming from the west side to downtown as a problem. It's the same as people trying to log in or trying to check out as a problem. But you're not sure where in that process actually the problem is. So you usually go to a second step, extended traces. Okay, on that checkout process, where is actually the real button neck? Which step? Is that the payment information, shipping information? What is actually causing that issue?

And that's what of most the APM tools around here actually give you. You can see where the problem is. But as a developer, what you want is not just detect that problem, but be able to fix it. So how do you get into this? Well the answer is usually profiling. It's actually getting to the root cause of what's causing that issue. So basically on that specific point, the real issue is construction. And as we know in Houston it's gonna take like 10 years to get done. So for the next 10 years it's gonna be a problem. Unfortunately, I can't fix traffic issues in Houston, I wish, but I can fix issues in my application. So the way I'm gonna do that is actually really Doing what we call continuous observability. Like continuous integration and continuous deployment, you can also observe continuously how your application is doing.

And that on all the different stages During development, making sure that every time you think your feature is done, having some tests in place and be able to profile what you've done and see the result and be able to compare with the previous version Then on staging and testing, be able to run those automated tests like Pandy did to make sure everything is fine and within the budget you fixed. And then on production, making sure everything is running smoothly because even with all those tests, sometimes you've missed something and you've deployed something that decreased performance. So usually for profiling, there are a bunch of really complex tools. I work for platform message and blackfire. io, we provide that APM and profiling tool

for a lot of different runtimes. Python, node, PHP, Go, et cetera, where you can dig into what are the different performance statistics of all your applications. So in that screen, I'm not gonna zoom, but you can see all the transactions and what actually Where your users and your CPUs are spending time on your app. But what's more interesting, and that's a Django app, is actually once you saw that this specific transaction has an issue, actually be able to get into the code that created that issue. And that's usually what you're missing with those tools. What's important there is that maybe you're gonna find an obvious blocker. A query to a database gonna Takes so much time. But sometimes it's a bit more tricky than this. In that example, I know it's a bit small, but we see that basically some function

that looks fast. I actually called a lot of times, like 25 times, 200 times. 200 times is still okay. I had a client in production with an e-commerce website. And they didn't know that on every product page they were calling the thumbnail 10,000 times. That was stupid, that was a mistake, but they didn't realize that because the thumbnail generation was quick enough, it's just that they were generating too many times. So with that, you're able to isolate exactly which part of your code is actually causing those struggles. And you're able to fix it. And then when you fixed it, what's interesting is actually seeing a comparison with the previous version and saying, oh, now that I've fixed that n plus one issue on my database. I can see that the process is way more streamlined.

I take a few more milliseconds to gather the result because my query is a bit bigger. But I do only one queries instead of 200. So it's better overall. And you can see the difference in performance on the memory, on the network. Especially useful when you're doing external API calls to be able also to qualify how slow those calls can be on the memory side, network side, everything. Once you fix those, as I said, it's really important to implement some basic tests. So defining some rules, it can be Loading times, time to first byte, whatever that is, uh the memory used by some pages, the number of queries, the number of models

they actually hydrate from the database. That could be anything. And writing those YAMLs, actually listing those different rules and assertion. So that then when you're running your CI, whatever that could be GitHub, Action, or Platform Message, whatever that is You're gonna get a report and a build status if you met that performance budget. And we see in that example that okay, the memory was good, but we've made too many requests, the roll time is too high So instead of pushing that directly to my production instance, I'm gonna rework this and making sure my performance budget is fine with it. Maybe change the performance budget for some reason. That's even easier if you're using a platform as a service to do that because every time you're gonna push something, you're gonna get a whole new isolated environment to test that feature, whether that's infrastructure change, like

runtime version, new databases, or just code and feature changes, you're able to test those performance issues on your new environment directly without impacting production or anything. So great to really automate and ease the development workflow. We've just launched a new product called WebSun, which is a platform service dedicated to people creating complex applications with a lot of different services with that. uh caching system, cues, whatever that could be in a lot of different runtime. We're looking for early testers, so you get credits obviously, and then just a quick chat with my solutions teams to be able to actually give us feedback on what we're creating. But we've made that platform

so really Python developers, Go developers, node developers feel more at home than anywhere else. So If you want to register for early access, the only thing that you can get with that is just one single email, no spam or anything. If you don't want to follow up, that's fine. No monetary commitment, feel free to sign up. Thank you. And have a good coffee, obviously.