Run your tests against Django's main!

Speaker 1: Hello everyone. Welcome to the talk. I'm Sage. I'm a developer at Worgebox. And I work on something called Wagdale CMS, which is an open source Django based CMS. Uh previously I also contributed to Django and Whiteel through the Google Summer Code program as both a contributor and network. And most recently I joined the Django. space program where we help onboard new contributors to the Django project and also internal packages as well So we often hear people

Speaker 1: say that Django is stable and in fact Django itself makes a commitment to stability as stated in the documentation. Um our aim is to provide a modern, dependable web framework of the highest quality that encourages best practices in all projects that use it. So how do we keep this table? Well, first one, the most obvious, is through tests. We have lots of tests. We have almost 18,000 tests as of the upcoming quite quote release. And the tests are run against all the gate-based backends supported by Django.

Speaker 1: So that's Postgres, equalite, MySQL, Orientb, and Oracle. And also only Arton versions supported for each Django version, also across different operating systems as well And also to minimize disruption, um so Django 's release at cadence is 0. 0, 0. 1, and a 0. 2 LPS, and then back to zero So for example, if a feature or a setting is deprecated in Django 5. 1, it will be removed in Django 6. 0 Okay, so this is the release process for Django. I don't know if you can see the whole way back there, but uh yeah, so for a given version,

Speaker 1: development happens on the main branch. And that's over a period of about eight months. And then we fork the main branch to uh branch calls cable slash a dot b dot s depending on the version which marks the feature freeze or the alpha release for a. b. So for example for Um character five point two when the alpha is released, uh we have the stable flash five point two point x. And then uh during the alpha period, new features will not be merged to um sorry, new features will only be merged to the main branch and it will not be

Speaker 1: backported to the stable AEPS French. So early development of the upcoming five point or some point well currently it's 5. 2 point Zero or just like quite two. Um it's currently in Lita? Alpha Yes. But the building for the upcoming Django six point z is happening in parallel. So new features will be added to six point zero. And then after about a month in the Alpha release, uh we have the ETA um release, which uh Well contain any bug

Speaker 1: pieces including one or bug pieces. So if you have if there's a bug injector that's been around for years, it can still go in during the alpha 2Bs and create After the key chart period, only critical boxes can be backported and and then after about a month The IC release is out, which marks the translation string freeze. So if there is any code that changes the translatable strings , it will not be backported to the um 5. 2 verge. And then after a couple weeks during the Odyssey release, the final release is uh So yeah, this is an example for 5.

Speaker 1: 0 and 5. 1. The 5. 0 release was out in December, but the alpha release was out in September. And then eight months is 5. 0 alpha. We have 5. 1 alpha in May. So yeah, and it goes on for uh So when is the best time to cache bugs? Does anyone have any idea? Any suggestion?

Speaker 2: As early as possible

Speaker 1: Right. Sorry. So I it's right before the PR is merched in height. So when it's still in a PR. If possible, we should fashion it and make sure it didn't get too main in the first place. But it's hard to keep up with all the VRs that come into Django and you don't know which ones actually affect your project. So unless you're a Django Hello, which in which case it's literally your job. So yeah, what you can do instead is you run your tests against Django's main branch This is an example for GitHub actions. So it's pretty simple. You just need to replace the installed Django version of the CI

Speaker 1: with the main branch of the P3 probe for Django because Git also allows you to install from Git directly using the syntax Git plus HTTPS and just flip a link to Django 3 Pro and use an app sign to note which uh branch you want to install And then just run your tests as usual. And if you maintain a tracker package, you might already do this. maybe with different Django versions. So I just want to remind you to make sure to also test against the main version, not just the release versions. So how does that

Speaker 1: help? Well, sometimes things do great. So whether intentionally or not For example, in White Tale, we test against Django's main branch, as well as these table branches as well. And just a few hours ago actually, I'll just give it to a screenshot. When we ran it against Dango 's latest main branch There's an import error saying that the sub -query 10 cost training class is missing from Django DP models SQL where And yeah, sure enough, if you look at Django 3 code popular today, the subquery constraint

Speaker 1: class has been removed. And this is an internal class, so it's undocumented. That's why it in this case Django didn't go through the fabrication process over two releases. In this case, the class was removed because it was more of a workaround to how a feature should have been implemented. Since the workaround is no longer needed, they removed it So how does that help you? Well, the good thing about running our test to games Django's main branch is that it makes it easy to find which commit in Django that broke your code. And in this case, uh well WyTol doesn't really use the code, it's just

Speaker 1: We use it to detect when the user has to use the subquery. And that's not something we support because this is a search backend that's connected to Elasticsearch. That's why for complex queries we do not support it. And in this case we use that class to check the uh resulting SQL query. So I guess it's also a lesson one to why you shouldn't use Django's internal APIs. But sometimes you do have good reasons to do so. But so if you do, just make sure you run your tests against Django's main branch. And so how does this help Django itself? Well, in some cases your tests might fail because it's a legitimate bug

Speaker 1: in Django itself. It's a regression. And Django's tests apparently don't have your use case. So if you're if you're not familiar, this is Django 's issue tracker called Track that you can visit on. So yeah, this is an example of issues reported by myself and my colleague Matthew. um which uh I think most if not all of them we caught because uh we run the test it for my double against Jacob. For example, last month, just two days before Django 5. 2 Alpha release, we had a bug in Django.

Speaker 1: The bug was that if you use multicable inheritance And then you call the full clean method on a child instance. It executes a database query when it shouldn't, when it shouldn't. And this was the regression caused by the implementation of the composted primary keys introduced in Django 5. 2. So I submitted a TKP Django along with a S case in the issue So yeah, and then the issue was probably fixed the same day, or maybe it was the volume day, I forgot. But it wasn't me who fixed it. I just reported it But uh I still get the credit in the commit message.

Speaker 1: So yeah, wouldn't it be nice to have your name and uh commit message in the January code?

Speaker 2: Yeah.

Speaker 1: Yeah, so yeah, run your test against Django's plane and report any QC following, because otherwise Django will not know about it. So yeah, that's it for me. Thank you.

Speaker 3: You didn't show whether you have a white tail if the test jammed on the white L C ion fails, whether it's only allowed to fail, you're just checking it once in a while

Speaker 1: Right, so let's see. We do not know it's fail, but I think you'll have pass this um continuing on error option that you can add for example if you just want it to be specific to the main branch test you can set for example if it is any flag I just said experimental and then Yeah, continue on error experimental. But I think um it has this annoying issue where if one of your CIM frames

Speaker 3: Okay, so it it reports it as an error and it doesn't allow much either

Speaker 1: Oh yeah, we need a we still allow it to get first, yeah.

Speaker 3: So is it your job to chase any of

Speaker 1: Yeah, exactly. Yes, yes.

Speaker 3: Okay, any more questions?

Speaker 2: Thank you. Would you recommend this for all packages and all Django sites or apps or like Only some cases. Is there a limited?

Speaker 1: I I don't see a reason why you shouldn't do this. I mean it's pretty easy easy if you already have your test and you already test against well you you have Facebook Jenga if it's a Django project right or a Java package and if you if it's a Django package then you really should test at least against the released Django versions But even better, if you already have the same format, it should be fairly simple to just add another job or testing against money I think so, yeah

Speaker 2: Anymore? Now we're back on thank you and save