Dennis Stritzke - Building Maintainable CRUD in Django

Speaker 1: I flipped the switch.

Speaker 2: Alright.

Speaker 1: Then it's looking good.

Speaker 2: Yeah, welcome on stage. Thanks for coming.

Speaker 1: Thank you I am Dennis and I want to talk to you about maintainable crut in Django. So it was about seven years ago that I first got into doing Django. I just got a new client and they had a rather large Django application and I didn't really know much about what that Django thing is. I did Python work before, but no Django yet. So what did I do? I pulled up the documentation. And in the top it has this sentence Django makes it easier to build better web apps more quickly and with less code.

Speaker 1: That sounds awesome. Especially before I was working with a lot of Go and JavaScript stuff and had to take so many decisions myself and then I read about queries and forms and all of those views and I thought that looks Pretty good. It seems like we can do something with this. And I started to work on that client's application. It was just a typical business application for our purposes here, so at least from the Django side. We had a lot of models in the system. There were list views, detail views, all this stuff that you know. Looking at some more of those views, we had

Speaker 1: also those detail views listing in a somewhat generic way. all of the properties of those models, then having some fancier features for if there were uh image fields referenced to do these kind of things. And obviously they there were other forms that stretched long on the pages. This is one of the shorter forms. And then I got to work adding my first model to the application. Felt good, defined the model. Put in a view, then a form, the template, another detail view, the deletion view, the do you really want to um delete this thing view? And there were a lot of things And that felt

Speaker 1: kind of weird, especially in the application There were a lot of generic classes like these. And look at the top. We have an e-plano generic list view. Eplano being the project and It seems like a list view. So obviously also for my model I inherited from that class which looked somewhat like this So it's I would say pretty easy uh to read, added some templates, and we got going there. But then I understand stood a little bit more and looked at some of the generic code. And this was one of the first

Speaker 1: instances where I actually found the stuff crumbling. And if you look at this object brand here, that was not what I introduced, it was in the code before. This is a generic template, but we have very specific code to show properties of the brand. And obviously we don't want to have that in any of those generic Things. And these kind of references to brands and any of those other dozens of models were spread all throughout the code base. And Then I thought back to that statement from the documentation again. Django makes it easier to build better web apps more quickly and with less code.

Speaker 1: I just spent quite some time duplicating all of those classes, some of those templates, had to build so many forms. and so on and I was really skeptical about that easier because that didn't really felt easier and there was a lot of stuff going on in the generic parts of the application that just felt off. But I didn't really know what that was about. Since then, about seven years have passed. I built that application. This application, applications with way more detailed views and nested list views, and more applications. What I'm just trying to say here is I build a lot of these things, these kind of um B2B back

Speaker 1: office applications where you have a lot of CRUD operations and a lot of permission handling and you really want to Build applications that feel the same and are very consistent to use for your enterprise users. And from that, I think I learned a couple of things how we can Build CRUD that is also maintainable a couple of years later. And to look into this, we are taking a look at a small sample application, which is a library application. And let's take a look at that first. Very shaky table. It's rather boring.

Speaker 1: We have a list of authors. We can look in there. Carlton hasn't written any books yet, as far as I know. But Carson has, and we can navigate to those books, we can edit books, save that, and we could also delete books. Wow, that is actually a CRUD application. What we want to do now is dissect that application and For that, we should look in the code. Because this application is built three times by me. Once in function-based views, then in generic views, in Neapolyton, and then we also have some shared stuff.

Speaker 1: I want to go through all of them with you and then look at some pros and cons of them and see how that is going to affect your application. First of all in the shared folder we have just some static CSS, we have a base template. the uploaded media and our database so that we don't have to um replicate that all of the time. We already have the generic application or generic view based application running and let's do what a Software developer that doesn't really know Django yet, but um does know Python and how web stuff works might do. Because if we start in this application, we see that there is

Speaker 1: a book URL, and that book URL has to be somehow registered in our application, right? In the Django documentation, we read that there's something like URLs. And opening that up, that looks good because there's something that is called book. Let's have a look at this. Wow, a book list view of model book, which what a coincidence has all of the properties that we are also showing here But then we are kind of stuck because there's a list view and we are beginner Django people right now. We don't know what that list view actually is doing and how does that render out anything?

Speaker 1: Let's maybe look into the directory that this file is in. There's other stuff here. There's some templates and oh nice. There's a book list view. We are rendering a headline and if object list wait, where's that object list coming from? I have no idea. But we can seem to iterate that and render things. Let's go back to that view again. Let's look at the base class. Okay, nothing in here. Let's look here. Also nothing. Who has done that game before? Yes, nice. Okay, and then um I think I didn't click where I wanted to click, but then we find somewhere

Speaker 1: the object list in there. And it works. Great in giving us those views because this is pretty nice. It's just two lines of code and we have a uh a list view of books. But I don't really understand what's going on here. Let's move forward to the function-based implementation. Let's stop that. Start the function-based views. Just very briefly, I refresh that page. It's just the exact same application, same templates, or at least the same behavior, but this time implemented through function-based. views. Let's do the same game again. We look first

Speaker 1: in the config directory, look at the URLs, and again it looks very similar to what we've seen. Just before, the only real difference is that in that middle column, if you will, we now reference functions instead of classes. Let's look into that function We have again our book list, but this time there is some creat operation that is pretty easy to use. Then there's some render that somehow gets the request. We immediately link this template, which if we look into that, it's the exact same thing, and there's an object list But this time

Speaker 1: I know where it's coming from because the object list is just the books that we fetched up here. Pretty easy to read. That must mean that I say we always have to use function-based views, right? Yes. Thank you. But actually, I am not so sure about that. Because also we have to just spell out everything all the time and if we then just take a brief peek into um This file, there's quite a bit of repetition in here.

Speaker 1: What's the actual conceptual difference between these two approaches? I would say it's locality of behavior. Has someone of you already read that blog post by Carlson Gross? Two hands. Nice. This is the whole blog post. It's rather short. And there's one actually important or the most important sentence of the whole blog post is right at the beginning. The behavior of a unit of code should be as obvious as possible by looking only at that unit of code. as obvious as possible by looking only at that unit of code.

Speaker 1: That seems like I'm saying again, use function-based views. Because let's look at this. I mean I basically already uh made that argument. I can look at this, I know what's happening here. No real surprises. But let's switch To my slides first. Because I want to have animations on that slide to reel the reveal them one by one, but those are gone. So you see them all at once. The rules of maintainable crud to actually do that and I read them out first. Build a generic crud layer embracing locality of behavior.

Speaker 1: Break locality of behavior to gain coherence. Localize any override of the generic behavior. A little bit much And at once we want uh or I will guide you through all of them by judging the current approaches that we already saw in code against them. First, the generic views and build a generic crud layer. When looking at that code and looking at the book list view, that is Quite generic because there's a list view that Django provides. There's nothing I have to do. And somehow it knows how to get the query set and do all of those things. And

Speaker 1: if I know the interface and the implementation of the list view and all of the other classes, I can do a lot of things within that class to actually actually customize that if I know that interface or dig through all of the code. So it's doing very well on the first part. the generic crud layer, but very poorly on the locality of behavior because it's just nested, I don't know, four or five layers deep at some points of where the actual behavior happens And so even if you are experienced in searching one specific thing, it even can take quite some while or other websites like that one where all of those methods are properly documented spun

Speaker 1: up. Then again, the generic view approach is very good at the second rule Break locality of behavior to gain coherence. If I use those generic views, I'm guaranteed that all of the views function the same Because there's the same code executed, I just plug in my model and everything else um is just being the same. And about localizing any override of the generic behavior is also something that

Speaker 1: generic views is good at, and for that we have to look at the author view. So we are or We are in this view where we have all the author details and list their books And what we're doing is we're using the generic mechanism that is provided by us, by Django to us, and overriding one part, the get context data. To set the published book books to the filtered query set just returning the author's books. What this is doing

Speaker 1: is Taking that specific behavior and puts it right in our code location that we are currently working on. But we are still missing the locality of behavior. And to briefly look at function-based views, we already know they are very good at locality of behavior. We've already seen that. There aren't any generic layer at all because there is nothing generic at it. We just get a request do something and then return something. We can make it into a nice structure, but that is totally up to us. And

Speaker 1: then also to the or especially the third part, everything is local. There's nothing to really overwrite, so it doesn't really Play a role there. But we want to build those very coherent big applications where I don't want to write all that code all of the time. And that's why we are taking a look at Neapolitan Next, which basically takes a little bit different approach to these kind of things. Has someone of you already used Neapolitan before? Wow, I can actually show you something new then. That's nice. Again, same application, same rando.

Speaker 1: You already know the UI. I'm not going to show you that again because it's the exact same thing. Also, the templates are exactly the same thing like we just seen before. But now we're taking the same route as we always have. We're looking to the config, going to the URLs, conf and again we are searching for books This time it looks a little bit different. Not as clear anymore as before because there's not really the books URL listed anywhere. But by just guessing with these two class names, I think we should look at the book CRUDVIO.

Speaker 1: Looking into the book CrudView, there's not much going on. We are inheriting from this ominous CRUDView, which is provided by Neapolitan, saying we want to render books And that we want to have some fields here. This produces the exact same output Let's look into the crowd view. This file is not that short. It's 513 lines of code. It's at least not 3,000 like the admin thing we saw. But this is basically the whole library. And let's take a look at our rules of CRUD

Speaker 1: again. We want to have a generic CRUD layer which we have to open up the view again you see here is provided by the CRUD view The locality of behavior, I argue, is in that file because there's everything in here. There's no other base classes you have to look at. There's everything can be understood in here. For example, if we look at the List view, you can see that there's some query set stuff going on, filter sets, pagination, and in the end we are rendering

Speaker 1: something to the response. You can really easily read that and understand what is happening. So we have the generic crowd layer by embracing locality of behavior. We are breaking that locality of behavior by subclassing this class to gain the coherence in our application so that just everything works the same. And in this approach, the localization of the generic behavior looks like the following. We just have to scroll up here in our AuthorCrowd view. Same thing again, authors, some fields. And again, we are overriding the getContext data. And if the

Speaker 1: role is the detail role, then we are inserting some published books here. We have taken the generic mechanism and made it overridable in here. One difference is we can do this with everything in that file, and if you need some kind of Um customization, you look in here, you search it through, and you are pretty easy to actually find it. That leads to the question, Neapolitan for Everything Maybe not. Maybe it's time for your specific thing, because if you just use Neapolitan

Speaker 1: It's just those crud things. You have lists, updates, deletes. And I don't know about the applications you are working on every day, but probably you need a little bit more than that. And I would urge you to build or extend this generic CRUD layer to include those things that are generic for your application. But specific to your application. Meaning for example, if we always show list tables in the exact same way, we might provide some base facility in though in this class that provides the base table, maybe renders

Speaker 1: it on uh the basis of those fields we had. And then if we just want to have a default table in our application we just override that class and everything is good Let's look at a few of those things. This is one of my applications and Most of these, or basically any application, has some navigation bar, right? And many of those business type applications, there's a lot of different um group membership permission things that also uh depend on

Speaker 1: which items in those navigation bars are actually shown or not shown and that's why in that project there is a back office view in this generic layer that always needs some URL pattern and URL name things and more importantly all of those different navigation titles on and sections and which icon to show. And I know that if I use that base class, just put some translatable strings in there, it will show up in here. And I also know that the permission handling that's also part of this kind of construct is taken into consideration here.

Speaker 1: And in my crowd view, it's basically what we just saw. It's extending or using that mix-in, listing the model, that's it. And most of the time there's quite a few things that accumulate in these kind of projects and that's why in at least my projects it's mostly a back office crut view that uses a few mix-ins like permission handling the back office view you just saw table view and most importantly the Neapolitan crud view in here and then there's all the kind of stuff happening that is generic to the application but

Speaker 1: or specific to that application but generic to every view in that application and it's easy to just override any of that behavior by overriding a method or a property here and have that customized Therefore, build your CRUD framework, be intentional about that, and communicate clearly what you are doing with this. Let's circle back to the application I've talked about in the very beginning, that e-plano thing with those very long forms. There was some experienced developer before that built a crud layer a generic crud layer like this. But then came other developers that were maybe not as experienced or not as communicated as much, and just

Speaker 1: put those very specific things in all of the generic parts of the application. And that's why I want to extend my rules of maintainable crud with that part. Anchor your CRUD layer in clearly communicated human intentions. I want to know and when I'm using your base layer, what the intention behind that is, because all of the code in there is there to make our application do something specific in a very consistent way. And I want to know why I should use that and not just Invent my own thing. Thank you

Speaker 1: Any questions to that?

Speaker 2: Amazing talk. This is uh like the first successful uh attempt I've seen of uh finding a uh a a really good balance and definition between uh those two uh world of generic and function based views. Thank you for that.

Speaker 1: Thank you.

Speaker 2: Um yeah questions

Speaker 3: Uh thank you for the talk. Uh actually uh I'm wondering uh what do you really recommend? uh to approach with this is based on the uh use case right using either uh I would say function based or uh class based or uh the new approach you came up with. What's your recommendation here?

Speaker 1: I think it really depends on what kind of application you want to build. So if it's A big application with just dozens and dozens of views that really have to behave the same. I think you have to just have some generic base layer there. But if on the other hand it's just like you are building that one endpoint that has just one view you are going to show, maybe just throw in the one template view or the function-based view. I think it really depends on on the complexity of the full solution or where you want to land on that. Spectrum because if you're going full back office crowd view mix in thing, um there has to um So a lot of care has to be taken that this is actually a good abstraction to use and not really um

Speaker 1: bogging down your project So no idea which one to use, totally depends.

Speaker 2: So what happens if you introduce uh a mix in in in this Does it solve anything or does it create do you end up with the same Django of mix-ins?

Speaker 1: Um it depends a little bit on what kind of mix-in that is. So before I was doing stuff like I was shown here, I was also just implementing a mix-in and then um introducing that in every class And now I think the main decision is is that something generic for the whole application? Then it just goes into for example that back office crowd view and I have one um location where I have to put that in or is that just specific to one, two, three views and then I uh put that into some of that subclasses um in the inheritance tree. And so some classes become quite cluttered in the uh override order, but most of them don't

Speaker 1: Is

Speaker 2: the difference if like who owns the mis uh mix-in in terms of uh locality of behavior? Is it like more local if it's my project mix-in than the packages. Can you see it like that?

Speaker 1: That is a good question. Um not act entirely sure. I think it's uh very easy that you can grasp it at once. So um for example with the navigation mix-in stuff, I wanted to have that in a specific mix-in because I didn't want to read navigation code when dealing with Other filtering logic um in there. But if you need to know anything about the navigation, you look into that mix in and can read it top to bottom and know what's going on. And I think that's way more important than who actually owns that code. And

Speaker 2: any other questions?

Speaker 1: There was one more?

Speaker 2: Yeah, we have time.

Speaker 4: Thank you. So um since the topic is specific about crude framework. So um could you could you please give us an example? How would you handle the atomic transactions and for what um for a specific operation either update or create or observe.

Speaker 1: Let's maybe just uh take one of those examples if I um would want to Change anything in the author um creation logic and would maybe override the process form method, which in the end gets my form, I can do Whatever, I don't know, uh something in here, put then my transaction atomic on this part Business logic here. And I think it totally depends on where that logic sits, but in the end it's always

Speaker 1: Overriding some part, putting your transaction boundaries there, and then doing whatever you need to do. Either calling out to something or just doing it directly in here.

Speaker 2: Oh, there's a question here.

Speaker 5: the name of a design pattern or a library and if it's a library did you make it

Speaker 1: uh it's the name of a library Carlson Gibson made it Um when I stumbled across these or I stumbled across those problems over and over again, I started building something, then researched into the topic more and then found the library and then That library was way better than what I build, of course. And I think he uh named that library because uh there was uh another library that was called um Django Vanilla Views And that library is more than vanilla. And what's more than vanilla ice cream? It's Neapolitan, the assortment of vanilla, chocolate, and sorry. That's the name story behind that.

Speaker 6: So one more question. So let's say let's say you have your own base class with all the mixings and so on and you have I don't know 20-50 models using that. And then you need to introduce some new mix in into the how do you handle like the risk between breaking 50 existing models that use the same hierarchy or another new feature. So

Speaker 1: in the type of application I have shown here and I mostly work on, I really want to do that in that generic layer and have the risk of maybe breaking all those fifty places that I'm using that and If I'm using that for 50 models, it's most likely not 50 views, but maybe, I don't know, two, three, four hundred views. And that's pretty realistic. And what I typically do is that I have this generic layer either in a real other package or at least in some separated code location so that I can test that. independently so that the generic thing is actually doing its generic thing um independent of any specific model and then roll it out to the full application to then only deal with

Speaker 1: the stuff that's specific to um other models breaking something there. And I would argue that's a hassle we want to have in uh For the sake that the application looks the same, feels the same and behaves the same. But it's a lot of work.

Speaker 2: questions um well that was a very cool talk so thanks a lot for that um and