Yak-shaving to Where the Puck is Going to Be.

Uh

Speaker 1: hello, I'm Carlton Gibson. This is me. No, that's me. Um I'm uh Carlton Gibson on GitHub. I'm on Fostered on uh Carlton give him Carlton there. I've got a website, I do a blog, that kind of thing. There's an RSS feed, you can follow along and view that. What else? Oh, I've got a podcast. Um with my friend Will, wherever Will is, I'm sure he's here somewhere. Um we have a podcast, it's pretty old school, we have guests from around the community and we uh we chat about Django. Um check that out if you haven't had a listen Darren, thank you for the introduction. As you um may know, if you don't know me, well between 2018 and the end of March this year, a couple of months ago, I was one of the Django Fellows Um the fellows are contracted by the DSF to the Django Django

Speaker 1: Software Foundation to do the day-to-day maintenance on Django. They do ticket triage, pull requests, review, releases, security updates, all of that stuff. It's the stuff that makes sure Django keeps going. Now, I've just stepped down, but that doesn't mean you get out of it. Go and sponsor the DSF. You, or better yet, you, your company, make a small donation that helps To keep Django going. You're making investment in the sustainability of the framework you built your business upon. Okay, so today's talk is about yak shaving. Specifically, um yak shaving to where the puck is going to be. Now I'm not 100% sure about the title.

Speaker 1: I wasn't due to give a talk, and then someone dropped out and I had to pull something out at short notice. Um Mark was like, well, could you do a talk? And I know, well, okay, uh, well I could talk about what I've been doing and what I've been doing is Yak shaving. Um There are lots of really interesting threads that have come together in Django over the last few releases, particularly things about D B constraints, moving default values into the D B, maybe even getting um D B level cascades and other things which all of which are levelling up the ORM every single release. There's all the exciting stuff about async. And there's the forms changes that David Smith has been working on the last few releases that he's going to talk to you about tomorrow. I'm not going to talk about any of those things today. So when I say to where the puck is going, I'm just talking about the very small bit of the I've been working on.

Speaker 1: Okay. So I'm meant to be writing my way back, aren't we all? In part because of life, but in part because I I step in part because of life, but in part I stepped down in order to get back to building stuff with Django rather than working on Django. I think we're at really interesting times from the framework. I can honestly say that I haven't been this excited about the framework and where it's going since the days of the DRF Kickstarter, if you remember that. What I want to do today is quickly show you a couple of side products or projects or byproducts that have come out of the work that I'm doing. I think they point to where the Django's going, and so I just want to share them with you So, if I want to talk about where the puck's going to be, I could just

Speaker 1: put up a slide that says this. Then we could all go down the bar and get a drink, end of talk, but thank you very much. HGMX is just wonderful. It's just wonderful. I've been using it alongside Alpine JS and Tailwind CSS and it's it's It's just lovely. I said that already. It's lovely. They give a really nice environment. The headline feature is lower complexity. The last I don't know what decade Front-end development have driven off a cliff of complexity and HTMX and the related technologies give it give a much simpler, let's say, almost old school approach that brings the fun back. For me, it makes doing things single handed feasible in a way that hasn't really it hasn't felt feasible for a long, long time

Speaker 1: So all three of those texts that I mentioned, HTMX, AlpineJS, Tailwind CSS, they leverage something that's called locality of behaviour. This is the idea that you can understand the source code that you're working on by only looking at the small portion that you're actually working on. That everything you need to see is there right in front of you. an HT exec uh a an HTMX example will be good here. So with H we've got an HTMX example on the left there and a jQuery example on the right. On the HTMX, we've got this HX get attribute that says when you click on this button, it will go off and make a request and it will replace that button with whatever HTML comes back. On the right hand side we've got the um the jQuery example which is the the two there. So everything there there on the left is in the one

Speaker 1: file. And once you know what the HX attributes do, you don't need to look anywhere else to understand the code in its entirety. But on the right, the jQuery example, let's assume it's exactly the same behavior. You have to have your behaviour split over two files, and you need to have both of them open to see exactly what your code is going to do. Right? So, as ever, with these things, it's not the only factor, but the first example has better locality of behaviour than the second. I have to say, I absolutely love this concept. Years ago, and when I say years ago, I mean still, I would stick lots of code in URL comms. Particularly with template view. Template view, simple generic view that Django Django provides. It's got an extra context attribute which you can pass as key or key or as keyword args in the URL conf.

Speaker 1: And so this can get out of hand if you do it too much. But in really basic cases, it can be a lot clearer than having a nothing template view subclass in a separate views. py file, which you then have to open, and then ha when you're reading the URL conf. Now disclaimer, I already said it can get out of hack. If you're in your URLs. py file and you can't find your URL patterns, then perhaps it's time to refactor. There are similar examples. What about when you've got a three-line form class in a separate forms. py file rather than having it next to the view that's using that form? Right? What about a four-line custom manager in a managers. py file rather than being next to the model that the manager is actually related to? Okay? And so on. Right? Like all of

Speaker 1: like all of these kind of considerations. It's a trade-off But especially when you're rapidly iterating on new code that's still in flux, emphasizing locality of behavior can really help you go faster. So Django 's generic class-based views. I've got three dots here in my speaker notes, I don't really know what to say Um a class is mainly a namespace. And a a class space view should be a simple thing. It should look something like this. We just import a couple of things there and we define a view with a get handler that returns a response. So it's a kind of hello world class-based view in Django Compared to the function-based view, well it's nested in a level and you've got the self-parameter, but the

Speaker 1: trade-off is that you've got a namespace that you can break your view implementation up into. Okay. So an object detail view would look something like this one. There's three bits of logic going on here. Each of them is itself moderately complex, so we pull them out into separate methods, and then we get our handler and our get handler then just calls them in turn. Our get handler remains easy to read. Get the object. Get the context. Render to response. It's clear as day, right? But, crucially, if we need to look at the implementation of any of the sub-methods, they're right there on the page in our editor. We just scroll up. But that's not what we have with the generic class-based views that ship with Django. Instead, we have this.

Speaker 1: Now I don't if you don't know what classy class-based views is, it's a website that provides a deep that provides detailed descriptions with full methods and attributes for each of Django's generic class-based views. I go so far to say that it's an essential reference Just as I always have the Django docs open, I always have classy class-based views open when I'm working on the generic views. I recommend that you do too. The issue is that the inheritance structure of Django's generic views is so complex that they've lost any notion of locality of behavior. Okay. So if we look on classy class-based views at the implementation of Django 's generic detail view get method, it's almost identical to the one we had. Passing the self uh object self.

Speaker 1: object into the context getContextData method, that's a bit of a nonsense because getContextData overrides that itself. It will set that if self. object is not none, but you can't see that just by looking at the implementation in the same class. Because detail view extends extend extends, sorry, single object mean And context mixing, both of which live in separate files, both of which define get context data, and both of which you'll need to look at to if you want to understand exactly what is happening in that call. Okay? GetObject comes from single object mixing. Render to response is in template response mixing. So if I want to understand detail view. get totally, I'm basically looking at having five files open. For three lines of code.

Speaker 1: It's the exact opposite of locality of behavior. Now I'm not the first person to say this. Looking at this a decade ago in 2013, Tom Christie put out a package called That Django Vanilla Views. It's a much simpler take on the class-based views on class-based views for Django. It has none of the mix-ins that characterize Django's views, and it eschews calls to super methods, where it's often just a single line. It might not be dry, but it's got much better locality of behavior if instead of calling the super method, you copy that one line into the subclass so that the whole implementation is there in front of you. I don't want to have to open a second file to go and see one line of behavior.

Speaker 1: Again, there's a balance here But I think the tendency to make we have a tendency to make a false idol of this dry principle. You need that you add enough hooks in the name of being dry and your be call your code becomes unreadable. Locality of behavior is the antidote to that. Okay, so I love vanilla views. I've had fun with them for years, but when I sat down in April, they weren't quite what I needed I wanted a zinc a single view class for my CRUD methods when I'm writing a model. I'd I'll define a get query set method, say, and and that will take the current user and it will restrict the query set to the current user's objects and I don't want to have to put that in a mix in in order to use it in multiple view classes. Uh list view, detail view, and so on. I wanted generic oblict object templates so that when I get my model I can put it on the page immediately

Speaker 1: I want those to just be provided for me. I wanted integration with Django filter, I wanted a few other things. So what's a little bit more than vanilla? Well, Neapolitan Neapolitan is my take on quick crud views for Django. Let me show it to you. I've got a Django model. Bookmark model there, title note, whether it's a favorite or not, it's got a URL. So then I want to get it on the page Um so all I do here is I import the Neapolitan CRUD view, I create a bookmark view for it, I tell it what model it is, I tell it what fields I want to use it to display in the template, and I say I want to be able to filter on whether it's a favorite or not. Okay

Speaker 1: , that's it. I add it to the URL patterns, boot up my run server, and it's on the page with a default penplay Neapolitan's CrudView provides the standard list, detail, create, edit, and delete views for a model, as well as all the usual hooks you'll need to be able to customize any part of it. Neapolitan provides base templates and reusable template tags to make getting your models on the page as easy as possible. Where you take your app after that is up to you, but Neapolitan will get you started. You'll notice that that's in URLs. py. You start customizing individual handlers, individual templates, overriding methods like get form, get query set, get context data, everything you used to, right? It's gonna get a bit much and you're gonna have to move it out of there into a views.

Speaker 1: py. The key point for me though is that your code lives in a single class It's all right in front of you. Maybe get to the point where you break it out. Just as it with Alpine JS, you might get to the point where you move into a separate your JavaScript into a separate module. But in general, you'll get an awful long way before that needs to happen. Okay? And Crudview itself is just a single class. You can read it top to bottom in the time it takes to drink a single cup of coffee. Okay? If you do want to see the base implementation, you only have to look into a single file. So that's Neapolitan. I recommend it to you. What's on the road map? Now now I've got to finish the docs for it. It was meant to be a stealth project. Then Jeff Triplett saw it on GitHub and put it in Django newsletter and though, well, I better tell everyone about it

Speaker 1: then. Um I'm gonna have to chat with Danny Elliott, because I think you could just read um Neapolitan source code. I think it's just this one file, and I want to tell you to go and read it, so we'll have a chat about whether that's morally acceptable or not. Um I want to improve the templates that come that the that come with it. Um I want to um get the filtering working in the default templates, get the pagination working in the templates. I want to ship a bootstrap version just to show how easy, because I'm using Tailwind CSS and Neapolitan's default templates use Tailwind. When CSS, but you could use bootstrap just as easily. So I want to put bootstrap in there. What else? I want to do parent objects. So here's the pattern that comes up a lot. You've got a bookmark, but then you want a foreign key to the parent user. And that's such a common pattern that I want to make that just just doable.

Speaker 1: You declare what the related field is, um, and then the list view needs a parameter for it because it needs to know which user to show. And the create view needs the user as well, so it knows what when it's creating the object which user to attach it to. I want to do partial updates, so particularly then with um Uh what's it called? HTMX. You cut you don't edit all the fields at once. And it's a really common pattern to say, look, I just want to s to to To update this bookmark whether it's a favorite or not, not do the whole form. Django doesn't really do that out of the fields, but it's out of the box, but it's really quite simple. I want to do that and then I want to book do um extra actions. Um Say for instance you've got an attachment view and you want an authenticated download. Well you've already got the the user filtering and the permissions and everything to get the user's di attachment.

Speaker 1: The authenticated download is just three lines and an extra An extra action. Okay. A lot of this is familiar as well from DRF. Um, so anyway, that's Neoparton, that's one thing. That's what I've been working on. I'm having massive fun with it, and I really recommend that you go and check it out. Let me just have a sip of water. I'll talk about one more thing. In the same vein, I've been working on a thing, a package called Django template partials. I've got the PyPy PyPI package, I've got the GitHub repo, but as I speak, it's not quite available yet. It will be another week or so because it's in my Project that I'm building. This is another I idea from HTMX. You've got a full template, but then when we do the HTMX update, we only want to get a small part of that back.

Speaker 1: Now with the DTL, um You can do that already, but you have to move that small fragment out into a separate include file or into a template tag, which is a little bit heavyweight. We it's much better if we have, again, locality of behavior so that the template partial lives in the same in the same file. And Django template partials is my take on that, that lets you define these inline partials and then reference them by name later on. Okay, so first of all you have to install it and then you load the partials tag and at the top of your template you've got start partial and end partial and you give it a name, great bit, and it's got some great reusable content. Um then later on in my main block somewhere I've got a bit before it. I just have a partial tag, but I call it by name and it it will appear

Speaker 1: nice and all brilliant. We can call it by name later on and use it But the thing that I that makes it kind of nice that be just beyond that is that it's got it's integrated with the template loader as well. There's already a package that does a similar thing to this called Django render block. That has a function called render block to string, which lets you pass template name and a block name, and then it will render just that block from the template. Right? That's great for m but for me it didn't do two things that I wanted. I want to reuse partials at multiple points in the page for when a section of HTML appears more than once, and you can't do that with blocks. Um but I also wanted it to integrate with the template loader so that you don't have to change the the structure of your view. Render blocks render block to string, you actually need to adjust the view logic to call the render block to string, and then you return an HTTP response instead of returning a template response just with the template name and the context as you normally would.

Speaker 1: Um with Django template part is you define some um template loaders. Now this is just some s some lines that you put in a setting file. I want to have a nice little function that could wrap loaders that I just haven't quite written yet But Django's default loaders are these first two. It loads it from the file system and it loads from app directories. And then we tend to we wrap that now with the cache template loader so that we're not pulling templates off the files. system every single time they're loaded and then um Django template partials loader wraps the cache loader. Okay And then with that in place, you can literally just in your view change the template name that you you have, and all you do is you uh you attach the little hashtag bit on the end, which is the name of the partial And you return your template response just as you always have.

Speaker 1: None of your other view logic changes. Under the hood, the partials loader checks. Is any of the wrap loaders have the template? And then if you pass the template the partial name, whether that's det um defined and it will do a template not found if not, just like normal. The point is that it's as transparent to the view layer as it possibly can be. You're still using the standard flow and you're still returning um the template response. So that's the other thing that I've been working on. These are my my two little side projects that have come out of yak shaving to where I think the thing's gonna be. One little thing just before I finish. Should we add this to Django? Well I'm doing both of these as third party packages. Just one moment. I've spent the last five years telling people that they have to create a third-party package if they want to add a feature to Django.

Speaker 1: Put it in a third-party package, see if there's community interest, then maybe it gets merged to the core if there is. Of course, as Floren will tell you, maybe it doesn't, right? Just Lives forever. For me, I think having a story about these template fragments or inline partials, however you want to call them, would be a great addition to the Django template language It's something that's come up several times over the years, so there's clearly some demand, and with HTMX being so popular, I think it would be a good time to have this in -core now. Now maybe what I've done here isn't the first version, right? Isn't the final version. Maybe there are other approaches that are better. I'd like to see us do something. So I'm throwing it up as a third-party package so we can experiment, see if there are any major blockers, experiment with the API, etc. All the rest of it. I think Neapolitan

Speaker 1: is likely not a good fit for Django Core itself. I'm having great fun building with it and I think you should have go too. The core API is pretty stable already, as we knew it would be, because it's just the same patterns as the class-based views, but just squashed up together. Once I've worked out the details of the bits I'm working on, I expect it to be stable, you know, kind of indefinitely. But it's quite opinionated, depends on Django filter for one, so I don't think it's a good fit for Django itself. Um Yeah, so where I would like template partials or something like them to make it into the DTL, Neapolitan's gonna be much happier just by itself. But that's it, that's all I've got to say. I'm around for the rest of the conference if you'd like to chat.

Speaker 2: Thank you so much, Carton. Alright, so if anybody has questions for Carlton, we have a mic on each side of stage and uh if you can make the microphone.

Speaker 1: But not from flock.

Speaker 3: I'm sure you're going to chase me out of the conference now. But um do you want to rewrite admin with crowd reuse?

Speaker 1: Sorry, d sorry, do I want to

Speaker 3: Do you want to rewrite the admin? Another new one

Speaker 1: I'm happy if a brave young soul wants to take on such a foolish endeavor. I have no interest in rewriting the admin with crut um with Neapolitan Crudview. Uh in principle you could get something uh quick going when once the filtering pagination is it starts to look a bit like the admin but The the admin has so many features and such depth and such established it no, there is no there is no admin three. It's not happening

Speaker 4: It might be a a question that's a little close, but do you imagine that there could be something like uh Napolin uh Napoletinian DRF?

Speaker 1: Well, uh I'd go the other way actually. So um we've spent the last decade being um API first. I think we need to be hypermedia first and API as well. So the one thing I didn't put my roadmap because it's kind of secret but you've blown it. Is um that I'm gonna I will add handlers to the list and detail view to handle post and whatnot for um uh uh uh JSON handling basically 'cause that's the only one it really cares about. I'm looking forward to the Google Summer Pro Code project this summer which should add um j uh content aware JSON passing to the request object at which point it would be quite trivial to to add to Neapolitan. Uh I need to work on the API there, which is why I hadn't been talking about because I'm not ex I've got ideas, but I'm not exactly sure how that's gonna look.

Speaker 1: I can't quite hear you.

Speaker 5: Thank you, Carlton. In your URLs. py you showed a pattern where one starts building something up in there and then When it's time for it to break out, it gets moved into other places. Now, would that be a good approach, for example, for building the application the polls application that we build in the Django tutorial rather than starting out with everything architected in the way that it is.

Speaker 1: Yeah, I think it would. Why does everybody love the micro frameworks? They love the frame micro frameworks because they have a great hello world. And it's something that even though you can write single files applications in Django, we haven't done a very good job in Django of putting front and center on the homepage. Why don't we have the Django Hello World on the homepage and say, look, you build your views here and now this is starting to get out of hand. Let's move it into a separate views. py file. And it's like we're we're giving people the aircraft carrier when they want the horse. And the aircraft carrier is great when you need the aircraft carrier, but

Speaker 6: Thanks. for Crude View. Did you ever think perhaps I could uh make this composable and and then get back to the explosion? Yeah.

Speaker 1: No, I never thought that. Um

Speaker 6: just having too much fun.

Speaker 1: So somebody asked me on Um Mastodon the other day, why didn't I use the the generic class-based views? I'm like the whole point is that I'm fed up of digging around in multiple files. Before I found the locality of behavior um essay that's on the HTMX website. I spent years talking to clients about this and being like, No, let's just put the form next to the view Right? The form is only this and you've got a whole separate file and can never find it. Let's put it next to the view and they're like, oh that's really unprofessional. And I'm like, no, it's it's 20 years of hard-earned lesson that you actually can write some code faster if you can see what's going on Um it's the lesson I took from with from Tom all the all these years of working with him. Is that the code that's simple and in front of you is more maintainable, more readable, and just

Speaker 1: frankly better than the code that matches all these great patterns that we read in the the the Java textbooks Kelsey