API Maybe: Bootstrapping a Web Application circa 2024

Speaker 1: Hello. Water tick. Slides tick. Hello, um I'm Carlton, this is me, that's my avatar there. I'm Carlton Gibson on GitHub, I'm on Fostadon, I've got a website with my friend Will, a co-host of pop course called Django Chat. Um if you haven't listened to that, please go and give it a listen. And we um get guests from around the community and we chat about Django. Um if you don't know me, between 2018 and 2023, I was one of the Django Fellows. Now the fellows are contracted by the DSF, the Django Fellows. Django Software Foundation to do the day-to-day maintenance on Django. They do triage ticket, pull request review, releases, all of that stuff, the stuff that keeps Django going.

Speaker 1: Now I've stepped down, this is a talk isn't a fundraiser for the DSF, but go and sponsor the DSF. Or better yet, your company, you make a small donation that helps keep the Django going. You're making an investment in the sustainability of the framework you built your business upon. So let's go just go and do that. After I step down. I started on a project called PARDS, which is in the monitoring and evaluation learning, which is a monitoring and evaluation learning tool for development and sustainability projects. I meant to say that we're hiring, but we're not hiring yet. We're a year in and we're going well, but it's this next year that's going to tell whether we can hire or not. So keep it in mind, maybe

Speaker 1: talk to me, you know. DjangoCon 2025. Today's talk is called API Maybe. It's about bootstrapping a web application, specifically our web application, in 2024. Before working as a fellow, I worked on Rett Django Rest Framework a lot. So back in 2018 when I started, it was API first. It may have been API only. The world was very definitely API first. You'd work on a team with a very clear division between back-end, built by one group of folks, that would expose a REST-like JSON API, and the front end, built by a totally different group of people, would then consume that in quite often

Speaker 1: quite often As the API's only client. Okay? Now you might be sat there thinking, hang on, it's still exactly like that. But spin forward to 2024, and things have changed a little bit. Largely in response to the seeming seemingly ever more complexity in the front end. The last few years have seen Uh Returns were focused on rendering HTML on the server and the emergence of a family of related front-end frameworks that go under the monkey of HTML over the wire. Now, HTMX is is oh, hang on, hang on, hang on, no spoilers. HTMX is the one that's probably grabbed the most attention in the Django space, but Turbo, Unpoly and others are doing similar things in a very similar way. So I'm going to talk about HTMX and all that. If you like a different one, you do you, right? It's the it's the the underlying idea that's important here. But something

Speaker 1: Else changed between 2018 and 2024, which was where did all the money go? The great inflation that followed the pandemic, the global supply chain crisis, and the Russian invasion of Ukraine marked an end of the zero interest rate environment we'd we'd had. We'd let's say we've enjoyed for the decade or so since the last financial crisis. You've all felt it. Tech layoffs are in the news every single week, it seems. But it means that you can't just go out and pick up a big old bag of cash and say, come on, let's build a And multidisciplinary team to bootstrap your product. On paths, we have a limited budget. We've had tight milestones, like 12 weeks to get from the idea to a proof of concept. Another what 12 weeks to get from the proof of the process of the process of the process of the process of the process of the process proof of concept to what you might think of as an MVP and only a few months to flesh that out with supporting bits like versioning and role-based permissions and you know all these things that a rich app

Speaker 1: needs. And then we didn't have cash for a big team. On the technical side it's literally just So there's no way on earth I could have done that if I had to take on Django and take on a big front-end framework like React. Okay. My contention is that by leaning into Django, you can get Just as much done as bigger teams using more complex stacks that Jan and that Django really shines exactly when money is tight. Django has always been the framework for perfectionists with deadlines. Well, when those deadlines Of financial all the more. Django is the perfect framework for our post-zero interest rate world. Ah but first, I want to talk about locality of behaviour.

Speaker 1: Locality of behaviour is a way of thinking about and assessing code. Some code has locality of behaviour when everything you need to understand that code is right there in front of you. The contrast is when you need to go and look somewhere else to see what the behaviour is going to be. This is what Karen was talking about just now. The idea is that other things being equal, greater locality of behavior makes your code easier to reason about, easier to maintain, and easier to iterate. Again, you'll be familiar with this. Tailwind CSS is a utility-first CSS framework that's very popular. It gives you utility classes that you embed in your HTML to control styling rather than needing to keep that code in a separate CSS file. Alpine. js is a rugged little jQuery-like JavaScript framework that lets you define your component behaviors again in your HTML rather than put

Speaker 1: putting them in a separate JavaScript file. Of course, HTMX adds a small set of HTML. HTML attributes that expand the native powers of HTML all again without leaving your HTML file. On top of these tools, I uh both in the name of locality behaviour, I um manage a package called Neotopolitan, which is my take on quick. Crowd views for Django and Django template partials, which adds reusable named um template fragments to the Django template language. Now I suggest I suggest that particularly with new code, while you're still working on it, while it's still in flux, a focus on locality of behaviour can help you go faster. But I don't think I've said that disclaimer boldly enough. So let me be a hundred percent clear. Locality of behaviour is a starting point, not a destination.

Speaker 1: If I've got a button that's styled with Tailwind, I'm not going to mindlessly copy and paste those classes around where every time I need a button. I might stick the HTML in a partial or a template tag. Much more likely I'm just going to pull it directly into my CSS piles CSS file as a new button utility, as anybody would always obviously do That's what Karen was kind of talking about with um AlpineJS. Alpine JS is awesome. A little one line handler is beautiful in your in your template file. Two lines, well no problem. And at any given point, just adding one more line of JavaScript to that, it it's not a problem. But at some point, probably quite soon, you're going to pull your JavaScript out into a separate file or into an Alpine plugin so that you can use it

Speaker 1: just as an attribute declaration on your HTML. You're going to get linting, formatting, testing, all the things that Karam. saying was worried about. And again, you're going to use your judgment as to when to sacrifice a little bit of locality of behaviour as your code scales to gain in maintainability. Just because we're talking about locality of behaviour doesn't mean we're throwing away good engineering practices that we've learned since our cradle. If you're using Neapolitan's CRUD view that I'll show you in a bit, you might say, oh well this logic would be better as a set as a separate view. Well so on. I'll try not to win because I say that of course you shouldn't put three thousand lines of code in your URL stuff. But I will say on locality of behaviour, hold the line. Holding the line on locality behaviour can really pay dividends.

Speaker 1: Dry, separation of concerns, sure, sure, sure, all those things are great. But for every one of those, there's a write it twice and or a three times refactor or um something like that. As I was writing this talk, Luke Plant published a nice post comparing these things to proverbs. They're great when they apply, but there's always a contradictory one if you need it, right? You can always find that one out. And again, we need to be sensitive with these things. As a junior I learnt a principle called protected variations. And the idea is that you you identify the areas of your code that are subject to change and you build a stable interface around them. Even as they evolve, they don't infect your code everywhere else. Well new code, when you're still working on it, when it's still in flux, when you don't really know what it's going to look like, is the perfect case of code that is likely to change.

Speaker 1: By focusing on locality behaviour, by keeping everything together, you'll m you make sure that any changes don't escape out of that file or or out of your module. If it looks like you're repeating yourself, violating dry, but if you extract the code somewhere and then reuse it, and then when you realize oh no actually I do need a change. You've then got multiple places to fix. Okay? You've violated protected variations. Now on the other hand if you just defer extracting that bit of code slightly, focusing on locality behaviour, you allow the code to stabilize. That means less churn. But it also buys you more time for the deeper patterns to emerge. Quickly refactoring to dry. It's all very well and good, and it can help, but it can be a premature optimization. It can lead to your code getting stuck in a local local optimization.

Speaker 1: Optimal. Buying the space for those deeper patterns to emerge is for me one of the greatest payoffs about thinking of locality of behaviour. It's I think it's really interesting to think about how far we can or how far we should push locality of behaviour. When we're building our applications. I think that's a discussion. I think it's case by case, app by app, team by team, but from where I'm sat it's super exciting. It's interesting, right? So, leaning into Django, what was the talk about again? Oh yeah. Yes. API maybe. Bootstrapping an application circa 2024. I think that Django has a grain. It wants you to do things a certain way, and the tip is go with it. Okay? So this is going to be a refresher on Django, right?

Speaker 1: The first thing you need is a model and a view. 90% of every app is basic CRUD operations Last year I introduced Neapolitan. It's my take on quick card views for Django. A year later, I couldn't be happy with it. Apart from the odd template view, Neapolitan has totally replaced my use of the generic class-based views from Django. So let me quickly show it to you again. I've got a model, it's a book. Bookmark model. It's got a URL, a title, a note, and whether it was favorite or not. I want an easy crowd view for it without it taking all day. So I import Neapolitan's CrowdView, I define a subclass, I tell what model, what fields to use, and what I just add it to my URL pattern and that's it

Speaker 1: Neapolitan's CrudView provides a standard list, detail, create and edit, edit and delete views for a model, as well as all the hooks you'd need to be able to customize any part of it. Provides base templates and reusable template tags to make getting your model on the page as easy as possible. Where you take your app after That is up to you, but Neapolitan will get you started. The key point for me is that your code for your view lives in a single class. So it's all right in front of you. As we said, maybe you get to the point where you break it out into a separate view, but f but general, you in general, you get a long way before that. happens. In the meantime, you've got a working space that allows you to develop your cult your code in a kind of controlled environment. It's all just in that one class and as long as I keep it there, I'm safe. Crogview itself is just a single

Speaker 1: class. It pulls all the familiar APIs from Django's generic class-based views, get form, get query set, get context data, all the ones you're used to. But with Neapolitan, I'm never asking myself, oh where's that implemented? Oh where is it overridden? I don't need a whole Separate website. It's in one file. It's there. It makes a big difference. I'm having massive fun with it and I recommend it to you, so go check it out. But once you've got your view and your model in place, the next thing you need the next thing is your template. Now I've been redesigning. Discovering Django templates. I don't have too much to say, just send HTML. But that was the payoff from the move away from API first. Just sending HTML is a big win. You're not creating serializers and then Building your HTML and a whole separate stack, but you're doing it right there in in Django, in the same

Speaker 1: environment that you're writing your views. After a decade of not really pushing their limits, I think it's an exciting time for Django's. Hey, what for Django templates? What's built in is great. I've lost count of the number of times over the last year that I've wanted something. I've gone to the Django docs and it's already there. It's like, wow, I've just never used that filter. It's been there for Fifteen units. Karen probably added it. Um writing your own custom tags is easy, it's powerful, it's probably uh it's I think it's a little bit underdocumented. I think writing custom tags is a really great tool. It's like, oh I've got a little bit of logic that I want to put in my template. And Django templates don't let me, but if I write a custom tag, I can im I can encapsulate that, I can test it, and it keeps my templates clean. And there are a number of packages out there that add things, um slippers, Django components, they the list goes on.

Speaker 1: It's it's a vibrant time and I think we've got a real opportunity now with the in with the with this HTML return to server-side rendering. thing to push the Django template forward in a way that it just hasn't been pushed forward for a decade or more. So that's exciting. So we've got models, we've got views, we've rediscovered the templates. The last thing we need is forms. The final component is forms, right? Sometimes I think their forms are the kind of missing link. They play this dual role in Django, being the data sanitation layer of your application. and of presenting the user the user app, the UI. People think forms are about rendering. No, they're about data sanitation, but they also do this rendering.

Speaker 1: If you want it's okay sending templates, right? But if you want your user to actually update your any uh your application in any way, you're going to need a form. Forms are your are your thing. So there's this conversation in Django often about where should we keep our business logic in Django? Is it in the view, is it in the model? Well, I think quite often it's in the ro it's in this the form layer in its role as a data sanitation layer. So I want to give you an example of adding parent objects to a model. So let's recall our bookmark model. Let's say we want to add a foreign key to the parent user. User. Okay? How do we go about associating that user when we're creating the bookmark? Well, the standard form valid implementation goes like this. We just call form. save, store that in um self dot object, and then we return a um a

Speaker 1: a redirect response to the detail view. Okay? But if we want to attach a reference to the current user, we're going to need to do something different. The usual approach is to call form. save with the commit equals false argument. When we add the use so we so when we commit that form. save, commit equals false, it doesn't save the object. We then um attach the request. user And we call um we then save the object ourselves and we probably need to call this form this mysterious form save many to many method, which we always forget to do. We don't on our bookmark model, but in general we need to do that step as well. And then it's only then that we can return the redirect response. Now that's okay, right? Like it works, right? But it makes an absolute mess of our view.

Speaker 1: More though, the view's job is about turning requests into responses. By having the user set in the view, it's it's It's making the view responsible for the business logic, the bookmarks need a reference to the parent user. What we want to do is to move that into the form. So if we create a custom form class with a custom init, we can override the init method to take an optional reference to the parent user. If we give it a um the user equals none keyword argument there. We can check that, and if we didn't pass an insert, That means if we're creating a new bookmark and not updating an existing one, then we can require that the parent user is passed instead of the instance. And we can raise a clear error, if not.

Speaker 1: Okay. Then we set the this we set the user on the forms book the forms book the the forms bookmark instance here ourselves at the bottom. And so when we come to save it later in the view The bookmark the the current user reference is already set, and we don't have to infect our view logic with that. So then in our view, all we do is we we tell our CrowdView class to use the bookmark form, and then we override the get form method here to if we're doing the doing a create method, get the current user and pass it to the form as we need to do. Okay? We no longer need to do anything custom in form value. And the responsibility for associating the bookmark to its parent user lives in the form

Speaker 1: layer where I'd argue it more rightly belongs. Now that's just one example, but you you can generalize. Where possible, you want to push business logic into the form layer rather than keeping it in the view. So the view does no more than call form is valid, form save, and branch accordingly to determine what response to reply. The parent object pattern comes up all for almost every model. Generally your models form some sort of tree and each each model refers to the parent above and needs that foreign key set. So I'm going to generalize this pattern and make it fully declarative in But there's nothing Neapolitan specific about this. You could do it with the generic form view from Django and do the same thing.

Speaker 1: So then that's our Django stack, and we want to spice it up a bit because Django's a bit boring and we can't be we can't just have full page loads and all the rest. So we need HDMX and and and and template partials. HTMX was the change that made all this exciting, right? All the related technologies. We want a a more we want modern, interactive, oh isn't that nice response from our clients, but we we want to do that still with Django. So So pushing HTMX, it gets you a long way, and it's worth I'll just say now because I won't remember to say it later. It's worth looking at the extensions on HTML. Like, for instance, the messages framework. You want to, oh yeah, you you successfully did something. Well, it's it's got an extension that will you can put up a type You could style that however you want. And it's got there's a remove me extension to HTMX that will take it out of the DOM for you know three seconds later or whatever.

Speaker 1: So you don't need extra JavaScript, you can just expand HTMX. I would look into that. Let me show you an example. So HTMX. You've got a simple button, it's got an HX attribute that you that that goes and fetches the clicked um URL when you click it, and everything it it it replaces whatever HTML comes back, puts it in the DOM. Now everything there is in the one file, and you know once you know what those HX attributes do, you can see the behavior without going to it anywhere else. On top of that, the thing that's really sort of pushed it for me is that is Django template partials. Now this is the I an idea from HTMX, which when you've got a full template, you perhaps only want to re replace a single

Speaker 1: So you've got, I don't know, a list of to-dos. And you update one to do, you just want to go and fetch a single to-do, not refresh the whole list. Well, you just want that small bit. And with the Django template language by itself, all we We'd have to move that into an include file in order to reuse it. But um Django template partials lets you define these fragments in line and then in the same template and then you can reference them by name later on. So you define a a partial in your view, you load partials, you def you use Use the start partial m partial tag. I've might even change that. I have, it makes it called partial def and m partial def now. That's okay, don't worry. The old names still work. Um so first of all we load them and we define our partial and then later on we can use it.

Speaker 1: And we just use this partial tag to say, look, load the load the the partial tag and it it gets included in the right place. And that that enables the reuse, but you can also it's also integrated with a template loader. So each temp each template has a has an a a dictionary essentially of defined partials and if you pass if you use the template uh Jackangle template partials loader you can just add the fragment identifier on to the template template name and the template loader will go and look for the template if it's found, it will look to see if the parser will defined on it, and if so it will ret it will just return that bit of the template for you. And it's a way of getting some reuse a degree of reuse out of the Django template languages. which is a little bit more flexible than includes.

Speaker 1: And that view, that change there, that would be just done in the view. And that's the only change you'd have. So if this is an HTMX request, load this template instead of the other template and carry on doing exactly what you've done before. The rest of it is entirely transparent to your view layer. You still replay return a template response as you always have and all the rest of it. So for me, the combination of HTMX and template partials has been really powerful. It effectively lets me stay in the one template file with the one view class supporting that. For much longer than it was possible to do before. Now, this is a real productivity boost. You can simply get more done, right? And you can tidy it up later, once the structure's laid down and once the patterns start to emerge. So finally, then I just want to mention our

Speaker 1: It's lightweight, comes in at a few kilobytes. Now if you only use it at one in for one thing in your salt site, well you'd be better off just with vanilla JS. But if you're using it over your site, those kilobytes buy you a lot. You get um state management. You get the r reactivity, you get the callbacks. It it's a lovely little library, so I do recommend it g giving it a try. It gives you a lot of ban for your bucks. It's it's easy to use, it's a simple script tag, and it's it's like HTMX and Tailwind and the others. Here's an example. Here you define a a component using X data and we'll give it a count, and then we just with the X text we can say put the count in this paragraph, and then on the with the on-click handler there we just say increment the count. And that will do exactly what you imagine. And that's it. It's lovely and declarative. Again, it's all in the file in front of you and it's really, it's got really good locality behaviour.

Speaker 1: I think it feels like jQuery. It's the first thing I've used in a long time that gives me joy. On the front end. So Alpine JS informs again. The problem that I had for a decade or more was that I was declared because I was API first, I was declaring serializers. And serializers were just as forms were, recapitulating a lot of my business logic. And then I'd want to do the HTML page and I'd have to create a j an HTML form and I'd be that oh no I've got to duplicate all that work and so it would never happen. Right? Well, I'm declaring all these forms. Now I'm using JavaScript. Do I have to declare serializers? I hope not.

Speaker 1: No. JavaScript has a form data object. The browser has a form data object. You can instantiate one of those. And then you can add your fields to it and you can wrap a helper so that it's just like JSONify or whatever it was StringyFy or whatever the thing is there. And you can send form data back to Django, where your Django view is already primed to receive that form data and then it can turn send back some HTML. So you can use Alpine JS in the rich quite a rich widget. We've got a drag and drop interface which we built in that AlpineJS and it's perfectly maintainable and it's lovely and it's like okay. You can scale up. But what we do is we trigger an event from Alpine, which is picked up by a helper, which uses HTMX to dispatch the

Speaker 1: the the request batch to the server and replace the um the Alpine enhanced HTML that comes back to the server, which then Alpine can can work with. It's a lovely setup. It really does work well. There are some life cycle tricks. You hit sometimes if you try and do something with Alpine before the DOM's fully fully settled in it just it just won't work and you won't get much debugging help from it. You have to do it. So there there's there are some tricks. There's a next tick helper which says wait till the DOM's settled and then do your alpine magic. If HTM HTMX you sometimes need to call preparer on any on any on on some new any new um HTML you put into the DOM with JavaScript, you need to tell HTMX that it's there. And likewise sometimes you need to tell Alpine that you've put some

Speaker 1: a new tree in, and you can call Alpine init tree to say, Hey, here's some new HTML. But these trick they're not they're not tri they're not tricky. I'd say it's a very good stack. And it's enabled us to get from zero to what's what I call a genuinely viable product in a quite short amount of time. of time. The next year, as I said, is going to tell whether we get you know to fully sustainable, but leaning into Django, adding HTMX with a few sprinkles of Alpine to enhance, it's a great combination and I I really do recommend it to you. Looking forward just to wrap up, I just want to talk about a couple of things that are on our robot. Mostly it's like keep refining, just keep doing the same. We're young, it was only me. There's lots to do. We have I have to-dos

Speaker 1: upstream this all over the place. But that will come over time. There's two things we really want to add. One is um async for collaborative editing. And this is exactly what the talk you just had was talking about. We've got a nice table view with data in, and people can edit the the the view the data. data in there, but it's single user. And that's not really a problem for us. But they're gone yet. But there's going to be the case where two users are editing the same data. And what we need is you to get a notification saying, Hey, this was updated and then we need the UI to go on by the way I'm going to refresh that data and give you some Nice animation so it draws your eye and tells you what happens and and all those other things. So we need to run either WebSockets or Service Center vents on the side of Django. Chip it on the question, I absolutely agree.

Speaker 1: Run your async on the side until you're 100% happy with it because your whiskey server is much more battle-tested. And then the other interesting request problem is about performance and latency. Pars is based in the server is based in Paris, in AWS 's Paris region. Our clients are based in Kenya, Bangladesh, China, Vietnam, Turkey, around the world. truly distributed. So performance is key. Now most users are office-based. We don't have problems with mobiles and poor networks and all of that, but the pure distance is enough to give us a performance, a noticeable performance issue. And we've done the basics and we haven't we haven't even pushed all the buttons yet.

Speaker 1: So there's a lot of roadmap to go. But the thing that we're thinking about is, well, okay, do we need an outpost in Nairobi? Do we need one in Shanghai? Do we need to be closer to the user? But what's interesting is that at a really high level. This latency problem was precisely where we were all meant why we were all meant to move to thick clients and rich rich rich SPAs in the first place. Supposedly smaller package sizes from shipping JSON to the client road than HTML would lead to better performance. Now with the modern web, we've seen that that isn't the case. Bundle sizes are so big that pages remain unresponsive for an age. It's actually slower for the browser to convert JSON back to HTML on the client than it is to just accept HTML in the first place. So I have to say, I really don't know where this question about performance and latency is going to go.

Speaker 1: But what I can say is that wherever we end up, it will be based on user needs, it will be based on evidence, and we will grow into it. We're not just going to default to it simply because that's what Facebook felt. They needed half a decade ago. Integrations. Well, yes, we'll need an API of course because people want to plug it into their Power BI, uh, you know, dashboard or whatever it is they've got. Um but the shape of that looks Much different from a REST API powering your main application. It's like can we get a CSV , a read-only CSV export? It's on a s it's a single thing that you just tag onto your CrudView and export in the right format that you want. You've already got your auth, you've already got your you tech everything that you've got in play, you didn't have to recreate that in a second step, in a second sort of stack again.

Speaker 1: So that's it. Leaning into Django. When I stepped down from the fellow role in order to get back to building things with Django rather than just working on it. I was looking forward to this opportunity. And I have to say I'm having the time of my life. In 2018 I never expected Webb to swing back Django's way. Then the thought was, well, can we merge DRF into the core? Well, I think that moment's gone. There were two many exciting things happening around APIs with Django for Django to pick one option and then crown it. But I hope I've shown you you know you how, by leaning into Django and then adding, you know, little bits of tech that spicy it up. That not it it's not a question you need to think about until you're well down the road. Maybe I well maybe.

Speaker 2: Thanks