Front-end architecture fundamentals for Django-powered open source apps

Thank you. I decided to cover quite a lot of front-end in Wagtail and spend a lot of time in particular focusing on both. The pain points and also specific things we would recommend people steal Very high-level introduction first. Why are we here? We're here because Wagtail is a pretty successful CMS. And the reason it's very successful in my mind is because we've spent a lot of time over the year Focusing on the user experience. And practically what that means is from release to release, making sure that the UI front-end code are as good as they can be. Um, but there are clear challenges.

Um I thought I'd um start with uh the blog post that Matthew mentioned earlier, just to hammer home the points that um keeping to up with those changes over the years is quite a lot of work and our product roadmap is jump packed of UX improvements from release to release. And the only way that can be possible is if we refine our approach to front-end development over time so that we have something that is viable for the long haul. There's no way whatsoever that we could pick a UI architecture, a front architecture for Wagtail 10 years ago, and it's still being relevant for what we want to do these days. So we have to find ways over time to adjust it. And by virtue of it being an open source project, it has to be quite a gradual transition from release to release

Just to give you a bit more context, so we're now in the Wagtail UI. And uh some of you might be seeing it quite often. I don't necessarily know how people understand how tricky it is to build something like this and to pack it with more and more and more UX improvements from release to release. So just to highlight some of the more advanced ones in there. The uh current concurrent AT notifications that say it works on in Wactail 6. 2 requires some polling behind the scenes and dynamic deciding when to render this UI element That's something we've tried really hard to build with Django templates and HTML uh Ajax techniques rather than React.

And uh that's again an example of something that's quite hard to do. The the minimap UI we have here, those sections live updates based on where on the page I'm at. And uh I get added and removed as well based on content editing changes. The live preview obviously is completely dynamic based on what I enter in the CMS and the accessibility checks. and content metrics that Albina has worked on over the last couple reasons as well. That's also fully dynamic Um there is commenting within the CMS that Emily worked on quite a few years ago. There's just so many things in there that are quite complex from a UI perspective. And um yeah, it's it's not just uh from a UI perspective as a software as a service

uh product that we sell. It's something that is very much a Python CMS, something that's based on the Django fundamentals. That means that people who use Wagtail expect all of this to be quite customizable as well. uh quite customizable by that I mean for for developers to tweak those components to their liking. So some people come to Wagtail expecting us to have a documented list of uh Oh, here are all of the templates you can override and here are all of the blocks within those templates. And that's just really hard to square that with the UI evolving so much from year to year And just to again uh hammer that point home, uh this very much built on Django Fundamentals. So now we're looking at the Django admin.

I thought it would be relevant to share share because as far as what tools Django provides us, they're very like cluster. on UI development. Django, rightly or not, they've chosen not to have much of an opinion about front end. In some ways that's great, in other ways it does mean we have to reinvent how we do this to quite a large extent So when they developed the the Django admin, the JC of Capital Mosses and Adrian Holobati of the world, they decided uh not to make much of the patterns they use. for the admin, which which was the most advanced Django app at the time, decided not to make those patterns reusable for uh things outside the admin. So there's this very famous saying that the admin isn't meant for you to customize and reuse for your sites.

That's definitely something that uh makes some kind of sense to me. But also the fundamental building blocks behind it, the kinds of APIs, the template tags you make available to build this, there's not that many of them that we can reuse either, with the notable exception of the the forms framework. Just to throw one more challenge into the mix, again, this is uh not software as a service, this is something that we have thousands of people out there looking to customize for their own projects. And this is also something that we have a good thousand contributors out there who are trying to work on together So uh no matter how much I'd want to rebuild it all with React, even if it might be a valid choice for on a technical basis

We also can't expect just any Django developer or UI contributor out there to know any one framework. So that's also one reason why we need to be very gradual with adoption of UI patterns. and why we need to be very careful with um tooling and documentation. So yeah, lots of challenges. I thought I'd highlight a specific one, which again is very timely. Matthew mentioned the performance audits. we've done recently and I thought I'd share some of the lighthouse scores in particular because as you add UI elements from release to release it does add up I don't know here who looks at Lighthouse reports too often. We're just gonna focus on the very high-level metrics here. We're looking at a report from Wagtail 4. 1, which is a couple of years ago, and now I'll switch over to the latest release

And as I go back and forth, it's probably very apparent to you that there's room for improvement to be polite and that some of those numbers here, they're not going in the direction that we want. To some extent, as we add more and more to the UI, this can feel a bit inevitable, but we very much want to look carefully at our UI development patterns. to make sure that we reverse this direction and that we have clear metrics on how fast the admin loads and that we can hit those metrics by picking the right architecture patterns So something I discovered while looking those audits is that Lighthouse also comes with a tool that allows us to compare two reports. and have a very clear idea from one release to another

exactly how much this is evolving. So here we're looking at two years worth of wagtail updates, about 10 releases, and uh we can exactly see by how much those metrics uh have gone down. And uh yeah, those numbers might not seem too good, but to me it's amazing that we get access to this data so easily and that we have the right tools in place. so that we can both investigate where to improve and also over time uh we can enforce that those numbers, you know, they go the the other way. Yes, so again, like thinking of uh challenges, definitely uh lots of this I haven't said the word out loud yet, but uh this is very much technical debt, which I

Like to rebrand in in recent years. I've uh been calling it uh code have instead, just like thinking of something more positive. And uh it's been a challenge for Wagtail for quite a while with it being such a long running project. So this specific thread I'm showcasing here It's from uh it feels like a decade ago now. Not not exactly, but it's from a while ago. And even back then we had pretty clear ideas of potential issues with the code base. and ways to improve them. And um I'm happy to report in particular the things that I had highlighted here six years ago now. we've all implemented them. So very much this is something that we think we're doing a good job at evolving the UI architecture and the front-end architecture over time.

One last challenge before we switch over to the improvements you should consider for your projects We also want Bactel to be more secure, more accessible, more performant, and all of this is dependent on refactoring that code, not just for the UX improvements, for the features, but also to hit those fundamentals. So CSP compatibility in particular, I won't dig too deep in. This is very much something where front-end architecture matters a whole lot, where if you choose Alpine. js you're screwed If you choose stimulus, you have some chance. And where, yeah, that's exactly the type of uh you know cross-cutting concern where we want to be uh making careful choices.

Oh yeah, and also there's lots of UI bugs for us to fix. So just something to keep in mind. This isn't like something that's just perfect and needs tweaking. There's like real issues for us to address. So how do you address them? One of the most concrete steps that I really liked is us spending the time as the the UI team for Wagtail. uh writing guidelines on exactly what we think we should be doing with the architecture of the project. There's room for those guidelines to be ten times more thorough than they are at the moment, but at a very high level They tell us what to do as far as HTML, CSS, JS, and SVG. There is some cross-cutting concerns here, but to keep it nice and simple, I thought I'd go through all four of those areas one by one.

Starting with HTML. So yes, H T M L Uh I mentioned React because uh people know how much I love it and I definitely don't think the whole Wackel UI should be rebuilt with React. But I do think there are very specific patterns in React that we should strive to bring to Django. So the very specific ones to keep it short is a better focus on reusable UI components So that it's easier for us to reason about the code and it's easier to work with designers as well. And better developer tools as well, the the React DevTools, if you've uh never used them, definitely give them a go. It gives you a good sense of uh what's possible if the frameworks allow

it. And and finally the maintainability. So being able to test UI code. I can't tell you how annoyed I am when people in the Django world this React here and there because they don't realize how good it is when you can uh actually write tests for your UI components, which is one of the big things that React has changed. And yeah, we don't want to use React. So what we do want instead is to make those innovations happen in a more Django-friendly way. I thought I'd highlight sleepers here, the library from Mitch. Wagtail itself does not use sleepers, but we do use quite similar patterns that we've just hand-rolled with our own little building blocks. The fragments tag from sleepers in particular

is to me something that's been essential in the Django world for almost as long as Django has been a thing And um yeah, we want to have libraries that do this, but we also want Django itself to evolve and get those capabilities. So I'm very thankful to Jake, who has been working on uh bringing this type of uh fragment template tag to Django itself. I'm really hopeful this will make it for the Django 5. 3 release in April um but either way it's just excellent to me that uh we see those libraries getting created inspired by projects like React and we see the improvements coming to Django itself uh in the long run In a very similar vein, I'm not sure here who uses

LSPs for Django templates. So is this idea that behind your IDE there is a language server that is aware of the language that you're coding in and gives you relevant suggestions for that specific language And uh there are a few options to do it for Django templates that uh have started popping up that I think are exactly what we need at this point in time. I don't know how many people use it for Wiktep at the moment, but it's definitely something where I'd really like all Django projects ever to have that kind of improvement set up. There's something else that I really like in here that I thought was interesting for people to think of. This is all about LSPs without too much change to how you code

otherwise, except for this one line here. which is typing within Django templates. I'll just leave it there. Lots to think about here. And yeah, DjangoPad Library, if you've not used it yet, you might have heard good and bad things about it. suddenly there's room for it to be better. Suddenly that's exactly the type of tooling that we need to build better UIs with Django Okay, over to CSS next. I see that timing's tight, so I might speed up a bit. With CSS, I thought I'd focus on colors first and foremost, because it's a good way to highlight how important it is for us to make Ragtail customizable. We have quite a big color palette

and what's interesting about it is that it has to support light and dark themes obviously Which is why we've introduced those fundamental color tokens that are just based on describing the uh you know perception of the color, the hue. Then we have those more semantic color tokens that allow us to know where a given color is meant to be used and also specify how it switches between light and dark themes. We've documented these heaps. So just go through some examples and uh head straight to the BackdUI to show you the dark theme. You've all seen dark themes before, there's nothing new here. You've seen our light theme as well. What you might not have seen before is other types of theming. So A very recent improvement we worked on

with Albina in particular is uh enhanced contrast admin themes. So that's an operating system feature. in uh mac OS and iOS to request more contrast from your UI applications. And that's something we've strived to implement in Wagtail And I think I really like it because it's a feature that speaks very well to the needs of our users to have more control over the CMS and to have just more accessible interfaces. But it's also a great example of how much we struggle with our front-end approach. So on Albina's pull request, we can see lots of different approaches on the same theme. Here we'll cite a button. scss SAS style sheet and we see we use a mix-in to add this contrast improvement.

Further down, we are inside another . ss file, but here we're using Tailwind and we implement similar improvements in a very different way. There is something that we definitely don't have the answer for just yet is do we need to write SAS and BAM only or tailwind only or somewhere in between. And I really wish I could tell you one way is better than the other, but at least at this point in time to me, there's clear benefits to both For colors in particular, for colors, I definitely don't think Tailwind's the right approach for things like layout. So the utility class is take you a long way. Yeah, again another pattern. So just to make it clear that it's a mess, but

in my mind it's a mess for the right reason. And so yeah, coming back to the UI, uh just to make it clear, this is the kind of small tweak that this allows us to do, and that will make it easier for people who have a low vision in particular to understand which UI elements are interactive and um yeah have an easier time using the CMS. And yeah I'll skip right over that but again it's not just uh light and dark it's not just for uh enhanced contrast we also want to support all sorts of accessibility features in operating systems to do theming so here we have an overview of all the different colors that the admin UI could be using based on people's operating system settings. We also have the admin

UI in Arabic that we want to support with just the same styles if possible. So lots of things here that we've been able to leverage from the fundamentals of CSS. to make this possible. Okay, now JS I'm gonna be a bit cheeky here and talk about Python. Sorry, Helen. Lots of people have seen this news of Python becoming the most popular language in GitHub. Personally the bit that I find the most interesting in here is exactly how that came to be So we see Python taking over from the Rascript here. But what we still write under is JS and TypeScript being two separate languages

And they make it very clear in the wording that they have around these charts that what's happening rather than necessarily Python taking over JS is just uh JS and TypeScript together having a bit of a split And uh the reason I bring this up uh is simply to say that uh I think TypeScript is there to stay and uh I see no no reason not to use it for complex UIs like that of Wagtail. You might not need it for all projects. There's certainly patterns in Tapsita are quite hard, like dumb manipulation. But I would say in the grand scheme of things, in particular with AI tools these days and again language servers. uh you just get better results with TypeScript, you get better QA, you get better productivity.

And um yes, another key aspect of our JS technical choices, as mentioned, is this framework called stimulus. So back in 2018, I believe, we spent a lot of time looking at our usage of jQuery and React and trying to understand, okay, exactly which framework is better for us in the long run. And the answer was neither of these. So we looked at things like Alpine JS, realized pretty quickly that Alpine was completely incompatible with the codon security policy. and we picked stimulus instead, which is a framework introduced by the Rails community. And um we've been refactoring our code over to stimulus in the past. I don't think I'm gonna show you much stimulus code here, because it's yet another thing to learn. But one thing I definitely want to show you is this code coverage chart, which looks at uh Wagtail's

UI code in this client folder. And what we can see on here is how much unit tests we've been able to add for the different parts of our UI and components here that tends to be React code. Definitely very testable, though there's room for improvements. Controllers, this folder here is our stimulus code, which is 90% tested. And the only reason it's not 100% is that we tend to be somewhat uh lax sometimes about code coverage. So for example, singling out this drill-down controller component. It's a very complex UI component, but the reason is not it's not tested. It's just because I was in a rush ahead of a release and I didn't find time to write the tests.

But where we do try, stimulus definitely makes it possible for us to get 100% code coverage for UI code. 100% is a bit of a loaded number for coverage. There's no really right or wrong figure here. But certainly there's no complex UI interaction that you can't test with stimulus. Okay, quick whistle stop tour of SVG. I wanted to mention it in particular because of the icons, which we spent lots of time. reinventing. A specific challenge I wanted to highlight here was how much time it took us to make icons reverse in Arabic right to left layouts. Again, I'm gonna be very quick here because it's quite deep, but just so you know that if you need it, there's extensive research out there on the 10 different ways that you could potentially do icons that mirror

Takeaway here for me is that again it's a good example of where we spent the time to make sure that Wachtel was as customizable as possible and that this was sustainable in the long run. I definitely think that in this day and age, just skipping over the fundamentals here, there is no reason not to use SVG. uh icons for everything. Uh these days if I see a background image in a project, that's definitely an anti -pattern to me for lots of different reasons. So yeah, long story short, lots of challenges, lots of room to improve, but hopefully clear path for us to get there