Wagtail, Reactivated - Headless Without the Headache

Speaker 1: So with that, I'd like to introduce our next talk. Josh Morantz is a full stack software engineer at the lab. That's the name of his company, The Lab. And he's going to be talking about Wagtail Reactivated, Headless Without the Headache. Thank you, Josh. Come on in.

Speaker 2: Hi, everybody. Uh So maybe I should have called this talk headless without most of the headache, because as we all know, there's no such thing as free lunch and web development or anything. But before we get to all that, hi, I'm Josh. I write code mostly Python and TypeScript, mostly for money. Occasionally, when I am on the internet playing certain games, I pretend that I am a tree wizard and write Lua code. And if that confuses you, congrats. That's the most confusing part of the talk. You'll probably be fine for the rest of it. Yeah, and I work at the lab. We're an agency. We do e-commerce, we do brochure sites, we do

Speaker 2: content and design and all kinds of things, most of which I'm not qualified to talk about because I'm just a web developer. But here, I'm here to talk to you guys about White Tail Reactivated. Uh Reactivated is a library. Shout out to I believe it's Sergio, who is the author of the library, for picking a good name that makes for a good top title. I do appreciate that. So what is reactivated about? If you look on the left here , you will see a pretty simple Django view. Nothing wagtail about this in particular. This example is taken straight from the reactivated docs, their website.

Speaker 2: And on the right, you'll see a Django template. Or it's mostly a Django template. The difference between it and a normal Django template is that it is written in React, in JavaScript, or TypeScript to be more exact. That's what reactivated is all about. It's about writing your Django views the way you are used to, and then without having to manually define an API or serialization layer most of the time. You can just write front-end code in React and it will get server rendered. Because that's a huge pain in the butt a lot of the time, server rendering JavaScript. But with this, it just works with Django's normal template engine, or

Speaker 2: not the standard Django template engine, but with its system for alternate template engines. And one thing to call out is that if you are like me and you like types, that those two bits that I am pointing at with my friendly little arrows, uh That is automatically generated types based on the based on Python's type system being generated in TypeScript, which then Define type data that gets fed into our template components. Again, without us having to write those types ourselves. And that's a big deal. Because not having to define the shape of your data in

Speaker 2: three different places across two languages is a big time saver. So let's talk a little bit about how this works. The answer is pretty much the way you would expect. In prod, your Django site is just a normal Django site. And when It needs when Django needs to figure out how to render a React component, it talks over a Unix, it communicates via JSON over a units, a Unix socket. to a uh React SSR server process. In this case, Vite. It is implemented as a Vite plugin. And the advantage of that over a typical API is that you don't need to define how that JSON

Speaker 2: is structured and also uh It doesn't need to leave your machine. So there are whole classes of security problems that you can pretty much ignore because all of the data stays on your machine the same as when Django is potentially talking to your database or something else. So why would anyone think that this unholy abomination is a good idea? Well, for me personally, I like React. Not as many people like React these days as when I started web development like seven years ago, but I still like it. So it's really cool to be able to still use it And writing API routes and serializers adds a lot of boilerplate.

Speaker 2: You have to define the shape of your data in a way that your database understands. You have to define it in a way that your serialization layer understands. And you also need to define it in a way that your front end, your uh deserialization layer understands so that it knows what kind of data it's going to be getting. And not only does this cut down on that kind of boilerplate, that kind of writing the same thing three times It does so giving you some amount of type safety, even across the language barrier. And I'm one of those annoying people who really likes TypeScript and type safety in general. So that's a big win for me. Another thing that I'm going to touch on at the end of this talk is gradual migration.

Speaker 2: It's gonna be a while before we get there, so I'm not gonna say much more, but this the a big advantage of this all being Part of Django's template engine is that the same way you have your normal Django templates and you could introduce an alternate Lang you know, template language, uh Python-based language, and have Django use the correct one for the correct template file. You can do that here also with React and you can mix and match which you are which is being used on your server and to the end user they can't tell that some of that HTML is generated by Python and some of it is generated by JavaScript. That is very hard to do. So let's talk about

Speaker 2: some wagtail stuff. To note There's a lot of code on these slides, and for the most part, I don't expect anybody to read all of it. Certainly not here. Some of the stuff that I am talking about is uh is closed source. Some of it is uh open source but not so well documented because Reactivated is not as mature of a project, uh certainly not compared to something like Django or Wagtail So I want to put code on these slides that y'all can use to reference it later, but don't worry about trying to read all of it right now. From these two s uh these two pieces of code, the Python on the left and the JavaScript on the right.

Speaker 2: This stuff down here is almost the entire is the entirety of what Reactivated deals with. You can see that we are overriding get template, which in a normal Wagtail view is just how you direct direct the system to know what template to use to render it. And we are passing in this typed Python object. And we are telling it what data should be serialized. And that's all. That's all that we have to do in order to get this server-rendered React working with this particular component. If you want to look at the results of this site by the way, you can head over to Wagtailbakery.

Speaker 2: dev. thelabdev. net That is this is our ongoing port of the Wagtail Bakery example to use Reactivated. Just to kind of showcase that it really to help showcase the similarity between what we do or what we what we do with this architecture and what somebody might be used to if they're used to the wiretown bakery uh example. So yeah, that's that's where we pull in the uh auto-generated types. The auto generation happens via JSON schema. So Python already has tools to, given a

Speaker 2: shape of a Python object, generate a JSON schema definition for that Python object. And then there are other tools that can consume a JSON schema and generate TypeScript. All that Reactivity does is put it together. It's all made with kind of off-the-shelf parts. There's no There's no custom magic there, which considering this is a third-party library maintained by a very small team, I think we can all agree is probably for the best. What element? So you can see that there are, for the most part, pretty standard bits here. There's only a few hints of things.

Speaker 2: that maybe you wouldn't expect to get out of the box. And that's because you don't. Wagtail or Reactivated doesn't handle Wagtail this normal this this kindly out of the box, we needed to write a little bit of kind of glue code which we have we which we have been sharing between all of the sites that we've put in production that use this architecture. So the nice thing about this is that routing is almost entirely Wagtail standard, which anybody who has tried to maybe make a Next. js site. that has a Wagtail backend uh probably knows routing can be really annoying. You lose a lot of the benefits of Wagtail.

Speaker 2: with a lot of integrations because you need to basically hard code your routes when the whole point of a CMS as dynamic as Wagtail is not doing that. This bypasses that entirely. Serialization mostly just works. All of the built-in field types from Django. Reactivate it understands how to serialize those. You don't need to tell it, hey, this is a string, turn it into a JSON string. Hey, this is a number field, this turn it into a number. Reactivated understands all that and you can just do it. And you get those types generated via JSON schema. But there are maybe some downsides. Um when it comes to more complicated objects.

Speaker 2: Let's say you have an image type that you want to serialize in a particular way in a lot of different places, you do need to teach reactivated what that type is and what you want it to look like before you can just hand it an image and it will know exactly what to do with it. And uh as you saw, the uh the template requires us handwriting some typed Python because Uh Python's type implementation type stuff is like not so mature, so it's not as good at uh inferring the types. from just from just the code.

Speaker 2: So we do need to make sure that the stuff returned in our context is typed. And the package that we wrote at the lab to deal with those problems is creatively called the Lab UI. It is not open source, but I think there is space for an open source unopinionated library. What we have just isn't that because we are not set up to maintain an open source a public open source library. If people are interested, we could probably hack something together tomorrow at Sprints. But also it's something that could very easily exist in the future So let's talk about what is in that library.

Speaker 2: And this I'm in these next few slides, I'm really going to try and focus on the downsides of this approach. gnarliest edge cases because I think all of us have, especially if you pay attention to the front end world, have heard enough from flavor of the week, oh, this is so cool and everything is so easy kind of uh kind of libraries. Everything is mostly pretty easy with this, but there are some downsides and I want to talk or some some rough edges and I want to talk about them. So for this, all we're doing is we've made ourselves a base page class that overrides, uh that is a proxy model for Wagtail's built-in page. It adds some tight boilerplate just because again,

Speaker 2: Python type ecosystem isn't as good at inferring things as you might expect. And it sets that is preview Boolean that you might have seen in the previous in the previous example. That just exposes to our React count uh React templates whether they're running in preview in the Admin or not. Most of the time you don't need that information. Occasionally you really do. So why am I wasting your time by showing you this when I'm trying to tell you that it is mostly nothing? Well, I want to emphasize that it is mostly nothing. Things there mostly work We don't need to do anything crazy, like, yes, there are some overrides that I'm not showing you, like for serve and serve preview, but all those do is set that one Boolean.

Speaker 2: Aside from that We activated leverages Django's template system and it just works. It really does. And that's pretty cool, I think. One of the most important features of WagTel is the ability to write rich text and have it be encoded in a useful way on the front end. Hi, some of you may have chatted with me about rich text things in the past, because I've certainly I've done enough about making this work in JavaScript in various ways. But this is really, really pretty simple. If you look here

Speaker 2: on On the right, you can see the arrow pointing at rich at the rich text function. That's the built-in template tag from Wagtail. So the the same code path that serializes rich text when you use that template tag in a Django template will also serialize that rich text to HTML. uh in the uh in the React uh templates. The next thing I want to talk about is this get serialized value class method. You can see that we're Registering this type basically for the field to tell reactivated how it how to handle generating types

Speaker 2: and uh serializing data when it encounters a rich rich text field uh field type. The this top method get serialized value is basically how you know given the Django object, give me JSON serializable data. And this on the bottom is the J is the code for generating JSON schema. And that's a little gnarly looking. Uh I know the first time that I saw this, I was kind of put off because JSON schema is verbose and pretty like that's a lot of code to just say hey we have an object with an HTML

Speaker 2: uh with an HTML property that is a string. But I would still argue, again, JSON schema being the way that this happens is a good thing Any tool that you like that is useful for generating JSON schema in Python, you can use it here. And that kind of uh be working with it within the ecosystem is the benefit that we get for having an a pretty common data interchange format at the heart of our type generation instead of some kind of custom magic. And so this is the last of the code examples.

Speaker 2: This is how we serialize stream field blocks. This client renderable block gets mixed in with the normal Wagtail block, uh Wagtail blocks of all the types And it's just just got some uh utility functions to again help with serializing data and figuring out the type. I cut out the get JSON schema function body because it's really long and really repetitive because it turns out that handling things that may or may not be null in JSON schema is a pain in the butt. But trust me, there's not much there worth looking at.

Speaker 2: So, okay, let's say that I've convinced you that all of this is easy. Why would you do this instead of sticking to something more established? like Next. js or some you know, w any of the other solutions for Headless. The killer feature in my mind of Reactivated is that it lets you put your React components into your Django templates. So you can see here, this is a Django template on the right. Include React. We pass in a string to say which template we're using, and we're passing in some data. And what happens then is reactivated, serializes that data, passes it to our node server

Speaker 2: or our vite server And gets back a string of HTML and includes it the same way that it would work with any other Django partial. You end up with server-rendered HTML that Uh the consumer, like whatever the end user of the website cannot tell was generated by two separate systems. That's a big deal, that's really hard to do with anything that I have seen aside from Specifically this library. This is the only one I've seen that has attempted this kind of thing, and it works. Moreover, you can also put Django templates inside of your React components. This client rendering barrier is the uh kind of

Speaker 2: the the counterpart, that's the word, to the to the thing I showed you before. You mix that into a block and the block becomes possible to be rendered with React You mix this into a block and it makes it calls in instead of rendering that block with React, it calls the Django render function, so you can render your normal just like a normal Wagtail lock, it gets that HTML and it passes it into the parent React component as an HTML string that it can then embed with inner HTML In my opinion, this is a really big deal.

Speaker 2: To give you guys some context as to why, the the anchor client at the lab is Temper, Temper Pedic. That whole family of sites is like seven different sites, all built with Django and with server-rendered Django templates And over the years, we've started embedding client-side React in order to add interactivity. Because that's the great thing about React. You can add a little bit of it. But we've had to but in order to keep server rendering the the sites that we need to server render, the parts of the site that we need to server render, we've had to jump through a lot of hoops because it's a pain in the butt.

Speaker 2: This is how we intend to solve that problem and gradually migrate our legacy sites, legacy content, some of which was written Twelve years ago, I think, as straight Django HTML. Theoretically, eventually it could all be It could all become React-based templates or not. It can still live together because you don't want to have to spend six months on a site rewrite to rewrite working code That doesn't like why are you doing this? Why are you wasting your time and money rewriting thousands of lines of working code? The whole reason that React was getting popular as I was learning web development is that it could be only a small part of your stack.

Speaker 2: The V and the MVC, as they said. And that has kind of gone away over the years. But I'm hoping that it comes back. And I think reactivated is part of how it could come back. So if you are considering making a headless site for a Django Wagtail project, consider instead Trying out reactivated and seeing about getting the benefits of a modern front-end stack without giving up. uh the the benefits of a of Django coordinating everything and all of those nice opinions and years of bug fixes that have come from that.

Speaker 2: Yeah, that's all I got. Uh are there any questions? For

Speaker 3: the folks who are managing your front end that are working in React, HTML, CSS, that kind of thing, are they now working in Django that React that way or is there some sort of handoff We have folks who are a little bit more savvy with Django trying to do it all.

Speaker 2: So the question was if we have the same people working in uh in React and HTML and the front end side of things as and also in Django on the back end side of things. And I would say that Mo yeah, most or all of the developers at the lab are full stack to some extent. There's a lot that focus more on the HTML and CSS side, but pretty much all of them like know how to add basic fields to a Wagtail page because that's not really that hard. And as long as the UK bits of like how to glue those two things together aren't too gnarly, they can handle that just fine. And uh some people, on the other hand, are more

Speaker 2: focused on database stuff or Django stuff. And again, as long as they don't have to get too into the TypeScript, it's not so bad.

Speaker 3: I think my question is for I I can I having a full stack developer comment comment at you uh something complex makes a lot of sense. but one of the benefits to having it split is that you can have Dational CSS and JavaScript people work entirely in the ecosystem, not getting hung up with Django This seems like it could be better or worse is the reason why I ask. If you're adding a little bit of complexity and you get some benefits, but you're adding something that is still a little shaky to Django, which you're solid

Speaker 2: Compared to the our Next. js sites, I would say there have definitely been fewer times that our front-end developers have had to dig into like weird API behaviors and stuff. Because oftentimes those the front-end specialists, a lot of times they focus on how to make a component look right and work right, uh, which is a very different part of HTML and JavaScript than how to load data performantly and asynchronously and yada yada, like without messing up server rendering. And because Reactivated handles relatively more of that compared to

Speaker 2: uh certainly any other um Any other integration between like Next. js and uh and Django that we've been able to build because Reactivated handles more of it in a more reliable way than our own, you know, built by me so I can say janky ass solutions. Uh we've we've There have been fewer times when we've had front-end devs blocked from making the thing look right and work right because of weird data issues, definitely.

Speaker 4: Um I'm curious, um obviously your focus totally on React, but like I used React that used Q I written stuff I'm curious if there is any potential benefits you see for like making adaptive product networks. Like I've seen integration guys view it I don't care for templates I know for graph is been doing the story so I'm just kind of curious like how much the integration was kind of like custom to it the app where it's plugging in a doc script.

Speaker 2: Um You know, earlier I I would have said that it a lot of it is very particular to React, but um Now that it we are that instead of having our own ES built configuration, we've we're basically just a byte plugin. I could easily see slotting in a different front-end library. The thing that a template based uh like Angular or View um front end slice would have to you know, you would have would have to be worked out is because of how React handles prompts, it is very like passing data in and it's like typing That or establishing the type of that data is like pretty straightforward.

Speaker 2: I'm not so familiar with viewers felt. Um, but I know in Angular it would maybe be a bit more of a pain in the butt to get that into your into your template layer. Uh but maybe not. Like again, it's a Vite plugin. It could probably be modified to work with other things. Just because of the name, I suspect it's going to say React only. You know, it would be a little silly to call it Reactivated and then you're using Angular, but Uh there's no reason that you know the reactivated project is open source and there's no reason that it couldn't be forked to work with view or Svelte or anything else, I don't think. All

Speaker 3: over used to be on the quite overteen requirement

Speaker 2: I have

Speaker 3: pretty new so I I

Speaker 2: No, definitely. I would like to look into that.

Speaker 3: I like to agree. I really understand what the process of people

Speaker 2: I mean, it's I imagine at some point you have to figure out how to serialize your data. I'm sure both of these projects have done that in one way or another. Oh, the question to repeat it for those on Zoom was if I was aware of Carl's library Django render. uh which was for a fairly similar thing, just rendering uh React using uh in as a Django template kind of thing. And I was not. All right. Well, if there's no other questions, then uh thank y'all for listening. Oh sorry, I didn't see you.

Speaker 4: Because uh I I think Server dame overloader, rendering and

Speaker 2: All right, so

Speaker 4: this do it uh this do it uh program that um within the server and the client I I think that 's what I want Why is the new approach and what the rendering process? What is the proposal that seems to be there?

Speaker 2: So the question was uh there are some uh detriments in terms of like cost of server hosting and cost of server time to rendering everything on the server. So why do we do that? I'm just going to answer briefly because the answer to that with something like Reactivated would be the same as why would you do that with Django? Which is uh SEO and performance benefits to having HTML that looks right and more or less works right out of the box before your JavaScript loads. There are lots of uh Google, for example, as one example, has

Speaker 2: has said, oh, our our crawlers understand JavaScript now. You won't get hit by SEO if you don't server render. It'll be fine. Demonstrably, from some sites that we have hosted, it is not always that fine. Server rendering does deliver benefits in terms of performance. and SEO. And any performance problems that in terms of server load that we have had, we have been able to solve with caching. And that won't work for every type of application or web page, but for the kind of stuff that we're building, it's the right trade-off. Yeah.