Create a full stack, reactive website in Django without JavaScript

Speaker 1: Hello, I'm Adam Hill and I'm here to tell you about my journey to find a mythical creature. Based on the titles of other talks for this conference and the number of questions I see while lurking in the Django subreddit it appears that others are looking for this mythical creature as well. Now many believe it doesn't really exist, but similar to my search for a friendly Yeti, I'm pretty sure I've at least seen a glimpse of it. So follow along with my field notebook while I document what I find. That is one mythical creature, but not exactly what I was looking for. The mythical creature I was searching for allows Django developers to build a modern reactive website without writing any custom JavaScript.

Speaker 1: Don't get me wrong, JavaScript is just fine. Sometimes it's even not that bad. However, I have run into a few roadblocks trying to integrate a JavaScript front end with Django There is the paradigm switch, not to mention the syntax differences from synchronous Python to asynchronous JavaScript. There is the high potential for duplicated business logic between the front end and the back end. There's the task of creating a REST or GraphQL API solely for your front-end. There's also the performance hit when the front-end framework is parsed, especially with slower machines or network connections. And then you have to make sure that your site is SEO friendly.

Speaker 1: Plus, there's the age-old keeping up to date with JavaScript build tools. So I could have just given up on my dream of finding this mythical creature, but I'm pretty stubborn. I just couldn't escape the allure of less complexity, a simpler architecture. faster development time, and let's be honest, less code. It sounds like a wonderful land of rainbows and sunshine. Especially when some intrepid explorers have already found similar mythical creatures. For example, in the land of Phoenix, LiveView provides front-end interactivity using the back-end language Elixir. Because of the robust support for asynchronous processes,

Speaker 1: it uses WebSockets to communicate back-end changes to the front end. In the Landa Larabell, LiveWire provides a front-end framework using PHP and regular old HTTP calls. It is extensive documentation and screencast, demonstrating all the things that it can do. In the land of Rails, Hotwire made a splash when it was released. It provides front-end interactivity using Ruby and WebSockets to communicate back-end changes to the front-end Some explorers even want to traverse across different lands, all the lands even. HTMX isn't tied to a particular server-side language and is a lightweight way to provide interactivity without writing any custom JavaScript.

Speaker 1: Highly recommended. It's very fun to use. But this is Django Khan, so we should talk about the land of Django, right? So let's get cracking. First, I'll briefly go over some code for a sample Django and Vue. js integration. Then I'll show a few Django packages that can create a modern interactive website without writing any custom JavaScript. The packages we'll look at are Sock Puppet, Reactor, and finally Unicorn. But first, we should talk about Vue. js. So I realize that there are lots of ways to integrate Django with a front-end

Speaker 1: framework. So the specific code doesn't matter too much. This is more to show an example of what might be required when integrating with Django. React would have different details, but the general gist would be the same. The first thing to realize is that most of the time on your local machine you have two processes running for your site to render properly. One for the Django backend and one node process for the front end. This is the typical Django dev server. It provides an API, usually REST for GraphQL. It also provides the beloved Django admin. And lastly, Django serves a static HTML page, although sometimes Node will handle that as well.

Speaker 1: Let's look at the code required for the backend. Starting at the URLs, we can see that Django is serving the index and Django REST Framework is providing an API route with the to-do viewset, which you can see right here. Let's look at that viewset. It contains a query set property that returns all to-do models from the database. Serialized with this to-do serializer. The serializer uses this to do model and outputs the primary key and the task for the to do. Serializers are also where the validation of business logic would occur in the Django

Speaker 1: Rest framework. So Django Rest framework creates a nice interface That lets you interact with the API. You can see that here. If I click here, you can see we don't have any to-dos yet, but we could add one if we wanted to. Let's look at the actual site functionality. So I'm going to start yarn, which will be on 8001. Let's try something out. Nice. It seems to work. Let's uh check out the front end and see what is involved to build this straightforward example. Here is the index page that is served by Django. You can see it's pretty minimal and contains a div

Speaker 1: element where the application is mounted with the ID app. Now the real magic starts in main. js. where you can see the router, store, and render method. Let's look at the router first. It will stipulate what URLs are handled by view. You can see for this example, the root URL is handled by a component named to-dos. Let's look at that to-do component. Now it looks kind of like HTML, but it has some fun new attributes. For example, this V model. is used by view to bind the input element to some task data. If we look at this button

Speaker 1: The click attribute will bind the button to an add to do JavaScript method with an argument of task. And here is how to check for any to-dos with VF and to loop over your to-dos with v4. It is all in the excellent Vue. js documentation, but it's definitely different than the server-side rendered Django templates. At the bottom of this component, there is a reference to the store. It is the central location that retrieves and creates to-dos. It will call this get to-dos method Which is specified here. That in turn

Speaker 1: calls another method in a service layer which is here, where finally it'll call the backend Django API. Again, this overview isn't meant to explain all the specifics of how Vue works It's more to show a typical setup when integrating a front-end framework with Django and all of its many complexities. I haven't even gotten to choosing and keeping up with a JavaScript bundler, deciding how to host the back-end and front end, or picking one of the many options to authenticate users. So that all sounds very complicated because it is, but there are some benefits to this approach.

Speaker 1: It can be useful to split up the responsibilities of a team. And it also provides a contract between the front end and back end to communicate to each other. And let's be honest Choosing a mainstream front-end framework that is well documented and supported allows for easier hiring, an ecosystem of libraries that work well with the framework. and the ability to find at least some answers to your pleas for help on Stack Overflow However, if the typical ViewJS integration is any mythical creature, maybe it's a Hydra? There definitely seem like there are a lot of pieces to wrangle and try to keep track of.

Speaker 1: And unfortunately, I am not Hercules. So I kept searching and I found a few other options for Django developers who don't want to wrestle with that additional complexity. The first Django library is Sockpuppet. It was originally inspired by the Rails library stimulus reflex. Let's see how it works with the sample to do app. Alright, so just like the Vue. js example, there is a to-do model with one field task, which will store all the things that we have to do. Let's fire up the dev server. As you can see, it's using ASC and channels underneath the hood for all the web

Speaker 1: sockets. And let's look at the website. It looks pretty similar to the Vue. js example. Let's try it out just to make sure that everything is working. Awesome. So let's look at the code and see how this is all implemented. If we go here, we can see the view. It looks like a class-based view and it inherits from the template view. And it's wired up, just like any other class-based view is, in the Euros up high. The git context data method should look familiar for anyone who has used class-based views before. This is where the initial state for the view is defined and passed into the view context.

Speaker 1: You can see that I'm getting a list of all the to-do objects. and storing them in this context variable to-dos. Let's look at the template and see how it gets rendered out. It looks like a Django template. You have if statements, you have a for loop here, looping over each to-do and our to-dos list, and we have the regular curlies to render out the variable Let's look at how we add a new to-do into the database. And that uses those reflexes that are part of the framework. Let's go up here inside of this input. It looks like a normal input. And then there is this attribute data-reflex. KeyUp is the browser event.

Speaker 1: To do reflex is the class that gets instantiated for that reflex. And new task is the method that gets called on this reflex. We can also look at this button right here, and it looks very similar except we're using click as our browser event. Same class for the reflex and then add to do for the method. So I guess we should look at the reflex to see how that works as well. That's right here. Remember that new task method that gets called on every key up in the input? Well, I'm just taking the value from that element. storing it as variable and then setting it to the session. When the addTodo method gets called, it takes new task out of the session

Speaker 1: and it saves it into this to-do model which stores it in the database. Since we are already rendering all the to-dos in a list, when a new to-do is saved in the database, it will show up automatically in that list, which is pretty magical. Let's look in the network requests to see how it works behind the scenes. Here is our developer toolbar. We see this is a WebSocket request and we can look at all the responses. Search the cave. And we see right here, add to do, and all the new task events that get fired. Sock Puppet is well documented and brings reactive capabilities to Django.

Speaker 1: while leveraging the battle-tested stimulus. js library. Another package that brings interactivity to Django is Reactor. It uses WebSockets and its own custom JavaScript library to communicate between the front end and the back end. Let's look at a sample to-do app for Reactor. It uses the same to-do model with a task field just like all the rest of them. Let's start up the Django development server. Again, you can see that it uses ASCII and Django channels for the WebSockets, which are used to communicate the state changes from the front end to the back end. Let's try it out in the browser.

Speaker 1: Should look pretty similar to our previous iterations. And it looks like it's working. Let's look at the code to see the details. So Reactor has the concept of components that encapsulate some interactive functionality. You can see this component template tag on the index page. You can think of the component template tag as an include, but it's got some extra, you know, magic. Here it's referring to a component named X to do. Here's the template for it. And it's going to look very similar to the Sock Puppet demo. You can do all the regular Django

Speaker 1: template functionality like if statements and for loops that you uh that you would expect. Let's see how you listen to events. Here's our input. There's a name and a value with new task. And notice this at sign and key up So this input element is listening to the key app browser event and it will call this task method in the back end Let's look at the button as well. It's similar. It is listening to a click event and it will call add to do in the back end And this prevent is a modifier that basically means prevent default.

Speaker 1: Now let's look at the backend code. Reactor looks for components in a file named live. py. And here it is. You can see the component X2Do is here. It subclasses from component. The first thing to note is this mount method. It loads the initial state of the component when it is rendered. Here we see I'm getting all the to-dos from the database and setting them to a to-dos instance variable. The second thing to note is that the methods that we specified on the front end will all have receive underscore prepended to them in the back end So the input element had task as the name of the method, and so here it is as receive

Speaker 1: underscore task. Here's the value new task that got sent automatically and we're setting it to an instance variable on this class. Here's our add to do. That was for the button click. And you can see that I'm taking the instance variable and creating a new to-do model, saving it into the database. And then I'm just wiping out our input element so that it's clean and I'm re-grabbing all the to-dos from the database. Similar to Sockpuppet, the list of to-dos on the front end will automatically be updated when this self. todos gets set.

Speaker 1: Let's look in the network requests again and we can see the WebSockets. You can see that this is a web socket, and here are the responses. And as I type, you can see The WebSocket requests go back and forth, and that's awesome to see how well that all works. Reactor is a simple-to-use library that quickly lets you prototype adding interactivity to your Django projects. Both of these Django libraries are super impressive. But they weren't quite the mythical creatures that I was searching for. So while bored at the beginning of the pandemic,

Speaker 1: I stopped searching and finally created my own project. I named it Unicorn, as an homage to the unofficial pony mascot of Django, and because of how magical it felt while prototyping the code. Let's see what our sample to do app would look like in unicorn. You just need to start up the regular Django dev server. And you'll see that it is just using just the regular Wish T. Let's go over the browser and see how it looks. Well this looks super familiar. Let's try it out. And it looks like it's good. So let's check out the code to see how Unicorn approaches interactivity.

Speaker 1: The model is the same as before? Well let's look at the index page. So you can see that Unicorn has the concept of components, and here I made a component named to do. And we can look at the template. Rendering the current tasks is similar to our other Django solutions, where we have access to our normal Django template syntax. We have if statements, for loops, basically anything you would want to do with a normal Django template is available. But let's look at how the input elements work. This looks pretty familiar, right? Here is the input element for our task, but notice the unicorn colon model attribute

Speaker 1: That will specify what field in our backend component will be bound to this input. In this case, the field name will be task. Before looking at the backend though, let's look at the button real quick This unicorn click attribute tells Unicorn to bind the addToDo backend method to the click browser event. Let's look at how this all works in the components view backend. You can see that backend views subclass unicorn view. Under the hood, that is just a subclass of template view. So the process of switching from a standard class-based view should be very straightforward. Remember when I said that the input element was bound to task?

Speaker 1: Here it is in the view. Note that I use types pretty extensively in Unicorn. to help specify metadata about the class attributes, but they are completely optional. Also note this hydrate method is called when the component is instantiated And for this example, I'm just grabbing the latest to-dos from the database so that information is as up-to-date as possible. And here you can see the add to do method that we're calling from the front end. It will create a new to-do model from the task and save it to the database, and then it just clears out the task. Let's look at how it all works in the browser. So as I type

Speaker 1: You can see that there's a post, there's a HTTP post for every event that gets fired. And there's some JSON inside of here in the request that gets sent over. And then in the response that comes back. So if you see all these posts and you get a little concerned about all the network traffic, Unicorn has a few other tricks up its sleeve. Models can have a lazy modifier that only fires an HTTP call when the element loses focus, or a defer modifier that only calls a method when an action event fires. So let's look at that real quick. So in here I can make it lazy. And now

Speaker 1: nothing is happening. Until I click outside and lose focus, and there goes the post. And let me show you the defer real quick as well Defer is useful when you only want to send an event when an action is fired. So you can see that no posts are firing. Even when I click outside and lose focus, nothing fires. But when I click on the add button, we see our post. So I've slowly been adding features to Unicorn over the past year, like polling, loading, nested components, and tight integration with Django database models and query sets.

Speaker 1: And because the code is all server-side, unit testing the components is straightforward. And things like authentication, they just use the regular built-in Django functionality. Unicorn provides everything I need for a modern website quickly and without any extra complexity No custom JavaScript, no additional APIs, no JavaScript bundling, no additional processes to run while developing on local, and no requirement on Django channels. It's quick to add into an existing site and make it feel a little more interactive. The projects I have mentioned most likely won't supplant the need for React or View anytime soon

Speaker 1: However, in my wanderings, most websites don't need anything that complicated. Most sites are well served by a server-side Django website. with a little bit of interactivity sprinkled on top. If you truly need an SPA, then by all means definitely stick with one of the major front-end frameworks. I think I speak for all the projects mentioned though when I say that they are only as helpful as the community around them. I'd love for you to help any of these projects any way you can. Starring their repo, trying them out on a new project, creating issues, updating documentation, even sponsoring the developers

Speaker 1: Well, hopefully that was informative and a little bit fun. I'm Adam G. Hill on GitHub, if you want the sample code, slides, or the transcript. And I'm the same username on Twitter. I'd also like to thank my coworkers at the Motley Fool who gave feedback on this talk, and my lovely wife Lynn, who did all the art for the slides.

Speaker 2: I just have a question. Question.

Speaker 3: Sure, sure.

Speaker 2: Uh do you think something like this would suit uh Like a project in production or something like that? In production. So yeah, exactly.

Speaker 3: Yeah, so um the three um projects that I talked about, so Sock Puppet, React Bear, and Unicorn. Um they are all great for prototyping um sort of newer projects. I don't know if any of them have been used in a big production scale uh sense. Um I'm still working on a a bunch of features to make uh unicorn sort of production ready. Uh but I use it for all my side projects. So it's been used uh but not not for a lot of scale

Speaker 2: So you don't have like uh a clear idea on uh performance wise uh on on how it would behave uh with a a heavy load

Speaker 3: No, not really. It's mostly a side project for me. If you think about all the HTMX uh talks that people did uh during this conference. It's very similar um thought process behind it though. It's you just do Ajax calls when um things are changing and you update HTML fragments from the from the back end and then you swap out on the front end. So it should be pretty similar to any other, you know progressive enhancement uh framework. If anyone else has any questions, I can try to answer them.

Speaker 4: Yeah, I have one one question. Uh uh I saw that uh uh that you need to enarrate from Unicorn View. Um what kind of magic happens uh behind in this in this base view?

Speaker 3: Um there's not there's not too much. Uh it's mostly setting up the template tag and um And providing a a few sort of like abstract base methods that get called, like the hydrate and there's a mount method. But it is basically a template view for all intents and purposes.

Speaker 4: Thank you. It looks really great. I'm taking a look at the at the website.

Speaker 3: Awesome. Thanks.

Speaker 5: Uh yeah, this this looks super cool. Um and and that was a great talk. Thanks. I'm curious. I'm interested in I have a use case in mind right now that I think would could be a good fit for this. I had two questions. One, like if I had a model with Like 20 fields in it. Is there any way around doing 20 post requests like every time like like binding each input to I guess maybe the defer thing handles that now that I asked that question out loud. And then the second question I had is like how how do you do uh UI feedback and validation and and that type of thing?

Speaker 3: Sure. So um there is there is support for Django models and and query sets uh in the in the component. So I try to provide you know some nice sort of um helpers basically. So you might be able to get around having all the fields defined in your component separately. For validation, I try to utilize the Django Forms framework. So it's very similar. So you can basically set a form class on your component and then it will use the validation just like a normal Django forms. use Django

Speaker 3: bits and pieces so it felt like it was part of the framework and not something completely separate. That was the idea. I think your other question was about UI uh feedback. There are some um oh gosh, what are they called in my Sorry I'm looking at my documentation site just so that I have the right so I have some um what I call loading states And dirty states, which are um are kind of taken from LiveWire, if you've ever looked at the LiveWire documentation, where I provide um a way to basically target a particular div. uh when things are changing in the back end. So uh I can basically like hide or show a div

Speaker 3: when that backend code is getting run. And the same thing for dirty states, which is basically like something changed, but it hasn't been persisted yet. So you can sort of like give some UI feedback in there.

Speaker 5: Thanks. Thanks.

Speaker 3: Yes.

Speaker 6: I do have a question So um you you put everything in the same components, but I was wondering if there was any type of signal uh that was happening in the in the back end uh that was sent to the front end. So if my to-do list was in another component then my input would my list uh still update and what if I have another browser open Will that browser also update once I add the list? And I guess that question works for all three frameworks, all three libraries that you showed

Speaker 3: Sure. So for Sock Puppet and Reactor, they use WebSockets. And so they're across different browsers, they will get updated. I know that's true for Reactor. I think that's true for Sock Puppet. For Unicorn, because it's just, you know. um HTTP posts uh and we're not using WebSock. Um it won't get updated in different browsers. It was basically a design decision that uh For me, like I wanted to solve the 80% of use cases and having browsers get updated in real time.

Speaker 3: is really cool as a demo, but like I don't see that very often in my in my work. So I sort of didn't feel like um trying to solve for it. For your first question about uh components signaling signaling to each other, I do support nested components. So If you have like um I have a to-do example, which is basically there's a list of to-dos, and then uh you can um inside of the table you can basically like delete or edit and so there's a it's not a signal but I can listen to your child components and you can call up into your parent.

Speaker 3: And so that's sort of how I've solved some of the like interaction between different components. Hopefully that makes sense.

Speaker 6: Yes, thank you.

Speaker 3: Yep.

Speaker 7: Thanks Adam for the talk. Um I'm just trying to figure out in my head how um uh how that compares to Hotwire and H HTMX. Um and you you kinda it kind of feels like you feel like that that's um a a prog progression beyond those technologies or is is that uh misstating

Speaker 3: um so I've only read through the Hotwire documentation. I have not used it um in production but My understanding is that they use WebSockets to sort of do their interaction. So it's a little bit different. And I think they have Like turbo sort of can be used in there too to sort of hijack clicks. So it's sort of its own its own thing. For HTMX, so the way that I think about it, HTMX and Alpine are really good as like a starting place if you just want to add some simple things in. And then

Speaker 3: view and react are like on the other side of the continuum where you basically like have to rewrite a bunch of things. to go into their you know world. Um and I'm sort of trying to position unicorn sort of in the middle where it's a um easy to add in Provides some interactivity that's a little more tailored to Django as opposed to HTMX, which is very generic and can be used with anything, but it's not completely like rewriting your whole site. to use a to make it into an SPA. So I think there's a middle ground there which again for the like the 8020 like I think benefits a lot of sites without

Speaker 3: You know, too much work.

Speaker 7: Okay, I yeah. So that so so um Vue and React have uh virtual DOM, right? Whereas Whereas HTMX and a hotwire explicitly don't. You sound like you're doing a bit of more magic in the in the template on the front end than HTMX, but y you probably don't have a virtual donor, no Am I right there?

Speaker 3: Yeah, no no virtual DOM. I'm using Morph DOM, um, which is a JavaScript library that basically you can you pass in your HTML and you pass in the HTML that you want it to morph into and it will like um smartly change your HTML from one to the other Most of the libraries that I've seen, so LiveWire uses Morph Dom , LiveView uses MorphDOM. I don't know if Hotwire uses it or not. But most of these libraries that don't have a complete virtual DOM implementation, they seem to use Morph DOM.

Speaker 7: Okay. Yeah. All right. Thank you.

Speaker 3: Yep. I can answer any other questions. If anyone has any. Or I'm updating the chat right now with the answers to a few people who passed in text.

Speaker 8: So uh can I ask you something? Uh in the socket puppet and reactor solutions, uh a web socket is created per page, per form Which actually is the case?

Speaker 3: A WebSec? Are you uh sorry, uh a WebSock is created per page? Is that the

Speaker 8: per page s so so you can get updates for uh for uh as many forms as a page contains

Speaker 3: Yes. Um from my understanding of yeah, looking through their code and their demos, that's what it looks like. Yep.

Speaker 8: Okay, so maybe a user can have If if it has many tabs of the same site, it may have multiple web sockets per user. So I I I don't think that this is very uh performant

Speaker 3: Yeah, I I think that's the way it would work. Um I think for any solution that uses WebSockets, it would create one WebSocket per tab, right?

Speaker 8: Okay.

Speaker 3: I think so. So then yeah, I mean the challenge is scaling out your WebSocket connections. Right.

Speaker 8: Yeah, yeah. Okay. Thank you.

Speaker 3: Yeah.

Speaker 4: Hello.

Speaker 3: Hi.

Speaker 4: Hi, can you hear me?

Speaker 3: Yeah, yeah. Is your mic working now? Yeah. Yeah. Yeah.

Speaker 4: Yeah, yeah. So let's say the replaced HTML contains a click or it contains a model. Um will it automatically bind like it's listening for a mutation uh mutation listener or like How does that work?

Speaker 1: Yeah, so um that's a good question. I don't know if I've tested it explicitly, so I don't want to give you the wrong answer. What happens when the so when the pen then the page renders, I basically look through all the the elements for things that start with unicorn colon. And what I'm not I'm not positive about is when I swap in the HTML, if I go through the new HTML um again. So I don't want to tell you the wrong answer.

Speaker 6: Sure, because that's a very important use case when you're

Speaker 1: Yeah. So um as long as the HTML Doesn't change, uh, it'll work just fine. Um what I'm trying to figure out is if you like add in new um unicorn colon attributes. after uh a transition if those will get picked up which um I'm not positive

Speaker 3: and so I can I can try to make an example I'll make an issue in in my repo and then um and I'll see if it works.

Speaker 6: I'll really appreciate it because I've been trying to figure out which uh component framework to use for this Uh I'm using HTMLX, uh I'm using uh a bit of turbo and I'm using some web components, catalyst. And uh it's kind of a mix. So uh this seems to be solving a lot of use cases for me and uh I I'm gonna try it out and this is one important one. because if uh the HTML is swapping uh then uh it has to work. Uh same with turbo if let's say I'm using turbo and I just rent the turbo renders the page then uh if unicorn is not listening then it's not gonna be able to um pick up the attributes.

Speaker 1: Right. Yeah, that makes sense. Um the other thing that I should probably add in is making me think about it is um a way to uh re -grab all the attributes manually if you want to. Um so I should probably add a hook in for that. But um yeah I have I have a couple side projects with that kind of mix HTMX and Alpine and um I wasn't using Turbo but Unicorn is like the hope is that it sort of alleviates all, you know, like trying to mix a bunch of things together.

Speaker 3: Yeah, that's that's what I saw. And I I I mean I really like the part where it's integrating well with Python because all the others you have to do some kind of tricks to get them work.

Speaker 1: Yeah

Speaker 3: Yeah, the the the goal is that it's pretty simple to add in and um and sort of gives you some benefits right away without uh jumping through a bunch of hoops.

Speaker 1: So

Speaker 3: I don't know if I'm quite there yet, but I'm pretty I feel like I'm pretty close to it. All right, I think I'm going to

Speaker 1: jump out of this meeting.

Speaker 3: But thanks to everyone for showing up and uh listening to me Answer questions.