Wagtail CMS: How to Add Template Fragment Caching

Hello, welcome back to another lesson on learning Wagtail. In this video we're going to be talking about template caching. Now there are several different ways to cache a website, and a lot of them are covered in other documentation and all sorts of different places. In this video I want to talk about template caching because for a lot of people this is the biggest, quickest win you can add to your Wagtail website without having to install another service like uh ElasticCache or Redis or anything like that. Now what template caching allows you to do is you can uh let me find a file here Just bear with me. So I'm gonna open up templates and maybe I'll open up home and we've got a home page here Template caching allows you to basically cache an entire template if you want, or a fragment of a template.

And the fragment part is actually really, really powerful here because a lot of times we don't want to cache an entire template. Especially if we have an authenticated user uh logging in, maybe we're showing different uh different data in the header or in a navigation or in some sort of slide-out menu or slide-out bar. Uh we don't really know every situation. But I think the quickest way to really get started with speeding up your website is to add template caching. So let's go ahead and get that started now. I'm going to CD into my website. Pip and shell. And I am going to python3 manage. py run server Open up my browser, go to localhost, port

8000. And as you can see, this is our regular, somewhat ugly website at the moment. Now we don't actually have a lot of use. for template caching at this point. Our site is very, very small. We don't have a lot of queries. Now template caching where it gets really powerful is uh queries or processing power. So anytime you need to uh maybe render an image Or anytime you have, when you open up your debug toolbar here, you have uh SQL queries. I would say anytime you have more than like 50 or 60 on a page, it's it's time to start thinking about caching. Now if you're not familiar with caching, the idea of caching is that someone lands on your website or an event happens and that event runs its process once. And then for everybody else, forever on until you tell it to stop caching, it will always run the answer.

So it's like if I told you to multiply 9 times 9 times 9 times 9, You'd have to think about that a little bit. But if you already had the answer and the next person comes up and says, hey, what is nine times nine times nine times nine? You would say, oh, you know what? I don't even need to figure that out because I already have the answer. I'm just gonna give you the answer. So that's what caching does. So I think in this lesson we are going to actually cache our blog listing page. I think this is a good place to get started with it. uh just because this might actually have the most amount of queries on any given page that we have so far. Yeah there's 28 on this one. And maybe we'll cache our header as well. The first thing we need to do though is we need to open up our editor.

Open up dev. py. Usually we open up base. py, but we're going to open up dev. py And this is because we only want template caching to work on our local development right now. Now if you want to apply template caching to your in-production website, simply open up production. py But because I'm running this on localhost, production. py is not running, uh, dev. py is running. So I'm going to throw it in here. All you have to do is switch it over to production. py. Now, we're going to add basically just one line in here. Well, not one line, one dictionary. It's called caches. We're going to give it a default cache, and this default cache. Also a dictionary, and we're going to say the backend should be Django.

core dot cache dot backends. file based dot file based cache. So this is your Python path and So it's looking in Django, core, cache, backend, file-based, and then it's looking for a class to run. So we have file-based cache in here And uh then we need a location. Because this is file-based, we need to tell it where to store the cached files. So we're going to give it a location. And this one is How am I gonna do this here? Let's copy path. And so this is where my website lives right now is under users, Caleb Tollin websites my website. I'm going to create one more directory in here called cache, and that's going to live beside blog and contact

flex and all that stuff. You can literally put this directory anywhere you want, as long as it's easy for you to find in the future. For me, that is right in the top level. directory here with all the other applications that we have. Now I'm going to save this and my terminal restarts. Everything is okay. And when I Refresh my page. We're going to see that nothing happens. Actually, something did happen. There's a typo in here. What did I typo in here? Django core cache backends? It's supposed to be plural. File-based. Uh file-based cache, that should be okay. And Django restarts and we refresh our page. Okay, there we go. Uh you can see that our queries actually went up by two

and then went down by five. So I mean this is going to be sort of all over the place because Django already has a little bit of caching built in. So on this page we have 26 queries. That's what we're going to work with. And our goal is to get that down to as as small a number as as possible without actually wrecking our application and making it too hard to maintain. So what we can do is we can open our blog listing page and what we need to load is this word called cash. That's it. Or as I like to call it, Johnny. Johnny Cash. And then what we can do is we can cache entire sections of our page. So where do we want to cache? We want to cache maybe this entire blog post and this entire blog post.

And there is a page two, so maybe we want to cache those as well. Now there's two ways we can do this. We can cache this entire for loop. Where are we here? For posts and posts. So we can take this entire section. and we can cache it, but that's not going to work with pagination because that for loop is going to change. So what we need to do is we actually just want to cache This section here and then this section here. And if anyone was to ever land on page 2 or page 3 or page 4, we would cache this section and this section So it doesn't have to do too many more lookups. Now I actually don't know how much of a gain we're going to get from this one. Our website doesn't really have Um enough data traversing through the database, like the Django ORM is not doing enough work to really make caching applicable.

But it's still an important thing to learn, so we're going to learn it. Uh okay, so let's jump into this. We have a for loop in here, and we just want to cache this section here. So we're going to use the cache template tag. How long do we want to cache it for in seconds? So let's cache it for 604,800. I think that's a week in seconds. If I remember off the top of my head correctly. And then we want to give this a name. Uh so let's name this. blog post preview and let's also give this a secondary ID that's not spelled right let's give this a secondary ID uh something like Self-ID would be the page. That's not what we want

because we want this to be unique, so we're going to use post. id. Now the post ID is coming from this loop, so I'm going to indent all of that. And end my cache. So what this is saying is anytime that your page loads, take this section We're going to cache it for one week in the equivalent of seconds. We're going to call it post post post preview blog post preview. And we gave it a custom post ID to differentiate this from all the other ones it's going to diff that it's going to cache And what I mean by that is we don't want this one section here caching four times, or in this case, on this page, two

times Because it's going to show the exact same thing every single time and we don't want that. What we want is we want this one section to be cached, and then we want this one section to be cached. And what differentiates those is this additional argument here, this post. id. So that's what makes this unique. They're all called blog post preview, but they all have a different post ID. So this could be post ID one, post ID two. Post ID 3 and post ID 4, something along those lines. So let's go ahead and save this and I'm going to refresh my page here. And we see that nothing has happened. However, if we come up into our cache directory, we now have new files in here And it can't even open this.

Let's see what it looks like. Yeah, we can't even open this. This is a. dj cache file. This is a template fragment cache file from Django. And We don't need to know what it is. Django can totally read that on its own. We just need to know that these files exist. So now when I refresh this page, my queries went down from 26 to 20, and it will consistently stay at 20. No matter how many times I refresh. So this is now cached. If I go onto page two, this is going to jump up again, 24, because there's a couple queries in here. And I refresh and it jumps back down to 20. So this is a perfect example of me, the first user landing on page two of a blog, not having it cached, and it said, Oh, there's 26 queries.

Okay, so go and work out all this stuff, and then go and cache these sections. And then next time Say that I'm user 2, 3, 4, 5, or 6 or anyone after user 1, basically your Django and Wagtail site are going to say, oh, you know what? We already processed this. Let me go and look that up I'm just gonna see if we have that file on hand. And if we have that file, I'm just going to completely skip doing the queries, and I'm going to show you what we have in that file. Now this is a really really powerful concept because some sites get really really big. Now again our site is not very big. However If your site has a lot of menu items or if your site has menu items in the footer, we can reduce those as well. And especially when you start working with larger sites, larger, more complicated sites with a lot of different types of data, different types of pages.

Even if you're using dot specific at any point in time, which is a subject that I've covered thoroughly in this series, or even if you just have a mega menu, one of those Huge menus that drops down has more menu items in there, and you're using some sort of custom menu system, which I've also covered in a previous video. You don't want your page having to do 100 queries or 200 queries. You want your page to be loading nice and fast. Now if you're wondering Caleb, why are you talking so much about queries? Well the truth is that queries are actually very time expensive It takes time for your application, your Django and Wagtail site, to go and talk to SQL Lite or to talk to Postgres or or MySQL or talk to any sort of database. That database then makes a request for data, sends that data back Django then parses that data to make sure that it's actually valid and

usable and then renders it into the template. Basically we're saying do all of that once and At the end of it, just save this little section here. And when that section is saved, you don't have to do all that work ever again Now I mentioned we are going to uh clean up the menu here, not cleanup, uh we're going to cache the menu here. So let's open up uh I think it's base. html. Maybe it's in header. html header. Nope, it's in base. html. I still haven't split that out. Uh okay, so for item in navigation menu, items. all. So this is where our our nav bar comes in, right in here. Now I can cache each one of these individually, but because this is not really going to change and it's the same on every single page, there's no content in here that's going to change unless I decide to change that menu.

I'm just going to cache this entire section. So Cache again I'm gonna cache this one for one week and I'm gonna call this one navigation. This one does not have any additional arguments it's just called navigation So let's save that. Remember we have 20 queries on this page. Let's refresh. And it's saying we didn't load it. I'll just show you that. It says invalid block tag on line 48. Cash, did you forget to register? Load this tag. Actually, I did. I forgot to load this tag. Now let's refresh. Okay, so I've got 20 queries on this page But if I refresh, jumps down to 18. Which is pretty nice because now our page is going from 26 queries to 18 queries.

This is making pretty good gains in terms of percentages Uh, do we have something for a footer? Social media. Do we have social media settings? Where are you? Yeah, we have site settings in here. So this is going to check our site settings. We have an entire section here. This is our footer. Again, because this is not going to change. What I'm going to do is I'm going to cache this for again one week. in seconds, and I'm gonna call this one footer. Grab that stuff and cache. So again, it's that first time that it's loading, your queries are not going to change, but then you hit it up a second time. And all of a sudden my queries drop by one more.

So that was not really a worthy gain. That was just one query. And the header was only saving us two queries, but this page itself was saving us six queries. Now if you're wondering about how to cache entire pages, you can cache your entire base. html if you wanted to. Because you're operating a content management system, you need to change content all the time, you're actually not going to see a lot of uh the changes because If you preview a page but everything's already cached in the template, it's going to show you the cached stuff. It's not going to show you your previewed stuff. It's not going to show you your previewed content. Now there are a few topics I would like to go over here. The first one is should you cache static stuff? Anything that doesn't have a query that's not running any sort of processing in the background. And I would say honestly no, don't don't worry about that.

Caching this? Don't worry about caching this because every single time you run cache, when Django runs this template, it says, oh, okay, so there's a cache here, I'm gonna look for footer. That also makes a query. It stores it in the database and says, oh, I'm gonna go and look for this file. So instead of looking for the file itself, it looks for looks it up in a database, looks it up in your database, and then looks for that file from there. So it's a way of keeping track of the files, which is why we have two links in here, but we only gained one query. Or we have three links in here, but we only gained uh two cached queries. It's because for every cache section that you have, it's also adding one more query. So this is not the most effective way ever, but if you have 60 queries on a page, what's better? Serving 21 queries or serving 60 queries

What's even better is serving 20 queries instead of 21 queries, but you can really go down a rabbit hole of caching and that can be a whole world of pain. So uh maybe just Don't worry too much about the small gains, just worry about the larger gains. This is where template caching is really really powerful because again you can cache entire sections of a site. And you don't have to worry about installing Redis or anything like that. Now what would happen if I changed this menu up here? So let's go in here, admin. We have snippets. We have a custom menu in here. Let's change that main menu. Let's add something in here. Let's go down and let's add Wagtail Docs

And let's go to https docs. wagtail. io, open in a new tab, save Now the problem with caching here is that it's cached, so it thinks it doesn't need to change anything. So when I refresh this page, there's no extra nav item in here And that's because we cached this entire for loop. So when, when, when, when, Django loads us page. Where is it? When Django loads base. html, it says, okay, this whole section, do a query lookup, so lookup in your database, something called navigation. If it exists. Don't worry about rendering any of this stuff. Just throw the file in there. It's already been loaded once. It doesn't need to load twice or three times or four times.

Just load what we've already saved. And that's exactly what it's doing. So how do we get rid of this? Well there are Wagtail caching packages that will get rid of caching for you, but what I want to show you is uh I want to show you a way how you can actually delete it yourself. There's two ways. There's two ways to do this. One is you can go into your cache folder and you can delete all of your. dj cache files. You can see that they're hashed, they don't have names. It works on a hashing system, so you don't have to worry about naming or anything like that But maybe you're on production and uh you don't have an easy way to uh delete all of this cache or you only want to delete certain cache. Well I'm gonna show you another way to do this. Uh not pipenv, I want to python3 manage. py I'm going to run shell plus

dash dash i python. If you don't have shell plus or iPython, you can just run python manage. py shell and that will get you into the Django shell. Alright, so we are in this shell here, and I apologize, that's going to be hard to see near the bottom of the screen, so maybe I'll make that a little bigger. I hope that's okay. And we need to from Django. core. cache import cache. And so that just imports our cache function. That's all that does And then we can run cache. clear. And as soon as I run this, it's going to delete all the files inside of my cache template, or my cache directory rather. And so when I open up VS Code, we can see that there's nothing in here anymore. So if I exit out of here, do I really want to exit?

Yes. Python 3 manage. py run server. And when I refresh my page. There we go, we have Wagtail docs up here. So all we did there was we said, oh okay, I changed something in my navigation, but it wasn't it wasn't taking effect, and that's because you have caching in here. And again, just be very wary of this because caching will cause this kind of headache for you. It speeds up your site, but it is also a tricky thing to deal with. So we updated the nav and on the site we said, oh, okay, well our link is no longer in there or was never in there in the first place. How do we make it show up? And we went into our Django shell and we just deleted all these files. Now again, if you wanted to, you could delete all of these files on your own.

Now another good place to cache is uh Basically any sort of file lookup. So where are we using a file lookup? We've got an image in here somewhere. Image, image, image. There it is. It's right in front of me. So we're already caching this one. This image tag Wagtail is basically saying load the image. Here's the image that we want, fill it 250 by 250, call it blog image so that we have access to it in the rest of the template. A good idea is to cache these as well, if you have them in a for loop, just because this is also going to have to do some sort of lookup. Even if it's just a file-based lookup, which is still very fast in Python, it's one less thing that it has to do. Now if you only have one of these.

. For instance, like a Twitter sharing image or uh a Facebook sharing image, it's not worth it. You want to make big gains with template caching. You don't want to worry about the small gains. The small gains aren't worth it. So that is template caching in a nutshell. There's a little bit more to it, especially if you wanted to uh if you wanted to show This page, maybe admins don't get cached. Maybe admins get to see everything that's live. Or you know, maybe you're on a blog detail page. You don't want to see the cached version of this page, you want to see the preview version because in here I should have just clicked that edit button. If you click preview, it's still going to show you the cached stuff, so you want to make sure that it's not showing you the cached stuff. Now there are ways

Around that, we're not going to go it in we're not going to tackle that in this video because I've already talked your ear off to death about template caching, but There are ways to just delete certain parts of cache automatically. Now I'm just hopping back over to dev. py. What I am going to do is I'm going to add this cache directory. I'm going to ignore all the DJ cache files in a git ignore. And I'm going to comment this out. So if at so at any point in time you wanted to add template caching to your site, you could add it by simply uncommenting this dictionary here. Now the main takeaway behind template caching is page speed. If your site is loading slowly, it's probably because it's making a lot of requests to a database, most likely. Not always, but most likely. And databases are fast at performing a lot of things.

Uh but going to and from a database is like driving to and from your workplace or your office 25 times in one day. It does not make sense. So you reduce that down to uh possibly one query, and all of a sudden your site is a lot faster. Alright, so I think I am done talking about template caching. This is a good way to gain performance speeds with your Wagtail website. Caching here, this template caching is actually not a Wagtail thing. This is a Django thing. And so this is one way to speed up your site. There are several, several, several other ways to do it. This is just one of them, and this is template caching. So thank you for tuning in and listening to me talk about template caching. My name is Caleb Tallin. I am an author at learnwagtail. com where you can find all of these other videos and other tutorials

And if you found this video was helpful, don't forget you can share, you can subscribe, you can comment. And as always, I will leave the code to the git commit down below in the description