Tutorial: Headless Wagtail on Google Cloud

Hey everyone and welcome to this session on Headless Wagtail. I'm Freddie Mixel and I'm thrilled to be here today to share a journey into the exciting world of headless CMSs. Specifically, we're going to dive into a powerful architectural pattern deploying a headless Wagtail CMS on Google Cloud paired with a lightning fast static Next. js front end hosted on Elify. Now, you might be wondering why this combination. What's the big deal about static sites and headless CMSs? Well, that's exactly what we're going to uncover today and um We're going to look at how flexible performance and how great of a developer experience she'll have on projects like this.

So let's break down how all these pieces fit together. So at the heart of our architecture is our Wacal CMS, which is running on Google App Engine. This is where your content team or like your team, whatever, they manage all the digital experiences or content or whatever you have that you're going to deliver. And then Wagtail exposes this through the API. And this is sort of like the bridge between the front end and the back end of this site. And then on the front end we're using Netlify in this example , which is a static site host. where we can build an X. js front end and then serve that statically so we don't have to continuously hit the API every time someone wants to look at one of our pages.

You can see here we have two different flows sort of outlined. a developer deploys a backend change. Whenever this happens, all your static assets, like anything inside Wagtail, like JavaScript for Wagtail itself, any customizations, any uh assets that are part of Wagtail, they'll all get uploaded automatically through the storage configuration to Cloud Storage. And uh App Engine will build your your new version of your Wagtail site and deploy it on your link. Then from there your CMS users can upload content. In this example we're showing specifically if they upload an asset like an image

That also will then get synced over to Cloud Storage and the database will remember the URL for that. Then also developers could make a change to the front end and they would push to GitHub which makes it available to Netlify. In our example here we have a deploy page on the Wagtail site which allows us to hit a uh a webhook on Netlify to build the site. And I've also disabled um deploy like deploys from Netlify itself technically we'll go into that a little bit later but that way uh the only place things could get deployed are from

this uh Wagtail front end. And you know, so that kicks off the build, goes goes there to Nell Fi. So let's jump into looking at the GitHub repo. Okay, so here we are on the Headless CatTube GitHub repo. You can see the front end and back end code are both separated by those directories. And we have the Netlify configuration for deploying the front end, this README file, make file and a gignore. The makefile is pretty important, so we'll quickly go over that. You can use this to create your virtual environment with PyEnv.

So you just run make dev env and it will install this version of Python if you don't already have it, and then set your local environment. to that version and then create a virtual environment called cat tube env that you can then run py env activate cat tube env so run this make dev env and then run this to go inside your virtual environment. And then you can install all the dependencies with make dev update And uh then you can run make debt freeze if you want, if uh you want to, but it should already have everything in there. Um And then you can run make dev and that will run the backend.

Well actually I guess you'd have to run make migrate first to create your database. Then make admin will. Create your superuser so you can log into Wagtail and then you would run make dev to run the backend Some other things we have here, proxying the dB. So you have to update this value here with your um host name from CloudSQL but you can proxy your database so that um you're able to Perform actions using your local machine against the database, like running a migration, for example And we'll get more into that on how that works with this repo specifically.

We also have deploy CMS here. And that is pretty straightforward. It's just looking for this app. yml file and deploying to my project. So you have to update this if you're going to use this repo. with your Google Cloud project ID, which is just right on the main dashboard page of Google Cloud, if you're inside your project already. And then make static just um collects all the static files. Um this isn't super important to run locally, I don't think. Uh Spore. That's something that we do right before we deploy. We'll make sure we have all the static assets already up on Google Cloud. And then we deploy the backend, sort of part of that flow chart that we looked at earlier.

And going back to the Cloud SQL proxying. If you look in your dev settings, you'll see we have this config here for use proxy. So if you pass use proxy true make dev then it will pull out the storages and the database password from the production configuration file But it will pipe it through localhost here. Well actually I guess uh the proxy is already so you run the proxy in one tab with uh makes make proxy db And then your other tab you run use proxy true make dev

and that will pick up on that proxy database and this will just make sure it reads from it and Yeah, uses uh the storages also so you see all your assets coming from the cloud. Nice way to just also test before you deploy to production, um even though It is just the data and the assets, but you know, just to double check that those are looking good before you with uh the build of Wagtail that you have before you push it up. So that's a nice feature of this repo. Um and then well I guess um we could look here at the models. Um this is just a YouTube video page that I created

with some idea with some values that allow you to build out functionality for YouTube iframes just as part of the example. And you can see that all the field or a lot of the fields here are added to the Wagtail REST API. And uh that's how we're sending it over uh to Netlify whenever we build. So now let's take a look at the front end. You can see services here, video service. This is just pulling in the next public backend URL so that this will be an environment variable you'll have to set to the URL for wherever you deploy your

Wagtail on App Engine. I just use the App Engine generated URL with the project IE in it usually. But um you can you're free to also create your own URL for that. But you can see this is um hitting the pages endpoint with specific fields of video ID thumbnail full and since it is a page type We're specifying videos. youTube video page as the type so we get only those back. And then we just have these this page basic fetch functionality. with uh types that send back the videos to the front end so we can read those from Wagtail

when the page is being built So if you just go up here into pages, good example here is watch by video ID. You can see we have some hard-coded data. And then we are receiving a video detail here which has a video ID that we pass to the YouTube video embed player. And then our SSG data here from Wagtail. We call the video service git videos. This is just hitting the Wagtail REST API during the node build. and it maps over the videos and creates a page with a video ID prop on each page, passes the paths back

And then this is just get static props for each individual page. So we then take that video ID param and pass it. to video service git video so we can get the full video detail page once we are on that and that is what gets passed down in here So pretty simple path for getting your data into your your or whatever front end you have. This example uses Next. js. But you might be using Zolid or you know you could even just be maybe you have a Python Ginja um architecture somehow that you know you want to just have your CMS live somewhere else. You know, it really doesn't have to be uh a front-end

JavaScript framework for you to use uh a headless architecture, really. I've been on projects before where built like custom Python static site generators and stuff and we've technically gone a headless route. Um so You're not really totally restricted to JavaScript in this. This is just uh really the example. So now that we've looked at the code base, um Let's go back and actually look at the deployed site and then we'll look at it locally and see how to run this code base locally. Okay, so here we are and our app engine deployed

back end. This is a site called CatTube which just holds a video directory full of cat videos, um and YouTube like basically it's just a collection of YouTube videos that either cats would like or they're about cats. And you can see we have a couple things. You can have an override thumbnail if you don't want to use the thumbnail from the video. And then it only required field here is video ID, which is used in the video player. And if we go back, um you can see um it is a little bit slow, and we'll talk about that here in a second. um because I didn't use a huge database, but um you can see this one doesn't have a thumbnail um

and it's totally fine, just needs this video ID So this is our Wagtail dataset that gets sent through the API and it's read every time we click deploy here. It will refresh and reload the page and it sends a webhook request. to Nellify to build the site and you can see this nifty little message here shows us where to go so we can see our new deploy is building and it will generate all the static pages for the Wagtail site

and then we will be able to switch traffic to this. And while this is happening, let's just go to the site overview and if you'll notice in here in this production deploys window. You'll see it says failed, but um it's do not deploy is the name of the branch. Um that's just something that I I set like the main deploy branch with a name that doesn't exist so that this happens basically so that um you can't deploy from the interface here you can't trigger deploy or anything because it will go to a branch that doesn't exist. That way you can make sure people are only deploying from this deploy page.

and kind of allows you to use Wagtail's page moderation feature to um get the content out whenever you're you know like on your schedule basically instead of um as developers are pushing to the main branch We can see this has finished building and I have the lighthouse Tools added in here with Netlify you can see it's got a performance score of 98. So we can just click publish deploy here And that will make the current version of the site live. And we can go to it right here. So You can see this is all of our content from the Wagtail site. This is our custom thumbnail.

You'll see it's a different image than the one that shows here. By default we just use the YouTube video ID to basically like you just create the URL for YouTube and add in the video ID and it will you can use that to get different versions of the images. These are like the WebP versions. So that way it's in next gen format for Lighthouse. And if you go on the about page you can see uh the link to the repo into my GitHub if you want to connect. And these other pages are just some filler to make the site look a little bit bigger. This is the video detail page that I was showing you on the deploy step.

So if we go back over to Netlify to our deploy You can look at the build process here and you can see that it generated all these different video IDs. based on that query that we wrote um of the REST API. So it returned these different video IDs, so then it then fetched those full video detail pages for the other for the for these watch um pages. So that that's how that happens. So this page can load super fast, doesn't have to hit the database at all. And if you can watch this video , you can leave a pretend comment, you can

like it, whatever you need. And yeah, that's basically the gist of how everything connects together. So now we can look at the site locally. So here we are in a project in VS Code. You can see there's some VS Code settings here to help you out a little bit. Black formatter for formatting the Python with uh organization of the imports and it'll format on save. We also tell ESLint to only run at frontend and Any. html files we tell it to use Django HTML. In case you want to add any templates to you know you could add add HTML to uh alter just the Wagtail interface itself so if you end up doing that then this will

give you the correct syntax highlighting. And then you'll want to have two terminals open so you could run both projects. So To run the backend, um first you would run make dev env and that will create your environment. I've already done that and I've already activated into my thing pie and activate catube env, which I've already done. You can see that I am in the catube env right here So now I don't need to migrate my database because I've already done that. I can run make migrate just to show you that. You can see it looks into the back end.

Oh, looks like I do need to. So let's see make my creations. So it looks like I altered the thumbnail uh field on the YouTube video page. So let's do make migrate. So create example here. So now the database has been migrated, so I should be right good to go. So let's do make dev. And you can see the back end is now running. And uh to go to the fr so now now let's start the front end also, which before we go look at the

look at them live. So you're gonna want to C D and do front end. I am using Z shell, but I'm just gonna show um C D just in case anybody doesn't know how to do that. And then in here you'll if you haven't already you'll want to npm install. And I'm using node version 18. 19. 0 if you're using nvm you could hook into it using nvm use from the front end directory and then we'll run npm rundev That will boot up Next. js at localhost 3000. So we have our Wagtail site running at localhost 8000

and Next. js running at localhost 3000. So if we go up here, we should be able to load the site and you can see it is slightly different videos on here, so that's how you know you're getting the uh data from your local Wagtail site and you can see this is a different set of data runs a little bit faster so it's Nice place just to set everything up, test your model, and make sure it's all working before you end up deploying either a change to Wagtail or to your front ends.

Could be either or really. If you want to test like a certain configuration on the Wagtail side of things with something like a new change to the front end you're able to test in all different types of ways with this setup which is why I really like it. Okay, so the next thing that I want to show you is how to proxy the production database. So we're going to kill the back end and the front end here and we're gonna open another terminal and And we'll just go back um into the main repo. So over here I'm gonna run make proxy db.

And you'll see that connects to my Cloud SQL database. And then over here I'm going to use use proxy true make dev And if we do need to run migrations, it should tell us when we start up here. Yeah, so it looks like we do have a migration also that we need to run on the live site. So that's something that we can do right here. The way you would do that is use proxy true make migrate. And this will just run the it'll connect to the database over here, run the migration on the cloud. Alright, and then

um let's just cover deploying. So we 're just gonna make sure this version is also live up on the live Wagtail site so we'll just do make deploy CMS and that will run our app engine command uh to deploy or go gcloud uh app deploy command that deploys to app engine with our project and we can see that process happening here Okay, so I fast forward to there, and you can see it is just finishing up and uh it has finished. So One thing to note, if you look at your Wagtails site, you can see my URL deployed right here. So

you would just go to this URL that you have slash admin to log in. Um but if you see some kind of an error or something happening when you deploy, just copy this here and paste it into the terminal, click enter, and it will show you the logs that are happening. So if there's like a Python defending that's not working or you know whatever it could be um that is how you uh debug issues that you're having once you deploy your Python uh app to Google Cloud Okay, and one final thing that I want to go over is just all the stuff that you need to set up in Google Cloud in order to get this going. So you can see I'm in my CatTube project here that I created for this demo

And this is just what the main page looks like. This is where you can get your project ID that you use. So if you just search for this value in the repo, replace it with your project ID, you should be well on your way. The next thing we'll look at is just the two buckets that you'll need to create. And you'll you can search for these values in the code base also and replace them with whatever you decide to call year. buckets but this one is just for anything that Wagtail uploads so you can see it's like Wagtail snippets, REST framework, just all the stuff, JavaScript, CSS, things that Wagtail itself is using And then this is just our images and their conversions that Wagtail will do.

They get stored here. So you'll need these two buckets created And you'll also need a database. I used a pretty small database, just a Postgres 15, um, with uh two CPU, um eight gigabytes of memory, um it's and it's in the US central. Um that's the it's a single region. So there are better configurations that you could use, but this works good for me for this um particular project. I would uh consult with someone uh if you're not familiar with how to s like set up a database uh because this could get pretty expensive um

so make sure you're monitoring the billing tab here to make sure you're not using more than what it uh you know you're willing to spend per month And the next thing is secret manager. That's another thing you'll have to set up. So you'll find all of these different keys inside the code base and really you'll just need to add all of these just like this into secret manager under your project that you've created. And your account info, this is just the JSON that's generated from your service account. So you'll you'll also need a service account. that you can create

on the credential screen. And that service account JSON file, you just paste like the text value of the whole JSON object directly into a field and call it account info. Build hook, that's just the URL that Netlify gives you for deploying. DB password is the password that you set up when you're creating this database. And Django Secret Key is just a key that you generate that goes into your production. py file. This allows you to store it secretly on the cloud so no one has access to that value. So that is pretty much everything. Thank you all for watching this tutorial and

demo on Headless Wagtail, I hope. that you decide to build uh headless wagtail sites on Google Cloud also. And thanks for watching.