How Wagtail CMS StreamFields are Saved in the Database (behind the scenes)

Hello, welcome back. In well several lessons actually, we've been talking a lot about stream fields. And in this particular video, what I would like to do is sort of show you how a stream field is structured in the database. Now if you're not familiar with databases, this video might not be for you, but if you are interested in databases or if you are already familiar with sort of the idea of how they work and how they store data inside of a relational database, then this video might be for you. Now I'm using a Mac, but if you're using Windows or any sort of flavor of Linux, you can really use any program out there. All we need is some sort of program to go and explore a database. You can actually even do this from the command line. It's actually quite easy. But for the sake of being visual

and having something nice to look at. I'm gonna show you how to do this with a program that's called db browser for sq light So if you're interested in this particular program that I'm about to show you, you can go and get that one for free. Just Google this name and you'll be able to find it pretty quickly. So I'm going to open this. So this is db browser for SQLite. Now we do not want to create a new database, we just want to open a database. Now I'm going to open up Websites, MySite, and dB. sQLite3. And if you're wondering where I got that from, uh, if you're running the default Wagtail setup, it comes with SQLite database Where is it right here? DB. sq

light three. Uh you can't really see it as kind of shaded out It's not really a file you want to open and edit anyways. It's just a bunch of crud in there that you're not really gonna understand because it's computer language, and that's actually okay. Because we have a program to do that for us. Our website right now has 35 tables. A lot of these come default with Django. As you can see, we've got Django in there. We've got auth stuff. Some of these are Most of these actually are Wagtail. So for instance, we've got Wagtail documents, we've got Wagtail embeds, form submissions, uh group page permissions, we've got all sorts of stuff in here. Um a lot of these things that we haven't even covered yet.

For instance, uh Wagtail image rendition. We haven't learned how these are stored, uh, but if you're ever interested, you can go in there and take a look But what I would like to cover in this lesson, in particular, are stream fields. Now if I open up Visual Studio Code, I'm going to see I've got an app called streams and I have a file called blocks. py and we have for instance in that last video we have a CTA block with title text button page button URL and button text Now in a regular database, in a regular model even, let's do this. If we open up our homepage model, we're going to see that we have a banner title column, a banner subtitle column, banner image

column. So let's actually take a look at that first as something to compare later. So you remember when we were getting those SQL errors like, oh, you're missing the content column, for instance? Well that's because home underscore homepage Home being our app and home page being our model didn't have this column yet. And we can even see that we've got a banner title, we've got a banner CTA ID, we've got banner page image. We've got a banner subtitle. These are fields that we have explicitly defined. Now content is the stream field. Now if we Check this out. Down here we've got content, type is equal to stream field, and we have a list of tuples, and the only one we have in here is a CTA block.

So let's let's explore this one as a fairly simple one. So let's go to browse data. And let's select the table called where is home? Where is home? There it is. And so we only have one particular instance of this page. So we have a table with one row in it The page ID is three. This is our banner title. Welcome to Startup Life. Banner CTA ID. We don't have that. There's nothing in there. Banner image ID, so this reflects back to the Wagtail Images. image, which is This fella right there. And all that is is a foreign key. So a foreign key is basically just a number to another table. With her banner subtitle And we have a banner subtitle, Rich Text Field, even stores it as HTML.

It's not all perfect HTML because Wagtail's editor, DraftTale, actually stores things like links as a different type of anchor link. So it's not quite regular HTML. It has some extra stuff in there that it goes and parses later. But the one we what really want to look at here is content. This is our stream field. So we've got a type in here. Actually, let's do this. I'm going to copy this into VS Code and we'll take a look at it in there because I can make it bigger in here So I'm just gonna paste this in here and fun fact, stream fields are all saved as JSON format And I can format this and we can see, ta -da! It's JSON. That's all it is. So whenever I'm saying a stream field is a complex data type, it technically isn't

It's sort of just a huge string with a lot of JSON in it. I mean that's that's the simple way to think about it at least. Now we can take a look here. We have a list, we have an object, and inside that object we have type is equal to CTA. We've got value in here. So the value of our stream field that we made in the last video, and then it's also an object. Or if this is JavaScript, it's an object. If it's in Python, it's called a dict, a dictionary. And then we've got title is equal to the title, text is text, button URL, button URL. Yada yada yada. So on and so on. So this is how a stream field is set up behind the scenes. Your ID is where this becomes complex. Because Wagtail will take this ID and it will work with us.

If it needs to understand that there is a call to action somewhere, Wagtail will understand that. And that's where this becomes complex. But really the storage behind it is not that complex. But let's take a look at a bigger page. In fact, if we open up our about page, we have a bunch of stream fields on here now. So we should take a look at what this particular stuff looks like. Let's open up our basic page or our flex page and we have page ideas for we've got a title in here and we've got content. Now we've got all this content in here. So let's copy all of that and let's throw you into VS Code and format it. So when we look at this, well I guess when it's formatted and we look at it, it's not really that complicated.

Our first stream field, the first one to show up, is Title and Text. Where did we get that from? Well, if we are in flexmodels. py, title and text. This is where I said in previous video, in a couple previous videos actually, that Wagtail uses this title internally. And that's exactly what it's doing. It's using this internally. The value is a dictionary. The title is equal to welcome to startup life. Text is a bunch of regular text. So nothing fancy there and an ID. Then we've got full text. Full text with some lorem ipsum in there, and we can see that this is a rich text type. Not only do we know that because it's called rich text, but we can also tell because it has HTML in here. And so when we scoot

on over, where is a good example? There's got to be a link in here somewhere. Here it is. So Just a couple moments ago, I said that links are stored a little bit differently, and they are. So instead of a h-re-f, like what you typically see in HTML anyways, You are going to see ID, and this ID and the link type together says, oh, okay, well, this ID is to another page. So link to page ID3. And when the page renders, Wagtail says, oh, I'm going to unravel that. I understand that this is page ID and we're going to link to that one. And it's going to basically create the href in here is equal to, because that's what the home slug is. Now the reason that Wagtail and DraftTale, the editor behind Wagtail, does this

is because if that slug for that page were to ever change, you're not going to get 404s anymore. You're not going to get missing pages. Wagtail will automatically update that for you. Okay, so we've got full text, simple, rich text in there. We've got cards. So let's take a look at what these cards are looking like. So we've got these three cards in here. We've got an image, we've got a name, we've got some sort of description and a button. Type is cards, value, title, cards. And then just for the repeating card, so this is our list block, this actually created a list of stream fields for us. Now if we reference this one directly by opening streams blocks. py and we open up cards. Where are you? Cards.

Our card block is a list block and it takes a struct block. And basically we said In this list block, repeat this as many times as the user wants, and add an image, title, text, button page, and button URL. Do this over and over and over again for as many times as the user could ever possibly want. And that's exactly what it did here is we have an image that's a foreign key to image ID4 We've got a title, we've got a text, we've got a button page, so there's nothing in there. And in the template, if something is null, it does not return. So if we said if self. button underscore page, this would not return. And we've got a button URL. For Matt Smith, we can actually read some of this stuff.

So we can see that his image is image ID3. Text, you can actually see some of the encoding in here. This is how that shrug is actually encoded. That's kind of funny. I didn't do that on purpose, but that's a really good example. So this shrug here, it's encoded differently. And we've got Jane Doe, and then the entire stream field itself has its own ID. Below that we have CTA. So if we look at this in order, we have Title and text. We've got full rich text. We've got simple rich text. Cards and then CTA. So if we open up our page again and we go from the top. What kind of stream field is this? It's that first one, title and text. We've got title and text. What is this one? This one is a rich text.

That's a full rich text. The next one is a simple rich text. We can't tell on the surface, which is good actually, but behind the scenes it knows that it's a simple rich text. And that just means this one was limited. We said only have certain features, please. And then the next one is cards. Look at that. We've got cards. We've got a title. We've got all of our cards. Basically, these are just objects inside of an array. If you're talking about JavaScript, so that's how a JavaScript mind would think, uh, because this is basically a JSON file. Or in terms of Python, you have a list, and each list item is a dictionary. And then lastly, we've got a call to action. And lastly, we have a call to action. So you can see that this actually preserves the order of everything too. Alright, so in this video, we actually covered how a stream field is

uh is stored in a database. And the database type doesn't really matter too much. I mean I guess if you're using MongoDB it's gonna be a little bit different. But in a relational database, yeah, it's pretty straightforward stuff here. The content, all of our stream fields always get stored into one particular type of column, and that is the text column All stream fields actually get saved as one particular type of column, and that would be a text column. And all Wagtail does for us is it says, oh, okay, so you have a bunch of stuff. In here, actually this is a terrible terrible example. Let's let's do this. Go to the about page and edit We have all these different stream fields in here.

I want to show them in order, but what happens if you want to move them up and down? Like this. I'm moving something up and down. So it's really just Wagtail saying, oh, okay, I'm gonna save this order and I'm gonna save it as basically a JSON file inside of a giant text field inside of the database. And when we go to pull that out, I will then Re-reference anything that I need to. For example, any particular IDs or the type, any sort of foreign keys, it's going to take that logic and it's going to work with it. But we also explored that there are different types of columns in our database from things that we've already worked on. And so that The banner title on the homepage, for instance, gets its own column. But stream fields don't, and this is where it becomes more complex, is because stream fields are like one giant column.

And this is how Wagtail understands that there is a particular order and how we can move that order up and down on our page. So that is it for this lesson. If you don't understand this, that's a-okay. You don't necessarily need to understand this because Wagtail and Django will take care of all of this stuff for you. As always, I am Caleb Tullian. I am the author behind this video. If you like this video, please feel free to subscribe, click like, thumbs up, leave a comment below, share it with your friends. on Slack or WhatsApp or your Facebook groups, anything like that. And I hope that this video added some level of clarity to how Streamfields work behind the scenes.