Auto-save design considerations

Speaker 1: Are we live?

Speaker 2: That sounds great. Are we live?

Speaker 1: Oh yeah.

Speaker 2: We're live.

Speaker 1: Ooh. Hey

Speaker 2: Sage.

Speaker 1: Hey Devo.

Speaker 2: What's up?

Speaker 1: Good, I'm good. How are you?

Speaker 2: I'm pretty good. I'm happy to see you again this week.

Speaker 1: Yeah, of course. As always.

Speaker 2: What are we discussing this time?

Speaker 1: Mmm, well last time what did we discuss? We discussed the roadmap items for the upcoming release, right? And I think we had a few items there and well the development for the upcoming release has already started. So yeah, I was just looking into Autosave, which is the next big thing because I I've wrapped up well mostly wrapped up the development for block settings that's currently in review so um if you're watching this and you'd like to have a look uh how we implemented it or give some feedback. Um it's already on GitHub. Uh we haven't merged it, but yeah, it's there. So yeah, I was just looking on

Speaker 1: uh looking at the next thing which is autosave. Yeah, that's the PR.

Speaker 2: Yeah, that's the PR. Um I was trying to catch up with you and here I am. Yeah, due to be reviewed

Speaker 1: soon.

Speaker 2: And now autosave?

Speaker 1: Yeah, um autosave. Um Matthew has done some work on the backend, mostly um to support JSON responses so you can call the save endpoint or like uh basically the Django create an edit view for the uh pages or snippets um in the virtual admin using JavaScript with um accepts application json content type so that instead of giving you a an HTML response, it will just give you a JSON response. So you can call that with um JavaScript code to do saving in the background.

Speaker 1: And yeah, now I started thinking about the front end and the fact that we have Quite a few different approaches to do essentially the same thing that is detecting changes in the edit form. And now that we are about to add yet another thing to the client-side code for the editor, I was thinking maybe we should try to unify the code a bit and just use one approach that is um have we used for all those different things? And what are those different things, right? So we uh on Wagtail In the edit form, you have the unsaved changes warning when you start editing the page and you haven't clicked save.

Speaker 1: Yeah, I think we might be able to demo it with the static build. It might take a few seconds before it starts kicking in.

Speaker 2: Ooh, the demo gods are not pleased.

Speaker 1: No, I think you just need to wait that right.

Speaker 2: Oh yeah. There's a cooldown Yeah.

Speaker 1: So um yeah, you might want to add some more changes. Yes. Yeah, so that thing. Um basically it detects changes in the form by uh so this one is done it's it's a no it's a code that has existed. It's uh So the code that powers that feature has existed for like I don't know many years at this point. Um but it uses event listeners for key up change on the form event and it's been migrated to stimulus but I think it was originally written in a combination of plain JavaScript and jQuery Um

Speaker 2: JavaScript and jQuery, my love language.

Speaker 1: I mean back then it makes sense, right? It's everyone's favorite.

Speaker 2: Yeah.

Speaker 1: And but also in WhiteTale you can add things like, for example, stream field blocks or inline panels where we will add more fields to the form, kind of like form sets, and when you do that, we At the moment we I don't think we dispatch a change event. So when that happens, um which means uh Um those new fields will not be uh well the addition of those new fields will not trigger an uh a change in the form. But of course if you if you start typing in um the fields that will also trigger it but yeah so on the other hand we have those event listeners but also we have a mutation observer

Speaker 1: that um looks for new elements in the form might not be the most performant thing. I I haven't um benchmarked it actually but yeah uh that's the way it has been for many years and it's been working. Just fine. So that's one implementation of like detecting changes in the form. But then we also have the preview panel, which Um you probably can't use the oh wait. Oh yeah.

Speaker 2: Well yeah.

Speaker 1: But So the way the preview panel works, because it's uh it automatically updates based on any changes in the form, and it works by having an interval that periodically checks for changes in in the serialized form data and if there are any changes it will send the form data to the preview endpoint basically But yeah, it doesn't use any event listeners or mutation observers. It just periodically checks for changes in the form data. So but yeah, I I feel like before we move on to implementing autosave, because we're basically gonna need the same thing where we

Speaker 1: um listen for changes in the form and then we want to update the page right by by sending the form in the background basically. But yeah, uh before I implement that, I think we can just reuse one of those implementations of detecting the changes in the form and maybe just unify them and just use that single implementation and then maybe when we detect the changes we just dispatch an event that will bubble up to um the document and then any code that needs to know when the form has changed can just listen to that event. So yeah, I'm still deciding um which one we want to keep and which one to remove, basically.

Speaker 1: So I was wondering if you, Thibaut, um as the Wagtail front-end expert, whether you have any opinion.

Speaker 2: Can you remind me where the code is for the um dirty form check?

Speaker 1: So it's now in a controller called unsaved controller.

Speaker 2: Oh yes, of course Um yeah, I have um fair bit of front-end expertise, but Wagtail is quite a big code base and um We've had lots of refactorings underway led by our core contributor LB, core team member LB , to stimulus. So I haven't actually seen this code in um In quite a while. Yeah, okay. So this is the unsaved controllers logic to retrieve the form data.

Speaker 1: Yep.

Speaker 2: And then there is the actual checking, I assume. Yeah. initial form data, current form data. Yeah for people who don't code much js uh this here actually is um method call because we use getters and setters on this controller.

Speaker 1: Yeah it's kind of like Python properties.

Speaker 2: Yeah exactly Um yeah, so I I think what's tricky here, Sage, is just uh both options probably have quite clear um drawbacks. So Have to figure out which set of drawbacks is acceptable, I suppose. I'm a bit surprised that here we have some exclusion logic. I guess it's specifically because in the context of those um unsaved warnings, we don't really care if those things change. Whereas for the preview we do care? Or it's just an oversight?

Speaker 1: So it's not exactly that we don't care, but rather we care about it so much that we detect changes to the form and the comments as two separate things even though they're still within the the same form element but we want to display Uh so if you try to add a new comment, uh it will add like you have unsafe change edits and comments, I think.

Speaker 2: Yep.

Speaker 1: Yep

Speaker 2: Okay.

Speaker 1: Whereas the preview, it doesn't care. So

Speaker 2: and so for autosave , I'm guessing we want to have um as many of the possible changes. as possible all bundled into this.

Speaker 1: Exactly. So it's uh closer to the previo the way the preview controller works, I would say

Speaker 2: Yeah. So I was a bit confused because um I guess even with um the current setup there are some fields you could add that won't serialize as far as So if you have a file input, like not a Django one even, but just like the HTML file input, fundamentally you can't still like this like this and Django it's a struggle for us as well. So how much do we have to be aware of those things or is it just like it's a trade-off that's autosave won't work with those fields.

Speaker 1: Oh that's a good question. So for for previews specifically, um the backend code at the moment doesn't handle um file fields because we store the form data in the session which essentially works by serializing it into the session object right and we can't serialize um files. So we at the moment just ignore file fields. But for the unsaved controller, I think there's a custom logic that basically just takes the file name, the size and the type just to detect if it has changed, right? Because you don't actually need to send that data.

Speaker 1: anywhere and for autosafe I think um I was thinking we would try our our best to support it by Um so once we detect that hey the the file has changed, right? Um the actual form submission, it uh I will I would try to mimic like normal HTML form submission as much as possible, which means we would also send the files as blobs in in and uh in the request.

Speaker 2: Yeah. I remember um a while ago also researching which other common fields might be skipped. This was from a way way older implementation of this code, but there were also issues with um I think radios or select multiple and checkbox select multiple. Um Which I'm guessing might have been fixed since or we would definitely fix as part of this new autosave work to make sure that we capture all the types of uh

Speaker 1: oh yeah that's a good point i think that might be why i'm using url search parameters to serialize the form which I think it does support multi multivalues because the the default form submission basically works, but like I think it's called like www form URL encoded. So it's the form data is URL encoded. Whereas in this case we are just stringify it using entries, which I don't think it would take into account multivalue.

Speaker 2: It was a much older implementation. So I think um The safest will be to um just check what happens with um more fields of those types.

Speaker 1: Right.

Speaker 2: Um that's type of thing, yeah. That's just like like the type of knowledge you have to have of um the relevant DOM and J. SAPIs. I much rather trust testing than remembering. Okay, so yes, this issue about different field types that you said we have some plans on. how to make it work better and potential bugs that we can definitely investigate more uh and then you mentioned so coming back to stream field um If all I do is um press this button here to create a heading block, then obviously there's not gonna necessarily be um relevant events, no

Speaker 2: necessarily new form data, or is the form data changed just because the hidden input that contains the stream field data has been changed

Speaker 1: the form data will change but at the moment it will not trigger a change event

Speaker 2: change events yeah yeah okay So yeah the trigger uh the the options are then uh event listener or set interval uh why not both What's the risk in sending out too many save requests?

Speaker 1: I mean it would definitely be spamming the backend, right? if he yeah if he set the interval at uh low enough value.

Speaker 2: Because those are those are quite heavy requests, right?

Speaker 1: Kind of, yeah, we need to validate the whole form Essentially. And if you have like a big page that could potentially be heavy.

Speaker 2: Okay. I was thinking even even before that, like just the network it's a heavy payload but you're thinking of the processing costs in a Django server

Speaker 1: so yeah it's I think it's both definitely yeah

Speaker 2: Okay, yeah, so the problem with its done server side is even before we could detect if there's been a change. we'd have to do the whole processing of the request for it to be converted into the relevant I guess Django models or Python at list so that's quite heavy so we do the check the comparison client side so that we only send the request matter and then I'm guessing once that goes through to the server there's gonna be a cost of um Saving the data if it's uh actually a change? Is there any kind of uh like

Speaker 2: before after has it changed check? Um on the server side

Speaker 1: i think there is but we save a new revision regardless so

Speaker 2: Oh, we save new revisions for now.

Speaker 1: Oh, um, for other things we will overwrite the revision. Yep.

Speaker 2: Oh okay. Yeah, because that was a big um design trade-off, I suppose. suppose for us that if we introduce autosave we don't necessarily want to have a thousand super granular versions in this kind of UI.

Speaker 1: Yep.

Speaker 2: Um

Speaker 1: so it will create one new revision when you first enter the page.

Speaker 2: Oh when you first start your kind of autosave session. Yeah, okay. So even that's we want to be really careful to avoid doing that if we can um so that if you just land on the page and you don't do anything it doesn't create an autosave session.

Speaker 1: Yeah. And that's also why um at the start of this demo we had a delay for for the unsafe changes to kick in because um Yeah, the the the shape of the form data could change while the page is being initialized.

Speaker 2: Yeah Yeah. Definitely slightly shaky foundations to build the new autosave logic on, but um nothing that's just real reality of working on a long-term project like this.

Speaker 1: And for people watching, I I can just guess that some of you might be thinking, why not use some kind of like WebSocket? or something to just just send the the fields that change right why do we not do that

Speaker 2: why do we not do that sage

Speaker 1: Um yeah, well the thing is if you want to set up something like WebSockets on the Django project, you need to bring in additional dependencies and setups like Django channels. and potentially other packages and because Whitetail is at the end of the day it's just another Django package, right? And if we were to use those technologies, we would essentially be imposing those additional requirements to our users. And Whiteel has not needed that for many years. And if we were to add that now, I think.

Speaker 1: It would also mean people who want to upgrade to the latest version will need to set up all the additional things that you need to get that running.

Speaker 2: Yeah. So I changed what I'm screen sharing just because I think Sage is definitely in the cards in some way in in the future. Because we do want to have way more. like collaborative editing features than just autosaving. I guess the autosaving the idea is it's super useful if you're working on your own on your page more confidence in the content being saved as you enter it, but also super useful when you work with multiple people because then they can just refresh their copy without having to coordinate with you whether you've hit save. or no so you can just work on the same thing at the same time if you coordinate like that um but i guess the future is definitely you'd see the updates on your own side

Speaker 2: when you're working with other people without having to refresh either. And at that point you can probably still do some kind of polling, but it starts to feel like it'd be so much faster to see the updates with WebSock.

Speaker 1: Yeah.

Speaker 2: So at some point we will probably have to say okay those features are only for people who are on a ASGI server configured this or that way, right? It's just we wanna

Speaker 1: Yeah, I'm not so sure. I mean it's it's it's too early at this point. It's just uh you know I'm not sure we want to maintain Like detecting oh you have it set up or you or you don't and then you have to like hide some features behind feature flex or something like that. Depends on how easily we can separate the two or um make it work, I guess. Because I I can imagine that being horrible to maintain like oh one thing broke in one um mode and it works fine in another

Speaker 2: Yeah. At this point are we on save draft without page reload or save draft automatically? Well we're discussing here sounds like we're on save draft automatically at the same time as without page reload. So the other option would be just have people click on the button, Sage. And you don't have to detect.

Speaker 1: That's a good point, actually. Yeah, but I I don't think the the thing is there's there are still cases where you do want well at least I mean the thing is Um we also for the MVP of other safe, we're not going to add the ability to Immediately render any form errors.

Speaker 2: Yes.

Speaker 1: That's

Speaker 2: it's time to l leave the happy path that's easy to design for. of success of all requests at all times and navigate the complexity of what happens if uh at the very least there's an validation error

Speaker 1: So yeah, I would rather keep the buttons working as they are for now.

Speaker 2: Yeah, yeah. Okay. So we do both at the same time. There'll still be a button and there will not be. Live form validation. Just there will be form validation, but it will be refresh

Speaker 1: So there will I think we will let you know that autosave has been paused, for example, because of validation error. And then if you know what is causing that validation error, like maybe you're missing a required field, and then you keep on filling that out. and then there's no longer any validation errors, then the autosave can continue. But if you don't know what exactly because the thing is um with Wiketail, because it's built on Django, you can have custom Django validators. and that may not be immediately clear to your editors, so you need um the errors to be server rendered. And because we don't yet have that mechanism, we will need to do a full page preload.

Speaker 2: Yeah. I popped this example of um multiple people working on the same page at the same time for reference. So here I uh logged into the CMS on two different accounts and on the other account I saved a refresh of the page and that's the kind of like prominent warning you would have for autosave, I would assume when there's validation error that we can't auto save anymore, something pops up on your screen that calls attention. You don't necessarily need to get to it right now, but you should definitely take notes and probably do something something about it?

Speaker 1: Yep.

Speaker 2: Um okay. Um yeah trade-offs um but there's no way we'd do all those things in the one release So makes lots of sense to go incremental. So back to your question. I guess my feeling is we need to do something that's um is the least error prone. So I definitely feel like set interval would be nice to have at some point just as a fallback but it really shouldn't be a small interval if we did that uh to keep the requests costs limited um But it still feels like it could be nice as like a confidence pullback

Speaker 2: that there's no because you know I I want to be careful that uh we have something that's resilient against uh us failing to identify some command scenarios. It'll be a really sad, sad time if we miss something and some custom widgets out there failed to autosave completely I feel better if it's like at least once every minute, for example. And then for the more like freshness of the content, we can can have more regular events-based checks, or am I overthinking it

Speaker 1: I can see where you're coming from. I just I'm not sure. Hmm. So essentially whenever right before we send the request to the server, we do want to check whether anything has changed. Right. Um the checking will happen right before sending it. So yeah. Um

Speaker 2: But the sending will not happen if uh there's been no changes.

Speaker 1: Nothing has changed here So you're saying we're keeping the the actual triggers uh for that checking. um as both using event listeners and intervals with uh with the intervals that's do something that's you know not as immediate.

Speaker 2: So I guess in my mind I haven't seen the preview code. Where's the interval?

Speaker 1: Um is there a set interval? Surely there is. I think

Speaker 2: Not in this file.

Speaker 1: What wait? Oh uh it's uh it's using uh set optional interval. Uh I have a utility because um You can disable the interval by setting it to zero in the Django settings and I wrapped it into a utility.

Speaker 2: Oh, and this is um yeah, Django City So in order for me to find the value, I will have to go to the Django doc Sorry, the White Tail docs.

Speaker 1: Yeah, uh White Tail auto update preview interval at the at the bottom.

Speaker 2: Thank you 10 seconds.

Speaker 1: No, that's the editing session.

Speaker 2: Oh my bad. Um 500. Wow. Okay. So I guess here these two for example they have very different interval values because this one the consequence of it being too low is Is bad front end performance? The consequence of it being too high.

Speaker 1: It's not as responsive when you edit.

Speaker 2: Yeah, and this one there's the server costs. So we want to be careful not to hammer the server. So in this case, we want to be careful not to hammer the server. But we definitely want the checks to be frequent enough that uh you don't lose work.

Speaker 1: Yep.

Speaker 2: I guess to me it sounds like maybe what we want is um frequent interval like the preview that is just then the bounces the calls so you don't call the server every 500 milliseconds So that solve both problems.

Speaker 1: That's kind of how the preview works.

Speaker 2: Yeah. What debounds do we have set? for the preview.

Speaker 1: At the moment I actually set I think I use the same value for the debounds. So you essentially have to wait double the interval

Speaker 2: So that feels quite low to me as far as like hammering the server.

Speaker 1: Yeah.

Speaker 2: So I'm guessing maybe Is the preview cheaper to render than saving a revision? Not really, no.

Speaker 1: The preview does not need to so with the Edit, you have to create or update a revision and potentially also updating the page instance to set things like the latest revision. um forwarding key and like last updated thing and maybe adding an audit blog right for example But with preview, you literally just validate the form and then put it in the session object.

Speaker 2: Okay. Sounds to me like it's definitely cheaper. But not like a order of magnitude cheaper. So yeah, maybe I am overthinking it a bit and we can just go with um quite small interval times two for a debounce like you did for the preview.

Speaker 1: I think there's also the question whether we want to keep this unsafe message because ideally if autosafe is happening you shouldn't need to worry about unsafe changes right

Speaker 2: Yeah, well I'm guessing that um you know it's um shouldn't be present if the autosave is on, but if the autosave is off, we would need something like this

Speaker 1: Yeah. So maybe we put the maybe maybe maybe we put the messaging for like autosave is paused, for example, due to validation errors or someone else saving a new revision. Can't do that uh footer instead Oh I I think uh I discussed this with Ben, our designer, yesterday, and we were talking about there's the possibility of people uh due to them being used to Wikto so much, they no longer see that message anymore.

Speaker 2: Oh yeah.

Speaker 1: Like they would just ignore it and they didn't realize that the messaging has changed due to other save so we might still add a small indicator at the top.

Speaker 2: I mean to me I have I haven't discussed this with our designer Ben in um in a while but I expect that eventually this would disappear and move all into the header area Because at least eventually you won't need to have a safe draft button anymore. And if you will, it will only be contextually. if autosave is off for some reason and the other actions like definitely aren't frequent enough to warrant always being around this that's why they're already in a drop down and also because this is pretty pretty valuable real estate on a long form like this like you wanna

Speaker 2: really be able to see more of the screen But like I'm talking, you know, a few releases in the future at the very least. Uh so yeah, back to also safe being off. The only scenarios are validation fails and And someone else has saved or can you also turn it off for your site if you wanted?

Speaker 1: Oh, that's a good point. Yeah, maybe we want to have the option to disable it. I I don't know. If if we if we're not adding that option, then I mean I would be fine with it. Then I'm just saying that uh the those two like um validation errors and someone else saving a new revision, those two sh I think should be the only reasons why autosave might be paused.

Speaker 2: Yeah.

Speaker 1: But yeah if you add an option to disable it, then we're back to current state of affairs.

Speaker 2: Yeah. I mean I can definitely see how You could imagine someone out there who might not want to use that feature and would like us to disable it. But uh It does take some like maintenance burden for us when you introduce those options. Um so unless there was someone who does manifest themselves like that I don't really see like a clear case for why you would not want this. And on the contrary, I do feel like fundamental aspects of how a product works. box like this. Um it is in our interests that it's predictable from one white site to another. Like you don't want to go to one version of Google Docs where you have to

Speaker 2: do control S all the time and one where you don't have to you'll just never remember to do it when you on the one that you you do need it. So that's the kind of feature you know it's gonna already be tricky to navigate because of um Wagtail sites having multiple versions over time, but you know five years in the future where everyone is in the autosave world i'd just rather everyone was in it um like for for the preview for example we we can disable the preview auto refreshing right and you can even disable previews But like why would you do that? Anyway, um yeah, so I feel like we can we could proceed without it being disable able

Speaker 2: until someone asks for it with an eloquent enough use case basically. And it feels like performance-wise, yeah, maybe it is a bigger interval than the preview but it doesn't have to be orders of magnitude bigger and um yeah definitely the unsaved controller needs some rework as part of your changes so we can probably stop having it based on event listeners and just based on interval For the triggering of the check.

Speaker 1: Yeah, so we also remove like all the mutation observer stuff as well.

Speaker 2: Yeah, exactly. That feels like you know obviously there's clear trade-offs here for performance, uh, but it feels to me like um It would probably be better performance-wise to have um a check no matter what every second let's say that's uh that check will be like somewhat performance sensitive but I don't think it's tremendous compared to having even listeners triggering those checks way more often when you're typing.

Speaker 1: Yes. I think that that was my FP when I was developing Wagtail um with stimulus we have like automatic logging of stimulus actions and because this is hooked up into um as a as a stimulus action for the event listeners you constantly oh I guess you you run the you run the build uh production build If you did then

Speaker 2: Yeah, I ran pressure and build this way.

Speaker 1: You don't get the logs. But yeah, essentially every time you type, you'll get those logs. So I would like to get rid of that.

Speaker 2: Yeah. Should we um disable them for the um dev build as well?

Speaker 1: I I d I don't think so. They can be useful sometimes. It's just uh at the moment I don't think there is a way to like filter out the events that you don't want to be logged. It might be possible if you just overwrite some methods in the application class. But yeah, if there is a way to filter certain events so that, you know, ones that we actually care about. That would be useful. To

Speaker 2: to do to do. Here we are. Debug. I mean um our contributor LB has got um good success out of um making improvements to stimulus itself in the past so um I'll definitely take a look at open issues and PRs for it and if it's not already a feature see what what they might be thinking. Um okay. Uh yeah, I I feel like definitely from a performance perspective, logs notwithstanding. uh it's the typing that you want to be not on a performance hot path so I really like the set interval I feel like that has some opportunity also for us to switch this

Speaker 2: if we wanted to um What's the what's the JS API when you want to defer something? Request idle callback?

Speaker 1: Something

Speaker 2: about?

Speaker 1: Don't think I've heard of it.

Speaker 2: Oh, let me check. Request idle callback. Is

Speaker 1: this a fancy new thing?

Speaker 2: It's a fancy new thing like five years ago.

Speaker 1: Is it kind of like request animation frame?

Speaker 2: Yes, so request animation frame. Like was the I guess animation specific way to do this where you want to defer something until future next frame to avoid doing things that are more frequent than necessary, uh really useful when you need it. And request idle callback is um uh about the the cpu usage i believe um just deferring something until um both the frame is over there's no like performance sensitive rendering and um yeah i don't think there's guarantees that this is called per frame or anything like that

Speaker 2: Um I'm surprised it's still not supported in Safari, but it looks like things might have changed at some point.

Speaker 1: Yeah, the TP is green, so I guess it's been merged.

Speaker 2: Yeah. That's really nice. It's the kind of thing that, you know, you can adopt it if you really have a need for it. But it's kind of annoying to have to code both the pathwork. it's available and where it's not.

Speaker 1: If request idle callback

Speaker 2: images I have written some code like this here and there but um clear clear maintenance maintenance cost there as well. Um yeah nice. I um I hope this is definitely an indication of the future and not just like in t technology preview it's enabled by default or something. something like that. But we'll see soon enough, I assume. Um okay anyway, uh TLDR, we can get rid of this code. Probably performance. be better. Are we gonna do some benchmarking Sage?

Speaker 1: Oh how do you benchmark this kind of thing?

Speaker 2: How do you benchmark um I think you would have to go in your browser With a dev dev build if possible and use the performance tools in here. Um with

Speaker 1: the dev build

Speaker 2: I so the dev builds um it's only gonna make a difference if you have dependencies like React that give you more logging in your dev mode. Um I have re crash to the dev build and I'm gonna display the stimulus logging in my console Oh I messed up something.

Speaker 1: How did you build the development build?

Speaker 2: NPM run starts?

Speaker 1: Yeah.

Speaker 2: Um

Speaker 1: start caching. Oh wait, you've disabled.

Speaker 2: Yeah. Uh what have I messed up? What have I messed up? Oh Oh someone 's using the wrong website version.

Speaker 1: It's so handy to have that version number directly in the settings. You think we should make it so that if you click on it, it will copy? Maybe not worth it because you can just cultivate yourself.

Speaker 2: I don't know Sage. Someone in the chat says, are we programming or fixing the issues? We're not doing either. We're just planning, I suppose, at this point in time, trying to figure out what is sensible for us to do. And then Sage will do the coding.

Speaker 1: I hope. Yeah, that that's the thing. These days, a lot of the time I spend is not really well not only on the coding, but also

Speaker 2: Yeah.

Speaker 1: Big chunk of the time is around the planning and like, is this what we want? Because like there are many different ways to do the same thing, but Yeah, it's it takes time to figure out what would be the most sensible approach.

Speaker 2: Yeah. It's a big code base. And it's not like we have another team that's out there telling us exactly how to do things, what we need to implement. For us it makes much more sense that you get lots of uh leeway as a dev to decide. what's the sensible approach as you're implementing it so that we don't spend time devising things that are just not buildable anyway Or you know, we spend way more time building things than warranty because we want to do it in a very specific way. Um Yeah, my uh command line tool to check lines of code says we have um S eight hundred thousand lines of JS and TypeScript.

Speaker 1: Oh, that's just JS and TypeScript?

Speaker 2: Yes

Speaker 1: Oh, I thought that's like the whole code

Speaker 2: base. But it's probably counting way more test files uh if I just look inside. SRC.

Speaker 1: Does it count snapshots?

Speaker 2: Oh sorry. Yeah, I was looking I was looking at other file types. inside the client folder. So inside the client folder, if I just look at like our actual admin front-end code, we're back to 50,000, which makes way more sense Um yeah, about fifty thousand lines of JS and TypeScripts. That makes way more sense. I'm expecting

Speaker 1: It's a bit low. I I would ex expect it expect that it in the hundreds of thousands, but not like more than two hundred

Speaker 2: Okay, this is only checking the client folder, so there's definitely JS code outside, but I feel like that's the majority stage. Maybe I'll publish like the exact numbers later. I'll check exactly which folders I'm ignoring and allowing and get a better count Yeah, maybe that's because I'm only counting um lines of actual code, not blank lines and comments that might be why you think it's too low anyway um I have installed the correct version this time

Speaker 1: Yeah, there we have the dogs.

Speaker 2: And now we can do a bit of performance benchmarking. So you go in here. And you're gonna decide exactly how you want to benchmark it. I think memory would be very interesting. And um I quite like to add a bit of throttling because generally when you benchmark you want to look at the painful scenarios. You have to decide carefully what you want to sample just because the performance cost of this sampling is really high. So I think for now we'll keep it simple. Then I'll just switch to a more expensive page too. render. Is

Speaker 1: the home page more expensive?

Speaker 2: I would assume The trickiest for us would be long stream fields. Do you want to do something more painful?

Speaker 1: A recipe page.

Speaker 2: A recipe page See that's the trade-off of having realistic content. I never remember which one it is that has the most complex content.

Speaker 1: Maybe we should have a subtle way to put that in the title of the page

Speaker 2: Large meets meat.

Speaker 1: Yeah, something like that.

Speaker 2: Okay. Let's see what happens. Um I will make this More desktop size, it'll be more realistic. Um turn that iPad air around.

Speaker 1: Imagine having that many devices. All

Speaker 2: all all those ones you mean? Like in my test uh lab.

Speaker 1: Yeah.

Speaker 2: Some people do. I mean like you know it's a really useful thing as a dev if you have access to like two or three rights.

Speaker 1: Yeah

Speaker 2: Okay, we are initializing and I think we can get going. I'll uh create a block and type maybe we can do some events interaction in here add a comment

Speaker 1: I can just sense that your console logs will be filled with change change. Well, I guess you didn't preserve the logs, so Yeah, and this is only after the full page reload.

Speaker 2: Oh yes, thank you. Uh my browser is struggling. Okay. They've released a new version. I haven't seen this insights tab before. So I think top to bottom is the right way to read this. Very top. This is the most helpful bit to figure out where you are, what you're looking at. I if I remember right, the yellow in this area chart at the top that's scripting and um purple is uh rendering Forget what the green is. So if you see big yellow blobs, that's the kind of things you wanna start to investigate.

Speaker 2: And uh you have your screenshots timeline underneath. Uh I think this has like a specific term I think we call it film strip, like kind of like a movie theater film strip. So you get to have a better sense of exactly what what might be uh causing the spikes. Um and then this is aligned with network events underneath. So here there is a really clear spike when I'm hitting save with lots of network traffic I assume we're reloading the preview around the same time as I've hit save which explains those requests So I think we want to look at like you know the intensive stuff that will definitely take lots of processing, like here, the loading of the new page.

Speaker 2: see why that might be quite intense. But then you also want to look at things that probably shouldn't be too intense. So here for example, kind of like a sizable spike you want to check that is actually for something that is a meaningful interaction. Any spike you feel like we should look at in particular, Sage?

Speaker 1: Maybe while you were typing.

Speaker 2: Yeah, so the typing will be here the keyboard events. You can see that there's lots of them one after the other. So clearly there's typing happening.

Speaker 1: Yeah.

Speaker 2: And clearly the performance cost isn't too high.

Speaker 1: Okay, that's not too bad, yeah.

Speaker 2: Yeah, but you know, like I'm on a very, very good computer, so you do want to keep in mind um Yeah, like this that it takes more than one frame at 60 fps. So you know 60 fps it's kind of like the gold standard for rendering. That's 14 milliseconds, right?

Speaker 1: 1000

Speaker 2: divided by 60. I felt like you'd know that say it's your gamer. 16 milliseconds. So one frame every 16 milliseconds. So anything like this that takes more than 16 milliseconds, you start to want to be questioning it. I don't feel like typing it. like 60 fps is necessarily a problem if that's not achieved because people don't type that fast but like still Yeah, I guess the top like APMs of Program Pro gamers is like 400 actions per minute, right

Speaker 1: I don't know.

Speaker 2: So yeah, you can't type more than like if you type 10 characters in one second, that's quite good Yeah, 20 characters. Well we have we have space anyway. So we can investigate one of those events specifically maybe. I guess we'll have a key up and key down for lots of these and we can zoom out on the specific one. Yeah, so the keyboard events stack and I think here will be our code okay some react event listening

Speaker 2: so i'm guessing this is because react has a global event listener for all key events and then it dispatches to the relevant React components and probably here there's not doing anything, it's just deciding that those events aren't relevant for it Oh, so there's no actual like wacktale processing for this one.

Speaker 1: Interesting.

Speaker 2: Um something has been debounced. Ooh, that's us right there.

Speaker 1: I know. That's that's the unsafe controller right there. I recall because there's the check. I think that's the name of the function method in unsaved controller. And then you call debounce. cancel. Yeah. Yeah.

Speaker 2: Yeah. So here I'm a bit surprised that our webpack source maps don't have more like user-friendly names. But I guess maybe that's the shortest. It's just we have quite long file paths.

Speaker 1: Isn't that I I don't know what would be more user-friendly than that?

Speaker 2: I guess just re dropping all of this?

Speaker 1: Oh I think I don't know I I I just I've always assumed that's just how Webpack works

Speaker 2: It is just how Webpack works, but there's lots of ways to configure a webpack.

Speaker 1: Oh, you can you configure the the protocol?

Speaker 2: Oh. Um I don't know. This is what I'm referring to for F just like the source mapping generation you can have it be really coarse and fast or really precise and slow Yeah, who knew there'd be so many options for source maps? I forget which one we're using in Wagtail. But anyway, yeah, you're right. At least the method name is preserved. And uh Then we know our uh unsafe controller check here that's one millisecond. So I think it's good, but definitely one millisecond times me typing 50

Speaker 2: Characters that's just a debalanced version. So it's kind of expensive just to debalance something. So it probably would be better for it to be on set interval, as we discussed. And uh I guess if you wanted to get the exact answer, you just benchmark them both, note down those numbers. Um

Speaker 1: what about that big yellow chunk?

Speaker 2: Which one?

Speaker 1: Yeah, just before

Speaker 2: that one?

Speaker 1: Yeah, yeah. Oh, is that The stream field thicker and rendering.

Speaker 2: So We have one to render the stream field block chooser.

Speaker 1: And that's React. No, no shades for React, but just It's stating a fact.

Speaker 2: That is React. Do you think it's a problem? I'm not sure it's a problem.

Speaker 1: Probably not.

Speaker 2: Because you know, like There's definitely benefits in using React for this because then you only render it with the correct content when you need it. Other libraries, like at least in the pre-React world, will have this pre-rendered somewhere on the page. And then we'd have to show and hide it, potentially risking like having it always rendered, just hidden away. Anyway. Measure. Definitely measure is the name of the game here. Don't just uh react. Um okay, so then we click and I'm guessing this here, sorry, um The the the second yellow spike is probably just um uh handling of the interaction with the block.

Speaker 2: And then this will be handling the creation. Okay, there's the preview refreshing in there as well.

Speaker 1: Yep.

Speaker 2: Okay, so there's just lots happening in sequence because

Speaker 1: Oh, maybe it's the mutation observer.

Speaker 2: No, it wouldn't be so big. Um do you want to hunt down for it?

Speaker 1: Hunt down.

Speaker 2: So we're on a smaller timescale. Um this is a browser extension. So this is not relevant. Reminder if you want to do this for real to do it without any browser extensions in your browser. More browser extension, more browser extension. Oh bitwarden. See this is why I prefer to not have my my password manager in my browser as a dev it's just like really unhelpful.

Speaker 1: It's handy to just command shift l

Speaker 2: It is handy. But at what cost, Sage? Now I have it in my stack trace. Ooh, under the minimap.

Speaker 1: Is

Speaker 2: the minimap of four frames? Why do we even render the minimap? It's not even displayed. Oh, it's just for this.

Speaker 1: It's updating the minimap, right?

Speaker 2: It's just for that Oh, is this worth it, Sage?

Speaker 1: I mean you want it to be accurate.

Speaker 2: Yeah. But here there's just so many ones anyway that they all look the same. We could just have a check like how many items are there to display, just find out that many. Yeah, anyway. 70 milliseconds. Um before that, what do we have? Some stream field rendering logic with animation right there. This is why you shouldn't do your animations in JS and there's lots of JS that needs to run to animate. Um it fits in one frame, I suppose, at least.

Speaker 1: Wait, how do you know that it's a single frame again? Is it the

Speaker 2: I'm just looking at the di duration, 14 milliseconds. Well actually no, I'd have to check. I think there's a different view you can check if you really want to be sure. Yeah, you see this red overlay here. It's meant to flag things that are too long, but at least in my mind, any JS that takes more than 14 milliseconds to execute is room for suspicion Though obviously you know not a deal breaker for all code not to happen inside one frame. Um yeah, okay, the um why does this happen here selected item change so that's the point at which you clicked this item and it decides how to insert the block

Speaker 2: yeah so the block has insertion there um so it's not so bad actually um there 's about what are we looking at here do i have um 500 milliseconds. I mean that's quite quite a pessimistic timing of it, but yeah, 500 milliseconds for insertion of one block. uh just because of how it's spread and within those 500 milliseconds like the js execution time isn't crazy big 7032 about 200 milliseconds of actual JS execution to insert a block. I think we could get it like 10 times smaller if we tried hard. But it's probably not worth it unless

Speaker 2: unless that execution time was to scale with the complexity of the page. That's probably what we would want to check here. That's like for a much heavier page than this one It's not like ten times more work. But surely we don't code that bad sage, right? Um yeah, and then this is my browser extension that muddies the water and let's see if we can find the um review preview rendering okay our preview call do you want to find if we can find our set interval oh ooh Timer fired.

Speaker 1: Shots fired.

Speaker 2: Set preview data So I assume that the bounce

Speaker 1: this is unsafe controller

Speaker 2: Thankfully someone isn't asleep at the wheel. Ah, what am I searching for? Sets optional So set preview data Where is it? Where is it?

Speaker 1: I think I usually add a Parenthesis and opening parentheses when I'm looking for oh wait, yeah. Sorry.

Speaker 2: It was a trap It's rare.

Speaker 1: Yeah, th this is a bit I don't know how you feel about it, but the thing is the function will not be debounds if you have disabled the auto update uh you know clicking the button will immediately do the thing.

Speaker 2: Yeah. Why is this one there?

Speaker 1: Um that's prettier thing uh or ESLint formatting. I don't know, but one of those

Speaker 2: definitely prettier. It adds

Speaker 1: yeah. It doesn't make sense, but that's that's valid syntax and that's like the convention if you have um multi-line type guineans it looks funny yeah but i guess it helps align things

Speaker 2: Yeah, I mean I feel like at this stage we should have

Speaker 1: something uh a separate site.

Speaker 2: Yeah, yeah, yeah. But anyway, uh it's not like we look at this code super often. So and then no one else does, so it's not like there's a ton of value in the API looking super neat. Okay, so back to our benchmarking, set preview data, 7 milliseconds. Which is the fetch call primarily. So where the hell is the there's no check here of the form data?

Speaker 1: There is. It's just that this method is only called when there is indeed. A change.

Speaker 2: Where did that happen?

Speaker 1: Maybe if you look at has changes. Check and update preview. This that that that's the method.

Speaker 2: I'm surprised not to see it in the cold stack. Maybe Okay, I'm guessing there is something in our um JS compilation that means it's one of those two Doesn't make too much sense to me, but I don't see why it would disappear otherwise Otherwise. Ah, there it is.

Speaker 2: Check and update preview. So we are inside check and update preview. What's this arrow? What is going on there?

Speaker 1: Well error? Oh, arrow? I thought you said error.

Speaker 2: Install timer. Oh okay, I understand. So this is this is the check and then it's the bounce for the the actual code okay I was confused so two milliseconds for check and update preview the bulk of which is sterilization to form the So that's the cost Sage, two milliseconds to scalize the page to form data. And our cost to process one key app event was One millisecond. So we definitely don't want to do form data stylization for every keyboard event.

Speaker 2: And uh CQFD, as we say in French, I think it makes sense to do it on set interval.

Speaker 1: Okay. form

Speaker 2: data solarization that's the expensive parts of the check and then the bounce the actual calling of the server And we can get rid of mutation observer, which I don't even know where to find in this mess.

Speaker 1: It is a saved controller, but yeah We don't know. Oh. Oh, you mean in the in the profiling.

Speaker 2: Yeah, if I just

Speaker 1: point.

Speaker 2: I'll just do the smallest amount of mutation that I can find. So let's mutate Something real cheap. That's mutated. enough right

Speaker 1: i don't think so we don't the mutation observer doesn't look for attribute modifications it only looks for sub three child nodes so If it's only toggling attributes and not adding or removing child nodes, probably didn't trigger. Maybe disable your Bitwarden extension for now and then try again

Speaker 2: Go away.

Speaker 1: Oh, I hate it when when the tilt uh uh uh obstructs.

Speaker 2: Let's do it the proper way. um in an anonymous window.

Speaker 1: Did you actually disable it?

Speaker 2: I thought I'd do it in an I'd do I'd um move to anonymous window but I'm I'm too late Oh I hate you. Yeah, we do tend to have um tool tip overload as there's these days. And you never know who's looking at this in um zoomed in UI where the tool tips take over your whole screen. Anyway Sorry, I'm still trying to find something that isn't

Speaker 1: if you delete a block instead of

Speaker 2: Do you think?

Speaker 1: Yeah Yeah.

Speaker 2: Processing Where do I find my mutation observer? I'm sure people who spend more time in this than me, they must be screaming, looking at this and saying, why don't you use feature X or Y or C

Speaker 1: Well we all have to start somewhere, right?

Speaker 2: Yeah. Biggest confession time is clearly I haven't ever uh benchmarked mutation observer before.

Speaker 1: developers these days.

Speaker 2: Yeah. Getting fancy rendering tools and Okay, I can't find it, Sage. I'm gonna I'm gonna give up. Um , reacts

Speaker 1: That's interesting.

Speaker 2: So the click event lots is happening when I delete.

Speaker 1: Delete block.

Speaker 2: That makes sense to me. I don't really know if Um

Speaker 1: can you search in this thing?

Speaker 2: That'd be nice, right?

Speaker 1: Yeah.

Speaker 2: See, that's what the power users are meant to be doing, I think. And saved controller I'm surprised it's inside the click event.

Speaker 1: Maybe because it's somehow I don't know, bundled as like still the same flow from the event of you clicking the delete button all the way. Yeah Is that the actual mutation observer?

Speaker 2: Invalidate contents path. I don't know Sage.

Speaker 1: Can we find mutation?

Speaker 2: That would be a good thing to search for.

Speaker 1: Hmm. Maybe you can't.

Speaker 2: I'm sure there is a way. I'll look it up after this. Interrogate the wisdom of the internet. I'm sure someone somewhere has already done this. Um TLDR, we have a plan

Speaker 1: We do.

Speaker 2: I know

Speaker 1: autosave is happening.

Speaker 2: Autosave is happening and we know how to trigger the actual auto saving and we will not make it disabled

Speaker 1: disabled

Speaker 2: disabled unless someone really really says why they want it And they commit to maintaining it forever. And um, thank you, Baptiste, uh, for mentioning that CQ with these QED in English.

Speaker 1: Oh yeah, that's I was I wanted to mention that but I wasn't sure if there if what what's the I don't remember like the actual um I know the abbreviation, but I don't remember the

Speaker 2: Oh the actual what it means when you say QED

Speaker 1: Yeah.

Speaker 2: Uh it's a Lat Latin quote, right?

Speaker 1: Yeah, yeah.

Speaker 2: Yeah. Something something demonstrate.

Speaker 1: Yeah. And thus it is demonstrated

Speaker 2: Been fun, Sage.

Speaker 1: Yeah.

Speaker 2: Um thank you Joel and Baptiste for saying hi on stream. See you next week.

Speaker 1: See you.

Speaker 2: Batista says it's quoted erat demonstrandum.

Speaker 1: Ps sounds similar to what I said, bruh.

Speaker 2: Have a good