Wagging HubSpot's Tail

Speaker 1: Okay, so I'm Simon, I'm the CTO of uh Para. And who are you? Um

Speaker 2: my name is Joost Meyerink. Um I'm product designer. Um so yeah, for Bearer.

Speaker 1: So I'm gonna talk a little bit about technical things and then Joost will talk about more interesting things Yeah, so what is uh para? So para is um well we call ourselves a startup, but we started in uh 2015 We do uh what we now call AI AI-powered candidate screening. Uh so we ask uh candidates to give us some natural language and then we run that through various AI, machine learning type of models to come up with scores. So we serve the HR industry. So it's kind of a SaaS platform built on top of Django since 2015. And personally I've been using Django since uh uh two uh yeah

Speaker 1: 2004. Uh so 20 years. So I'm the champion of Long Chango users. So Hotspot is a CRM uh with um sort of a CMS bolted onto it. And the uh problem with the CMS is that it's um it's um it's it's a hosted CMS, so um if you're developing for it there's no sort of local testing, etc. So it's a very long uh code debug type of cycle. Uh of course they have a proprietary um CMS uh database. And um yeah well it does have an API so that's a good thing and uh templating language uh similar to uh Django's of course.

Speaker 1: So we're gonna tell you a story of how we got up here. So when the company started they said oh we need a website Now the tech team wasn't going to build them a website, so they said, okay, we'll we'll make it with WordPress. Okay. And then they said, oh well, no, this is the marketing team. Oh well we can't modify it to do what we want. So they said, can you help us tech? And we said, we're not going to touch WordPress because we all had uh PHP PTSD. Yeah, so okay, make us a website. All right, so you know i in order to reduce the load we used uh like a static uh sort of HTML type of um

Speaker 1: generator, which used uh markdown, which is a you know plain text language. But then it turns out marketing apparently can't can't possibly write anything in plain text So they said, oh, I can't understand. So at this point I said, well show me all the documents you've written in Word. Yeah, okay, that's also zero. Anyway. They need a new website. Now at this point sales had adopted Hubspot, so oh, we can put a website in HubSpot using the HubSpot CMS. Great. So they hired a marketing agency to build the HubSpot templates. Then of course they rebranded the company again. So they hired another website agency to build another set of HubSpot templates.

Speaker 1: Um and then finally they decided that the the the current sort of third generation of HubSpot templates were really not working for them and they need to do it yet again. And though at that point they said came back, can you help us And what did we say? Yeah, we're not going to touch HubSpot. Yeah. Okay, but they're desperate, so please make us a new website. So we looked around. And which has Wagtail, of course, because you know we've been using Python for so long and Django for so long. Actually, there's really not much choice. And everyone's happy. Okay. Well, in reality, um

Speaker 1: so we thought, okay, well how difficult can it be to build a dub dub dub site? We thought okay, could we can knock this out in a couple of weeks And we can get to try new stuff, which you know engineers always like. And we only have to do it right, you know, once. Yeah, famous less words. So four months later , few weeks ago, yeah, we went live. And here it is. And uh We're very proud of it and uh it's beautiful. Um yeah, so please visit our website. Um yeah, so let's get technical for a little bit. Um So in terms of uh system architecture, pretty standard, this is just um this is basically a simplified version of what our product is.

Speaker 1: Um So um yeah the front end is served by uh by CADI. Uh so for us that's new because we were using Nginx and CADI is much much easier so I really uh recommend it The Django process is running in HyperCorn , which we chose over Daphne because it has uh automated restarts. And you know it's AGSI because we wanted to try AGSI rather than UWSC. The databases on the back end, of course, uh Postgres and Redis. And for a queuing system we use a thing called RQ. If you need a queuing system, I really recommend RQ. It's built on top of Redis. It's really simple. The code is really quite small And you can

Speaker 1: sort of understand the code and really sort of debug what's going on for when your queuing system stops working. And of course the the static assets are served from the cloud front from S3 So our deploy architecture looks like this. So we push code into GitHub Actions. That triggers an action which Builds the back end as a Docker image which we push into the Elastic Container Registry of AWS. That triggers another action which checks out the code again And then builds uh the front end assets or pulls all the front end assets and builds it into uh you know webpack packages. Um And then it it it sort of pulls the newly built Docker image and then uses that to run um

Speaker 1: collect static, which then distributes uh all the static images, uh code, all the static images, all the JavaScript, etc. into S3. And then we trigger another image which uses Helm to deploy into the Kubernetes cluster. So static images first, static assets first, then deploy the code which is a Expecting those images and static assets to be there. Very important to get that the right way round, we discovered. Yeah, so um talking to HubSpot, so um the the the the sales team of course still use HubSpot as a CRM um and uh the function of the website is of course to to capture leads, etc. So we um We need to um capture

Speaker 1: data in in our site and then do some processing on it and then send it into the hub spot. uh database. So on the left is the um is the UI from from Wagtail and on the right is how it's rendered in our in our website. So HubSpot forms they all have a unique uh identifier which is uh basically a UUID. So what we do is we um we record that UUID because we might have multiple forms on the page, right? And then we um we send we we we send to the front end the block ID So the block ID allows us to get back to the HubSpot ID, which allows us then to post that into HubSpot.

Speaker 1: So once we've got a if we if we've got a page URA we can go from a page URA to a page object and we can given a page object we can run along all the blocks in the page we can find the the right block ID And then from that we f we can figure out the HubSpot form ID and then send that back to HubSpot. So in terms of flow, what it looks like is The back end is sending to the front end the the block ID and the data that they that they want. So basically the form structure. The front end renders the form in some kind of rudimentary React uh code which I put together. Um then it posts back uh to the back end, you know, which page was this and which block was it. And the bl

Speaker 1: and the backend then figures out, you know, what I need to do with that, uh, puts it into the queuing system and then the queuing system sends it back into the gaping moor of HubSpot to uh to go into the uh uh CRM. So, oh, we got a new um new contact on the database, etc. Or somebody wants to download some content, etc. etc. Yeah, so that's kind of how the flow works. Um so I mean it basically works, it's live, all's good. But we just built the forms that we need. So we're kind of hard-coded, you know, kind of download forms or contact forms. In the future we could make it generic if we ever have any time to do this. So we can you can actually read from the HubSpot API the sort of uh like a JSON structure of how the how the form

Speaker 1: it is is meant to be configured or displayed and then we could write something which would then um you know render that properly into the uh into the front end and then into the back end but right now it's just basically predefined So that's my bit, and now as your host.

Speaker 2: Yes. Yes, so um as a CMS, um Yeah, the perfect CMS. Well uh I've been working as a designer for uh a lot of years. Uh started as a graph at a graphic design company, um fell in love with uh web design and uh star Designing uh websites and I was like, oh, this is amazing, and they were built in WordPress. I was like, Oh, this is good. Um, later on I uh met a lot more developers and they were like WordPress is uh shit Well anyways um let's get back uh over or uh in the future a couple of years. Uh I started working for a marketing agency which ac actually built their own uh CMS. So I designed their uh CMS uh which is

Speaker 2: um yeah one of the most fun uh projects I had and that actually sparked my interest in becoming a real product designer so designing uh actual products instead of just websites and Again, a couple of years later I joined Para, uh this was uh one year ago, and I saw this amazing uh website built-in HubSpot Well just sarcasm, but anyways, um I was immediately uh yeah just sending a lot of feedback to marketing like can we do this differently? Uh there was a Well, terrible uh way of just managing the content, which is uh the core of what it should do. So uh I'm gonna zoom in on a couple of requirements I had As a designer, I want

Speaker 2: there there are a lot of more requirements of course, but I want three things. Simple stacking of blocks. Um so I but I want there to be some logic to it, right? I w if I see a website, I want it to be nice I want the marches to be correct. Um I want to pre-define what templates you can use. This is one of the quirks that some uh CMSs have. You can add like pages anywhere. You can add an article detail page as a direct child of a contact page, whatever. You can do anything, just select a different template and you're all good to go. So I want to pre-define and I want to allow some templates to be shown in different places. And the third one is a Dutch uh term, Hoofterproof.

Speaker 2: I don't want uh WYSI-WYG, uh rich sex fields. Um I'm a designer, I want things to look uh really neat. I want it to yeah I don't want uh some content uh marketeer to just completely mess my design. Um So this is why I want to pre-define what allowed content elements there are within blocks. So let's zoom in on the first one, simple stacking. Well I also worked as a front-end developer for a couple of years, which was one of my side thingies, so I made the front-end or at least the HTML and CSS. I'm a visual guy, so I don't like JavaScript. I just want the things to look pretty. Um so simple stacking. How did I solve this? Well what we have is we have three different backgrounds. Uh probably it looks like one here because the contrast is really bad on this beamer.

Speaker 2: Anyways, um we have the off-white version which is actually the one in the center. There's a border you can see it. We have a white background and we have Uh sorry, an off-white, a beige and a colored background. So that's one thing you have to keep in mind, right? They have the second thing that adds to it. We have different types of margins between blocks. We have like this one on the top, it's a a small bit, it's forty-eight pixels, three RAM in our case. We use uh 16 pixel of course. And we have default uh spaces. So the first thing I did is I just wrote some generic code to just fix this. Um and It worked. If you combine two blocks, it uses the correct margin.

Speaker 2: And well, this is a simple simplified version of it. But anyways, this is what we do in SCSS. Um and it just lo yeah I love CSS uh S CSS so uh it it just works good. Yeah you can actually see the the different backgrounds here, but anyways, believe me, it works. Um the second one, which I said was uh the predefined templates Well the only thing that's actually predefined in our templates is the header. Anything else, well, it's just a uh a block area, right? It's it's a place where keep people can put blocks. So anything between the header and between the footer is actually free to use the blocks I define, of course,

Speaker 2: because I don't want people to just ram. random blocks in uh in some pages. So first one is of course a home template with a nice hero header with a lot of color Then we made a second version of that which is for landing pages. Just a bit yeah less uh distracting header, just a cleaner view. Then we made a default page of course. We have resources, which is like uh uh one yeah, well it has different types of resources in there So a type page below that would be the articles page and below that of course is a uh detail page. Is it shown already? Oh yeah. So this is the detail page

Speaker 2: So this is uh one of the ways I um limit the content marketeers to do uh what I want, um basically Then Hoof2 Proof, it's one extra level. Um I wanna restrain them from using any content element they want. So Uh what we have is blocks. So this is for example calls to action. And blocks are different types of elements, right? So we have blocks. A featured slider would show something like this. So there are like a couple of customer stories you can click through them and go to their detail page of course. And we have the image and text block, which is I sometimes call it a zigzag because you see these a lot on SAS websites, right? Image on the left, text on the right, and then we switch it around a lot. It's uh a basic block. But let's zoom into

Speaker 2: that one, right? It's a block, but actually it's a block with blocks within because you have text, uh you have list items, you have images, videos, whatever. In this case I limit them from using images because I don't want images next to like a large Beautiful image and button groups. But that presents us with the next problem, right? We make blocks with blocks within with blocks within Um which is one of the requirements I have of a CMS. And a lot of the CMSs don't actually work this way. But then again. Luckily for us, Wagtail has stream fields. So this is one of the things we used a lot, and this is also uh where I get it back to Simon

Speaker 2: with a famous quote Yeah.

Speaker 1: Yeah, so we we love stream fields. It's stream fields all the way down in our system. If it's blocks within blocks within blocks, but um this is a really a question to the audience. Maybe I I missed something when I was developing it. So um Yeah, so you can have a stream stream block within a stream field in and it renders in the UI for in in the editing UI, but I can't put a struct block within a stream field I have to sort of either make a list or embed it within a stream block. So we end up with this sort of double hierarchy. in in the editing UI where we've got like abstract and then within abstract there's another abstract. Really I just wanted we just wanted one uh in the in the UI and I spent a long time reading the code. I really couldn't figure out how to just give us sort of one

Speaker 1: level of let's say uh uh blocks. So please let me know if I missed something. Um and Yeah, a warning about um JSON in the databases. Uh so I I feel a bit about JSON in the database is similar to r re regular expressions. So There's a famous quote from Zowinsky, you know, if you if you think you're gonna solve a problem with regexes, now you've got two problems So that 's a bit how how I feel about putting storing JSON in the database. You know, now you now you've got two problems. Uh because you know data structures change, so JSON needs migrations. I know Wagtail has uh as uh stream field um uh

Speaker 1: migrations. And you know if you if you forget about that and then you update your stream blocks, your stream fields, and then Just just just run the code again, then you're you know your your your page is gonna crash. So I if if I was doing it again I I might look at like whether generic you know foreign keys could probably do something uh equally good. And just as a small aside to all any any sort of back end developers in the room, if if you ever give front end developers um the ability to read and write data into a sort of a random JSON field, they're going to use it to store state, application state, and then they're going to blame you when the front end stops working. So yeah, don't do that. Um

Speaker 1: yeah. Done. Any questions