New ways to deploy your Django app v3

Speaker 1: Hi everyone. Uh I think I'm on and this has started. I'm Tom Dyson. I'm very happy to be here. I'm gonna talk to you today about alternative approaches for hosting.

Speaker 2: Sorry, Tom, Tom, sorry. You'll have to restart again, sorry, uh when I tell you.

Speaker 1: Okay, cool. Just t tell me when.

Speaker 2: Start.

Speaker 1: Hi everybody, I'm Tom Dyson and I'm here to talk to you about alternative approaches for hosting your Django projects. Let me share my screen. Right. So I'm from Torchbox. Uh we're um an agency in the UK. We've been using Wag uh Django for uh almost since the beginning, pretty much since the beginning. Uh we're we also the agency behind Wagtail, the open source content management system that I hope many of you are familiar with. But I'm not talking about Wagtail today. I would encourage some of you to, or any of you, to follow along with this talk. Using the link here, tomd. org forward slash deploy.

Speaker 1: Most of this talk is going to be a live demo and um the the the steps that I'm going to be going through are exactly the same ones on those notes. It may be helpful for you to go along with those. You will just need a few things in order to complete all the steps. The most important one is a computer with Python on it, which I would guess most of you have. And then it would be useful to have a GitHub account, some sort of disposable Linux virtual machine. And I've put some links in in these notes. with some a couple of sites that have a credit, $100 credit to get you started if you don't have one already, and a Google account. So the the purpose of this talk is to kind of address this question that I see a lot in Django support forums. We see it a lot, in fact, in the Wagtail support forums as well, but I think

Speaker 1: it's more particularly around Django generally. And it's people who 've gone through the Django tutorial, who've worked out how to build a site. and then they get stuck about and then then the next step is making it live and they get stuck there. Um and you see a lot a lot of people making You know, asking for help like this and they often kind of apologize, which I think is a shame because you know they shouldn't have to apologize for this. The the skills for deploying sites are pretty different to the skills to create sites. And um I feel like I I really like this last point here where someone replies to somebody else who's been struggling, where they say, um, uh it's good to know I'm not the only one. gonna keep on struggling till I don't realize it's a struggle. And that's it's funny and it's nicely written, but it was also I think it's it's a shame because um

Speaker 1: I I think it's true. And for those of us who have um who have deployed a Django sites using Nginx and or Apache and Gunicorn Supervisor, or worked out how to use Heroku or these other systems, then if you do it enough times, then then you know it's hard to remember how much of a mountain it is to climb for people who have just learnt Django and are suddenly presented with all these options And I should be clear that there are lots of fantastic options for deploying sites as you become more familiar with it. So there's the do-it-yourself approach, like the one I described, where you you have your own Linux servers and you uh install Nginx and Gunicorn or USG and so on. And then there are kind of great platforms like Divio where a lot of this and they provide 24

Speaker 1: -7 support and a lot of this is handled for you. So but but what I want to talk about today is are three options that um are not so well known and that I think could be Really a good useful starting point for people who who have just built their first Django site and uh and and want to find quickest ways of of making it live. The first one is called Doku. And the way a way you could think about Doku is like a really simple open source Heroku. So it's actually a long bash script that manages Docker containers on a Linux server, but you don't really need to know how it works. You just need to know the two lines to install it and then how to push your site to it to make it live. The second one is static site generation. And this feels a bit like Back to the Future.

Speaker 1: When I was first building websites in the late 90s, I would uh handcraft HTML files and um FTP them to uh to a shared server somewhere. And this is kind of going, it feels a bit like going back to that because you're you're building sites out of static HTML pages, but with a very different tooling approach. And there turned out to be quite a lot of benefits to this, which uh I'll go through towards the end And the last one is a relatively new product from Google called Cloud Run, which is like you take your docking containers and as they say in their marketing somewhere, they they convert containers into URLs. And there are a few rough edges around this, but I think the model, this kind of serverless model of serverless containers, that sounds like really uh jargony, but I think this serverless model is a really interesting one and I predict it's going to become the

Speaker 1: the standard way of uh of deploying applications. So it's interesting to see how that's working now. All right. The the most part of this talk is going to be a live demo, and I'm going to try and do a lot of these steps in the time that we've got available. It's going to be a bit of a race against time. So you'll have to forgive me if it doesn't go perfectly and we don't get through everything. And I hope some of you will be able to follow along too using that link tomd. org/slash deploy, which will take you to here. So we are going to start with providing we have something some of these things by building a Wagtail site locally. I've used Wagtail here. This can be any Django app. Wagtail is just a kind of is an example of a fully featured Django app that's quite quick to install.

Speaker 1: So that's why I've used it here. But this is none of this is specific to Wagtail. So in my local environment here, I would create environment and install Wagtail. I've done those first three steps so you don't have to watch the that scrolling past, but I'm now going to start a new Wagtas site and go into that directory. And the next thing I'm going to do is install a package that actually I wrote about a month or so ago called Wagtail Fake News, which is um uh a way to create a lot of dummy content on your site. I did it really for testing, particularly for testing um up pref uh volume so to see how well it's kind of scales when you have more volume. So we add that and put it in our requirements.

Speaker 1: txt And then we're going to add it to our installed apps in settings. So of course there are more elegant ways of doing this, but I'm just going to push that line to the bottom of my settings file. And um and the next step is to run and migrate. So this is common again. This is a standard thing that as a Django developer you'll be used to. So creating all the tables that Django needs and then that Wagtail needs and that Wagtail fake news needs. That's all done. And now we have this new management command. I didn't copy that correctly. make fake items. So this is the one that comes from Wagtoe Fake News and this is going to create 50 fake news pages each and 50 images and it's going to associate images with each one.

Speaker 1: That happens pretty quickly, seven seconds. And one last little step. Magtower has a default home page, which we don't want, so we're just going to overwrite that template quickly with that line and then we're going to run the server. So most of this now is hopefully feeling quite familiar for uh for Django sites. So now I'm running my server and let's check, see how that's worked. So in a new browser tab, I am Checking here that this news is linked and I'm just going to check something actually in the Slack to make sure that my I'm not seeing anything this

Speaker 1: like. I'm just hoping my screen share is working. I guess otherwise someone would have told me, but I'm just not seeing the screen share indication over my screen. Right, so now we are seeing Uh can someone just like give me a positive sign in the general channel on Slack just to say that you can see my screen? Working fine. Thank you, Darren Jones. Uh little moment of panic there. Okay, so we can see now that this uh my site is running. Um And that's great. So that's like, I don't know, how far we're in now, five minutes, and we've got our local Django site running. That's basically, you know, just credit to Django for making that site really easy. The next step, now we're getting on to deploying.

Speaker 1: So now we are going to install Docu on a Linux server. And I would generally use one of service like DigitalOcean or Volta. And so I would create something like let me see if that's going to log me in. Create a new server on Volta or DigitalOcean, one of these services So go here and cloud compute and then I can choose my location. I usually like I always feel like servers in Paris a little bit more stylish than anywhere else. So and then get the latest Debian. The small one is fine as long as you've got about a gigabyte of memory and uh add your SSH key to it and probably give it a name and hit deploy now.

Speaker 1: I'm not gonna do deploy now because I've already done one and it just takes a couple of seconds, but uh you might have done something a bit like that as I was talking. So here is my the server I've created and then I'm going to log into that one. So here in this tab with a black background I've logged into this that that server And the next thing to do would be to install Docu, which is these two lines. So it gets the bootstrap. sh and then runs it. I've done that already, it takes about two minutes, so I didn't you don't need to see that running up my screen Once that's done, then you go to your server's IP address in a browser. So if I get the IP address here. And make a new tab and go to the browser.

Speaker 1: That loads this initial Doku page. So Doku doesn't have a web interface generally, apart from this initial screen, where you just check, you can provide your SSH key. and give it a host name. I'm just going to use the same IP address and tell it to start. That's done. And then it's just going to redirect me away to the Docky Docs which I don't need. So I'm going to close that tab Okay, so Doku 's installed. The next thing I need to do is create an app on the server. So if you use Heroku, this will seem pretty familiar. This um uh a lot of the commands are quite similar to uh you could say they've been inspired by Heroku. So we've created the app and now we're going to install the Docu Postgres plugin. So DockerU has the concept of plugins and um

Speaker 1: most of them, I think they probably all are themselves Docker containers. Again, you don't really need to know that. But they're ways of providing different services on your Docu instance. So the Postgres one is probably the one of the most popular ones. And that means that we now have Postgres 11. 6 running on our Doku instance. Once that has finished, which shouldn't take much longer, we can create our first database. So we now get this Postgres create command that didn't exist before because we've installed the plugin So that's going to create a database called DemoDB , which will take a second. And then we're going to link that demo db to our new app. So we created our app called Demo

Speaker 1: App and we're going to link the database to it. And again, this is a bit like um how uh how Heroku works, where you make these variables like the database URL available to your apps. Okay, so that's all done. Something worth knowing about Docu, and in fact lots of these kind of mod services, and Heroku is similar, anything where you're using Docker generally. is that they are ephemeral. That means that every time you deploy them, they're recreated from scratch. Which feels a bit weird when you're used to it, but it turns out to be a good thing and it helps you design your your application in in I think in a correct way. So follows this this set of rules, principles called the 12 factor design, which you may have come across. I I recommend looking about.

Speaker 1: But it means that uh If you, for example, if your app has user content, you have a content management system and you're uploading images, then they're going to be lost every time you redeploy, which is no good. So we need to have some sort of persistent storage. So one approach might be use something like S3, but um Docu has uh an option for this as well. So you will create a directory and then we're gonna mount it. inside docu giving it the name and we're again we're going to make that available as a variable called media root which our application is going to need to listen to on Right, our application that we've built so far, so this is going back into the white tab where we built our application.

Speaker 1: Um, that uh because it's uh like a Django app doesn't use Postgres by default So we need to make sure the Postgres is installed and that it's in our requirements. txt. And then we need to tell our app using settings to use these variables that are now available. So we now have this database URL and Media Root. available so I'm gonna add those just plonk them at the bottom of the file which isn't really best practice but it's fine for now okay so we now have our app And we have installed Doku. Now we're going to do our first deploy. So deployments in Doku

Speaker 1: are all about Git. So we're going to create our Git repository, add everything to it, first commit, and then we're going to make a new remote. And so I need to put in the IP address of our server. Which should be somewhere in my clipboard history. There we go. And adding a new remote called Doku. And then the final step is to git push to Doku. This is the bit that's going to take a minute or two. And while that's happening, I can just kind of recap a bit on what we've got to so far. And I've got actually some slides to kind of help us think about this. So we made a simple Django site We installed Docu on a VM using those two lines. We enabled Postgres in that Doku

Speaker 1: instance. We added persistent storage. And we just ran git push to deploy. So the um the the making into simple Django site is what you do normally. Installing Docu and the Postgres things are one time only. Um after that and and and the persistent storage is probably for each app and then the git push to deploy is what you do for for all your apps afterwards. So we can see what's happening now. We pushed that up to Doku and Doku immediately saw the Docker file that existed in our in our app and so understood what to do with it. So it um It's following the instructions in the Docker file. It's currently now installing all the contents of requirements. txt. So all those things are going to be available to our app.

Speaker 1: It will link up the Postgres database, hopefully, and make that available to our app. And once it's happy that everything's there, it will deploy it One of the things that Docu does is check that everything's working before it makes the before it di deploys it live. which is really a kind of valuable step, but it does slightly slow things down. And you can do it in a more intelligent way if you like by creating your own custom checks. This isn't the most interesting part while it's just uh waiting to see what the installation part is. Um and then but now it's doing the

Speaker 1: deployment. So uh these steps will look familiar to anyone who's done uh Docker before. And maybe this is just while this takes another 30 seconds, I could just start talking a bit about Docker, which is we don't have time in this talk to kind of go into information on to kind of full background on Docker. But Docker is a way of packaging up your your applications in a that's a in a like a very lightweight virtual machine. And it's increasingly becoming a standard for deploying your applications across lots of different systems. And it also can be used useful for local development. So when you create a Wagtail site, it creates a simple Docker file for you. But there are other ways of doing that.

Speaker 1: So if you have a Django app, if you follow, for example, one of the cookie cutter steps. then um uh the the cookie cutter, the Django cookie cutter, then you'll also get a Docker file. So that's now finished and it's telling us that our application is deployed on a live server and we hold our breaths. type it in and it's live. So we've built our site. We have uh installed docker do we have uh deployed a site to do and it's now live however there are no news items on it because we haven't created any content on that one yet. So we're going to need to run a couple of tasks. And this is back on the server. We're going to run a management task. And you can see the syntax for this is Pretty much like the normal Manager Pi that you you start it with Docu run and then you specify the name of the app.

Speaker 1: Takes a few seconds longer than uh than a normal management task because it has to spin up a container quickly in order to make that available. And then similarly we're gonna create some fake items just like we did before but this time we're running it on docu so again a few seconds for it to spend the container and then it starts creating all those fake items. So now if we look again in on that one but on our live server and go to fake index we see all our site and all the pages that have been created and just to confirm that the admin user worked I can hopefully log in now using the details I just created

Speaker 1: and browse through Wagtail and let's edit a page. Maybe this first one. box last resource check this is working I guess the nation mark we'll change that image to audience democratic modern and uh bolden some of that and preview it and that all looks good. I must say I find these uh I find these randomly generated content really intriguing and uh often makes me think Sometimes it does the kind of sound really weirdly profound. Never direction what? Price might sell. Rate really positive.

Speaker 1: Sometimes feel like uh who needs artificial intelligence when uh randomness is this good? Okay, so to recap on where we've got to. We have Git push to deploy. We now have a live site production ready running on our $5 instance. This could handle probably five or six reasonable sized Django apps and for five dollars a month and you get pushed to deploy and it's a you know Pr pretty pretty quick process so far. Okay, next up is static site generation. There are quite a few ways of um generating a static version of your site. The one I'm going to use is based on Django Bakery, which is open sourced by the LA Times. In fact, we're going to use Wagtail Bakery, which is just a sort of bit of a wrapper around Django Bakery that does the more convenient things for Wagtail

Speaker 1: sites. So install that, add it to our requirements. txt, and then make a few changes to our settings again. Demo settings base. py, add it to the bottom. So we're adding bakery and wagto bakery. We're telling it to build out to temp slash dist, and we're saying that we want all our published pages. Those are going to be the pages that we want to be live. And this gives us a new management command, build. And build will take that view and create published versions of all those pages. usually takes about five or six when this like that can see nine seconds.

Speaker 1: And so if I now uh check in my What do we say? Temp slash dist. There we can see we have it's created some pages, and we can see for each of those pages it's created a directory and uh a page inside it. So it's got we now have static content for all those pages, which is great, but it's just running on our laptop, still on our development environment. So now we need to publish that. For that, I'm going to use Netlify. There are lots of other ways we could do this, but Netlify is uh Netlify has a lot of benefits. So it's an excellent service. And you you can install the command line version like this. I've done that already. But uh you can do that, it'll just take you a couple of minutes.

Speaker 1: And um Alright, now I'm gonna s go into that directory. So this is the one with all the statically generated content. And I'm gonna run Netlify init. So the first time you do this, you'll probably need to authenticate, and you can do that against your GitHub credentials. I've done it already, so I can just go ahead like this, choose my team. Hello. Hope no one has used that before. Yes, so that insight has now been created. Um, but now I need to deploy. So the first one was just creating the site I'm going to do from this current directory. It's going to check and see if there are any files that exist up there already and they don't because it's the first time we've done it.

Speaker 1: And then it's going to push all those files up. And once that's done, it is hopefully going to deploy them. And this is normally super quick. Waiting for deploy to go live. Just while that's going, I'll show you what the UI looks like. in um Netlify on the web UI. So here I'm now seeing the manual deploy. So that's now been created. The site's not being deployed as we know. This is sort of hanging at that last minute. So I'm going to do the classic, turn it off and turn it on again, stop it and try start again. This I'm going to do prod, which is going to make it go live straight away. That's a usually a separate step, but uh This time I'm just going to do it in one gray. This time we're getting some nice emojis that we didn't get last time, which is a bit odd.

Speaker 1: And this step, uh, once it goes live, should give us a static version of the site hosted on Netlify, which has its own collection of CDNs and works across clouds and has a lot of other interesting features, particularly around things like serverless functions and authentication and so on, which we're not going to go into in this talk I'm hoping that's going to finish in a minute and I can do a big kind of celebratory ta-da. If it doesn't, I'm going to have to wait and go back to a previous one to show that I that it is working. So This is one I did just slightly before and uh click on it here and you can see now this is running on Netlify. app. and uh all these pages are available.

Speaker 1: So the content of our Django site, the output of our Django site is now running on Netlify. You can see on Netlify. app, Netlify knows nothing about Django or Python or databases, it's hosting the static pages, but it's doing them in a very efficient way and also interestingly a very cheap way. That one hasn't gone live, so it's lucky I had that back up for me. Hopefully it will do before the end of my talk. So you'll believe me. Okay, so where have we got to? Live demo two. We generated our static pages. We used um we used Django Bakery, but there's a few other approaches we could have used too. We installed Netlify and we deployed to Netlify. And uh great, next one, then the final one of this demo is using Google Cloud. So like I said before, this is a new product, new-ish

Speaker 1: product, recently come out of beta from Google and called Cloud Run. And you have to start by installing the Google Cloud SDK , which I have already, but you could do doesn't take too long. Cancel that one now so we can go through this. And then you have to create so Google Cloud has this concept of projects. So you can kind of group all your stuff together in different projects. So This is what it looks like in the console. Here I've created a project called DeployTalk, which has that ID. So I need to tell Google Cloud The name of my tool uh project. Um

Speaker 1: And then I you have to enable the minimum APIs, which I've done already. That says which what bits of Google Cloud we want to use. You have to set your project ID and region. That's going to be useful for some uh uh shell scripts later and we're gonna it's this approach is gonna be a little bit similar to how to the docu one where we get a um a Docker a Docker image and we we submit it. So now from our app, this is kind of there's two key steps with this. The first most of it is kind of boilerplate really. This is the first key step where we're going to take our current app. Actually, we're not going to do this now because I'm in the wrong place. I need to go back to my uh demo app And we're going to take this current directory and submit it as a

Speaker 1: submit a build for it. So it's going to create a Docker image based on the contents of my directory, which apparently I've got about 19 gigabytes in total. And it's gonna push all that up to Google Cloud's image storage and uh make it available for other services to use. We had this first step called use Canico, which is quite a recent feature. It's a sort of strange name, but it uh basically enables caching of your build step. So you can see in some of the lines coming up here that This build is quicker than it would have been otherwise because it's I've done it once before and everything that hasn't changed in my app can just be reused from the cached layers

Speaker 1: So having done all that and runManage. py, it's creating that , adding all this to the cache, and it's going to put the the whole image, make the whole image available for us to deploy. There we go, done, took 48 seconds. So it's created this image. And the final step is to deploy that image. So I'm going to paste this in and then and run it. And then I'm going to tell you. what it means. So we are saying G Cloud is the general term. Run is because this is about Google Cloud Run. The action is deploy and demo app cloud run is the name of the service that we want to deploy And it's going to deploy a service from this image, which is the one that we just uploaded before. So we created an image for demo

Speaker 1: app image and we're now deploying it. And this should be fairly quick because uh Google Run service just needs to fetch that image from locally from elsewhere in Google Cloud and create a revision for it And that is now done. So and it a bit like Netflix, it signs you a um uh a new URL and a paste that in and hit enter. And then one thing you'll notice here is that this is slow. This is there's the this is because of um one of the issues with serverlessness generally is this this so-called cold starts. And cold starts are because uh in this case when we the first time we request the app, it has to uh

Speaker 1: has to s start that container. And this container is a rather inefficient container because the way that the Docker file works. in that you get out of the box in Wagtail, it runs a migration just to make sure that your database is up to date. Which you don't re you really don't want to run all your migrations the first time anyone requests your site. Of course it's uh Then and there's a lot you can do to optimize that. So you that's not kind of best practice for production Docker files, and actually you should move that step out of your Docker file, which um and there's lots of good documentation on how to do that. Once you once it is up and running, then uh Google Cloud does a really good job of preventing cold starts afterwards. But for that first one, uh

Speaker 1: you you want to find ways of reducing that cold start. So we now have our set that same app running in, I think, three different places. We have it on Docu, we have it on Netlify, and we have it on Cloud Run. Let me just quickly show you how that looks on Cloud Run in their UI. This is a couple of interesting things about the setup here. Also I find that this UI quite slow sometimes. That'd be a shame if that doesn't come up quickly while that's going. I'm just gonna recap on what we've done. We installed the GCloud CLI that made this possible. We built an image. And we pushed it up to Google Cloud Storage and then we deployed it to Google Cloud Run. This is still loading.

Speaker 1: Sometimes you just give it a little tweak The thing I wanted to show you here is how you can configure your the scaling of your instances, which I think is the really interesting thing about uh Google Cloud Run is the way that it scales not just upwards so uh you can scale to uh a thousand by default it's uh a thousand instances of your container each one of which but by default can handle 80 concurrent requests, but also it scales to zero. So here's the revision I just created. For my next one, I could set some other variables like How much memory? So this is currently running on a tiny amount of memory, so I'd probably want more like half a gig.

Speaker 1: And I could change the number of CPUs. Uh, and I say how many requests I have. But this is, I think, the really interesting part. So This will currently scale up to a thousand instances if you need them without you having to do anything. Just as your traffic increases, it will create new instances when it needs them. But also it will scale down to zero. I gave a talk about serverlessness in in uh Florence, I think three years ago, is it two or three years ago? And uh I talked about this idea of turning the lights off when you leave the room, which I think is one of the most interesting aspects of of serverlessness that um when people aren't on your site, it's kind of not financial or ecological sense to keep that server running. And that's that's what I like about this sort of serverless approach is that You are only paying when people are hitting your site in this.

Speaker 1: And this the servers are even only alive when you're hitting them. So I think scaling to zero is a kind of really interesting component of all this. All right, I want to talk very in the last remaining minutes very briefly about the pros and cons of these different approaches. So Doku, I think this is the easiest way of getting your site into production. So we did it, I think, in under 10 minutes. This is a standard Django site You can go live in 10 minutes if you have a Linux box available, a $5 Linux box. It also encourages good coding practices. So this that 12-factor design I talked about before, and it's super cheap. You can run A few Django sites offer five dollars a month and these are like you know that you can do this in production and we've we've done this successfully at Torchbox. On the other hand, Docu doesn't have horizontal scaling, so If you need to handle, if you need more instances, you can't. You can't have lots of Docu

Speaker 1: instances next to each other. And in a similar way, there's no redundancy. If the hard disk dies on your machine, then all your sites will go. The infrastructure is on you. Static site generation. The pros are pretty convincing. It's the cheapest way to host your site. You cannot be faster than a static site, and you cannot hack a static site. So it's you know it's it's it's pretty compelling for all these reasons. On the other hand, you get this build time delay. So if you have a really big site and you have to generate the static version each time, then there's a there can be a pause between making the content and having it live. There are more moving parts because you need a Django server and a static site server like Netlify. And it may not suit your app. So it works well for kind of static, like a content manager type sites. But if you need personalization or you have logins

Speaker 1: or lots of forms, then it's probably not going to be the approach for you. On the Google Run side, it scales forever, not quite forever, but if you have if you need more than a thousand instances handling AT concurrent users each, then you've got bigger problems. Scales to zero, as I mentioned before, I think that's really important. And it's the future, that's a bit glib. But I do think this model is the is is in is the way that uh that app hosting is gonna go. On the other hand, you have to deal with cold starts. It's still complicated. So I showed you the easy parts, but once you get into connecting databases and persistent storage, there's some more complicated instructions, which I've linked to at the bottom of my notes. And finally, databases are not serverless. So you there are some sort of serverless database products and

Speaker 1: Aurora on AWS serverless is one, but it's It's not it's I think it's just really hard for a relational database to be available as serverless model. So it's um uh it's it's currently still feels kind of a bit hybrid, but I do think that this is the way that increasingly apps will be hosted. Okay, if you need to deploy as quickly as possible, then I think Docu is a good approach. If your site is mainly about information, then static site generation gives you that speed and robustness and it's super cheap. If you have to scale, then I think a cloud run type approach is the way to go. That is it. I am sorry for going on so long and I'm going to stop now, but I'm really happy to answer questions either on Slack or uh in the uh in the Q<unk>A section afterwards. Thank you all very much.

Speaker 3: Recording is on.

Speaker 1: Hopefully some people were able to follow along a bit. Did anyone did anyone try to uh to do the steps as I was talking?

Speaker 4: I tried to, but I got uh uh slightly stitched up by Vault because there was a point where uh pip didn't work. So That was when I died.

Speaker 5: I just looked at the page as soon as you loaded it up and knew there wouldn't be enough time to follow along with having to register and get up these instances of the droplet type things that that look like some heavy lifting.

Speaker 1: Yeah, I think in retrospect it would have been good if I'd uh just have shared those instructions a bit more beforehand. But anyway, hopefully some people can can try afterwards and and ping me if they uh if it works or it doesn't work. Yeah,

Speaker 4: definitely will do when I s when it's not. It was just time pressure was was a bit too much, and I thought it was better just to watch at that point rather than miss it all.

Speaker 5: It's super exciting to try out and it's nice to see that the the heart of the the Google Cloud Run thing is pretty simple because every time I've looked at that documentation. I just said this will be five hours work and I'm not ready for it.

Speaker 1: Yeah, I think that's right. And and I think they're they're missing a trick there because actually the like you say, the heart of it is is straightforward. It's those two commands to to build the image and then deploy it. But um the all the stuff that's around it and and actually someone's written some really detailed instructions on how to get Wagtail working on Google Cloud, but um the the bits around the edges are quite painful and uh I think they could they could if they kind of think hard about simplifying some of those steps um then I think this it becomes a very attractive model

Speaker 6: Tom, I have a question regarding uh the the uh scaling, you know, like at at some point you said that like it can also go to zero in turning the lights off And and h how does that factor in with the cold start that you mentioned earlier? Like I was I'm like a bit confused with that. So you don't want that, right? As as a as a as a as a as a product developer You don't want your uh you don't want uh bad UX for instance your user experience. So maybe you can say something about that

Speaker 1: You do you want both, right? You want you want it to scale to zero because um you don't want to be paying for it when it when no one's using it. But when people start using it, you want to uh you want it to to come on quickly. So People kind of try to work around this and one thing people do is they like uh they look at their traffic and then they might just kind of ping the service at busy times to make sure it comes up. But I think I think that's I don't think that's the right approach. I think that you need to hope that Google and Amazon and these other services are going to get better and better at doing cold starts. And actually Google's one is is pretty good. And then you need to think about ways of optimizing your site to make it work well for cold stots. And um I that definitely isn't the case for that simple demo application, which as I said runs manage. py

Speaker 1: migrate at you know this uh when a container starts, which is crazy. So you you you you would you would do that as a standalone step and then reduce the startup time as much as possible. And then you can get it down to it should be like less than a second for a cold start, which is still You know, you want you really want your your your your general request to be under maybe like 250 milliseconds. But um I I feel like that that gap is closing.

Speaker 6: Thanks

Speaker 5: Does Doku um work as easily with uh compose files as it does with just draw Docker files?

Speaker 1: Yeah, I haven't uh uh I we've I've used mainly with Docker files. Um so I'm not sure about composed files. It also works plain well with just like plain Python application, so it will detect if it's if there's no Docker file, it will detect if it's a Python application or Ruby or JavaScript. Um I don't know so much about Docker Compose, but That would be interesting to know.

Speaker 5: Thanks.

Speaker 7: Um any any performance issues was uh running docu inside a very small VM? um Docker engine running because like there's one GB of memory.

Speaker 1: It seems that um on Linux that uh there's there's very little overhead. So Um I haven't done any really careful benchmarking, but um I've certainly you know six run some some reasonable size sites on Doku on a one or two gigabyte VM. with uh you know with one or two V CPUs and uh performance seems really good. And I would say like on a par with like a or even maybe even slightly better than a $25 Heroku Dino.

Speaker 7: Thanks Is

Speaker 8: Zappa uh not your cup of tea?

Speaker 1: Uh no, I like I like Zappa and um I uh I just um We we've had a couple of projects where we've we've used Zappa for in production and um just as soon as you start getting into kind of setting up VPCs between Zappa and uh uh you know, RDS, then it just starts feeling horribly complicated to me. And that's that's the point at which, you know, and I'm not a professional DevOps person, but that's the point at which I sort of uh glaze over really. But um yeah, I really like Zapper and there's some amazing like you know magic tricks in there. And then there's this really this cool project that this same guy did which is related to Zappa where you can use SQLite database

Speaker 1: on an S3 instance. And so you know as as as your as your database back end, which seems crazy. So every time you make a rec do a query, it will load up the SQLite instance from S3 and query it. And then save it back again, which is obviously is it's not going to work well for like really heavy multi-user stuff, but does work surprisingly well for kind of mainly read-only data. And that is like a serverless database. Um so there's a lot of amazing magic in Zapper, I think. It doesn't I haven't looked recently, but uh I got the impression that it wasn't being super well maintained or like really heavily maintained recently, which is a shame.

Speaker 8: I've had issues. I've had issues with um upgrading Python from 3. 6 to 3. 7 with Zappa.

Speaker 1: Alright.

Speaker 8: I probably shouldn't talk about that.

Speaker 6: How did you get to selecting a docu and uh the the call approach? Did it was uh uh was it like a part of the device strategy or something like that

Speaker 1: Uh I'm sorry, your audio broke up a little bit. Could you just say one more time? Did anybody else hear and can can repeat it? Sorry. I've just seen a couple of questions in chat as well. Um General cost of Google Cloud Run for typical small site. Generally, uh can't remember what the the Google you get you get a pretty high number of um of free requests. It might be a million or something I I should check into that, but for a typical small website you might find that Google Cloud Run is free.

Speaker 1: Um can you recommend estimate actual cost? It's pretty hard to to estimate on Google Cloud Run. Um and it depends a bit on your on your site, but for the main the sites that we make, which are mainly kind of content managed sites, like Wagtail type sites. We uh a lot of the load is generally handled by a CDN, something like Cloudflare or CloudFront, which can mean that um that most of the activity to the back end is editorial, uh, which is often within the free limits anyway. So you can um Uh if you if you have a s CDN to handle your front content, then it can mean that it's extremely cheap or even free.

Speaker 1: I think it's very funny.

Speaker 7: It's off topic a little bit, but uh have you ever tried to create a VM out of a Docker container in Google Cloud?

Speaker 1: No, I don't think I have. Have you?

Speaker 7: I I I I saw there's such uh feature. Like uh they you can give a container from registry from B CR.

Speaker 1: Oh yeah.

Speaker 7: But I couldn't manage to make it work. I couldn't find any more documentation as well.

Speaker 1: Right, yeah. Actually interesting. I I I came across that last night as well when I because I did think Maybe it'd be better for people to create the their VMs for Docu on Google Cloud using the compute. And then I noticed that feature, but I I haven't tried it.

Speaker 7: Um, would you would you consider adding Kubernetes as a fourth option?

Speaker 1: We uh I think Kubernetes I think i if you um if you if your skills are developed enough to use Kubernetes then you this this talk probably not relevant for you. We at Torchbox used we adopted Kubernetes about three years ago and um uh it was a mistake for us and I feel like And last thing, I mean, maybe it's changed, but um at the time it felt like it's it's obviously a tool that's great for Google or any kind of big product that needs to be able to scale really quickly and have full control over that. But for us, it really was the wrong level of abstraction. And uh we found that it just, you know, the developer experience was was really tough. So um

Speaker 1: We we kind of went full in for it and adopted it and then and then backed out about six months later. I think you know behind the scenes, then I'm I'm sure a lot of these services are using Kubernetes and uh and probably Cloud Run is is some Kubernetes type thing in the background, but I think it's uh for most people I think the I don't think it's the right level of detail. Yeah, I'm I'm keen to s to see the next start, the next talk, so I I'm gonna join then but I'm I'm really happy to answer questions on Slack um or or DM me. Thanks all for for watching and talking. Bye bye.

Speaker 7: Thank you. Thank you. Thank you.