Automating initial deployments with django-simple-deploy with Eric Matthes

Um

Speaker 1: hello, my name is Eric. I hope you're having a good conference and thank you for making time to listen to this talk. I'll explain the background image in a bit. We're going to talk about four things. What is Django Symbol Deploy? So what? Why is it important? How does it work? And what comes next for the project? So what is Django Simple Deploy? Deployment has always been difficult because there's so much to take care of. But there's been work over the years to make it easier to manage. The first major simplification came from Heroku with their git push Heroku main workflow. After that, other platforms created their own simple deployment strategies as well. Here's Fly's deploy command. These were a significant improvement on earlier deployment workflows. Here's platform.

Speaker 1: sh's push command. However, even with these tools, you still do critical steps in the configuration work. It's really easy to mess up configuration. The commands are simple, but the configuration work is not straightforward or obvious. All these tools and others like them are platform specific. Each tool lives in the hosting platforms world. None of them are centered around Django. But what if? What if we had a deployment tool that lived in Django's world? What if Django had its own deploy command? Django Simple Deploy does exactly that. It lets you run manage. py deploy. Django Simple Deploy is a plugin -based, platform agnostic, Django-centric tool for automating initial deployments. It's actually three steps for the fully automated workflow.

Speaker 1: You install Django Simple Deploy and a plugin for the platform that you want to deploy to. You add it to you add Django Simple Deploy to install the apps, and then you call manage. py deploy. So let's see it in action. My window manager is doing weird things. We'll try it. If it fails, you'll see it on the slides. So we're going to look at this. This is a blog project. This is a local project with no deployment specific code or configuration. It's what you get after running start project and building out your site on your local computer. So let's make sure it runs locally. All right, and all right, I can switch windows.

Speaker 1: All right. Okay, so that's running local on the development server, um, like we've uh most of us have seen many times before. So what we'll do now is we'll close that out. And we're going to deploy it. So we're going to deploy it using Django Simple Deploy. So we'll uh install Django Simple Deploy. And the plugin for uh deploying to fly. io Hey, live demos. All right. And now we need to add that to um settings.

Speaker 1: I don't know. Okay. Okay. Should be worth it. Uh all right, we'll commit that. Alright, so now what we've done is given my version my installation of Django a deploy command. So we'll run that. Python manage. py

Speaker 1: deploy. Dash dash automate all. All right, so it's asking if we want to do that. Yes, we do. We're telling it we want it to make configuration changes to our project, commit them, and then open this project. So if it works, we're gonna be interrupted in a few minutes with a functioning deployment. All right. So what? So why is this important? Why does it matter? Django now has a consistent deployment process across many platforms. These are the steps for deploying to fly. io, is what we just saw. And this is for deploying to platform. sh, different plugin.

Speaker 1: And this is for deploying to Heroku. And the DSD-VPS plugin works for DigitalOcean, Linode, Hetzner, or any VPS provider. This one is currently at a 0. 1 release. So we have one tool supporting many platforms. If you want more control of the process, there's a slightly longer configuration-only mode. Installing a plugin automatically installs Django Symbol Deploy. We add Django Sip of Deploy to install that apps, as you saw earlier. You create the external resources needed for deployment on the target platform. In configuration only mode, the deploy command configures your project for the target platform but doesn't make any commits or pushes for you. You could take all the time you want to inspect the changes that were made.

Speaker 1: You can modify the configuration in any way you want. This is the output of git status after configuring for deployment to fly. io. This is why deployment is hard. These are all the things you have to get right for an initial deployment to fly. These are all opportunities to make a typo, to forget a step, to be bitten by outdated documentation, and more. These are really nice things to have done for you, even if you decide to make some changes of your own. When you're satisfied with the configuration changes, you make the commit, and then you run the target platforms deploy or push command. And because the configuration was managed by the plugin, your deployment should work the first time you try it, even if you've never used that platform before. So when you use automate all like we did in the demo, these are the steps that are carried out for you. All right, it worked.

Speaker 1: So uh is now deployed. Question. Question? Question? Question. Um I I'll take questions at the end. Yeah. So okay. Django Subdeploy is an abstraction layer between hosting platforms and the Django developer community. These are excerpts of the Django documentation pages for fly. io, platform. sh, Heroku, and DigitalOcean. This is where most people go when targeting one of these platforms. We hope you read closely and I hope that their docs are up to date. Ideally, each plugin would have several maintainers. It's especially helpful if some of these people know much more than I do about each particular platform.

Speaker 1: Plugins capture our shared experience and shared best practices about deploying Django to each platform. These plugins should capture more Django-specific best practices than what any platform provider could be expected to write into their documentation and tools. Maintaining the core Django Simple Deploy library mostly centers around keeping a clean interface between Django Simple Deploy and the plugins. That means providing clear and consistent ways to get information about the user's project, modify settings, add files, and more Django developers get working deployments the first time they try. They get the benefit of our collective experience and shared knowledge. Developers do need to learn details about how their platform works. They do need to visit their platform's documentation. However, they don't need to dive deeply into each platform's documentation before they can even get a working deployment.

Speaker 1: It's much easier to learn about a platform once you have a working deployment. This is in contrast to trying to learn enough about a platform just to get yourself to that first working deployment. Anyone can write a plugin. You can target a new platform or a new deployment approach for an existing platform. More on that in a bit Use cases. People new to deployment. I gave a lightning talk about this at PyCon in May. And afterward, a high school teacher came up to me all excited. He had a group of students who had built a non-institified reservation app for their school's culinary arts department. They were excited to use their app for the school's culinary night, but they got stuck at deployment because they just couldn't figure out all the steps in a reasonable amount of time. This is a hugely important example. These are exactly the kinds of people we want to keep in the Django community.

Speaker 1: When deployments work, we keep people in our community. When deployments never work, we lose people. Sometimes for a while, sometimes for good. Experienced people new to a platform. If you have experience with one deployment approach, it's certainly easier for you to make sense of a new platform's process. But you still have to read the platform's docs and hope you get all the little bits right. Instead of doing that, you can run deploy, then git diff, and read through all the changes that were made for that platform. It's an amazing way to learn about a new platform and then go back to their docs Teachers, writers, presenters. The abstraction layer is really important for teachers, writers, and presenters. If a platform changes its process and you use Django Simple Deploy, your resource is correct again as soon as any one person updates the relevant plugin. You can also teach deployment without a bunch of boilerplay.

Speaker 1: You don't have to skip over details. Instead, you get to discuss details about a deployment that's already working. The main point is you don't have to avoid deployment in your teaching and presenting work. Even if you've done 100 deployments to a platform, it's really nice to have the process fully automated. Platform owners, it might sound like we're doing work that platform hosts really should be doing for us. But Django Simple Deploy sits right on the boundary between the Django world and the platforms world. We're doing some of their work, but to our great benefit and in a way that we control. I think this is about the best balance of responsibilities we could hope for. How? So how does this work? Let's first look at what the core Django Simple Deploy library does. It first

Speaker 1: registers a plugin. Does the plugin support fully automated deployments? What's the platform we're deploying to? And what's the plugin name? We start logging. We make sure the command that we're processing is valid. We let the plugin do some validation. For example, is the right CLI installed if needed? Is it authenticated? Is it user on Windows or Mac OS? Linux doesn't need any special treatment so far. We inspect the user's project. What's the project name? Where's their project root? Where's their. git directory? Is there a clean git status? Where are the settings? What package manager is being used? What are the current dependencies? If automate all was passed, we make sure the user realizes what we're going to do on their behalf. And finally we call out to the plugin for the rest of the configuration and deployment work.

Speaker 1: What does a core library offer to plugins beside the information just shown? There's a bunch of utility functions. For example, you can easily add a new directory to the user's project. When a plugin calls add file, the file is written to the project if it doesn't already exist. If it already exists, such as a Docker file, core prompts the user for permission to overwrite the file. The plugin doesn't have to worry about handling all that, it just gets to call add file. Core is inspected how the user is managing dependencies. As a plugin author, you call add package and the deployment specific dependency is added to the user's requirements in the appropriate way. You can add multiple packages at once. For fully automated workflows, you can commit all changes when you're ready. Get confirmation is useful if you want to get confirmation before taking a specific step during the plugins

Speaker 1: work. For run quick commands, you provide a command, core makes the appropriate subprocess call. Quick commands should never appear to hang. These are simple subprocess calls, for example, setting an environment variable on the remote server. Long-running commands that might appear to hang are handled differently than quick commands. For example, create, deploy, and push commands. You can write output to the screen and to logs as appropriate, and there's a number of other utility functions as well. So let's look at how the handoff to the fly. io plugin works. In the fully automated mode, the plugin creates an empty project on Fly. It also creates Postgres database. It says two environment variables, onFly. io and debug. Fly uses a Docker file. Fly is motivated to show the smallest reasonable DockerFire

Speaker 1: Docker file. And they're getting started docs because every additional line in their Docker file is an opportunity for people to mess things up and end up with a failed deployment. Since we're automating the process, we can write as detailed and nuanced of a Docker file as we want. This is a great place to share your expertise. You could audit the DSTFLY. io Dockerfile template and suggest changes that improve how Django works in production. Fly. toml is a fly specific configuration file. We modify a number of settings for production, debug, static files, database. Most production deployments require a few dependencies that aren't always included in development environments like Gunnarcorn or White Noise. In the fully automated mode, there's some work to do after making all the configuration changes. We commit changes, run the deploy command, open the project so it appears in the user's browser, and grab the deployed URL.

Speaker 1: The very last step is to summarize what worked. In configuration only mode, we tell people how to review the changes that were made. We also tell them how to finish the deployment. This usually means running a platform-specific push or deploy command. We tell them how to view their deployed project. This might be an open command along with a deployed URL that we grabbed earlier. Finally, we tell them in a big picture way how to push any changes they make from this point forward. This is usually something like commit your changes and run the push or deploy command. In the fully automated workflow, we display this deployed URL and tell the user how to push subsequent changes. Writing a new plugin. You don't have to take apart an existing plugin if you want to write a new one. There's a tool that builds a new plugin for you so you can focus on the actual custom deployment code you're interested in.

Speaker 1: Here's a summary of the overall process. You clone or download the DSD plugin generator tool You run generate plugin. py, you install that new plugin to a Django Simple Deploy development environment. You run tests. You write your deployment steps, and this is the bulk of your work. You don't really have to think much about Django Sibyl deploy. You just drop into the generated plugin and script the steps for your platform or your specific approach to deployment. And a baseline of tests are included. You can expand on this set of tests to confirm that your configuration steps are working as intended against a typical Django project. The gener generated plugin has everything you need to package and publish your work. This is what it looks like when you run the generate plugin. py script. You're asked a few bits of information

Speaker 1: which are used to build out the infrastructure for a new plugin. You end up with a complete project. All the bits that reference the platform and plugin name have been filled in for you. The plugin generator does more than just sorry. The plugin generator does more than just find and replace your platform name, but you can see some of the work that was done here. Most of your platform-specific and approach-specific code will go here. And you'll also fill in the bodies of the prep and concluding methods shown here. So let's make a new plugin. Two live demos at once. Yeah See if my shortcuts work. Okay, so this is a DSC plugin generator, and we're going to run

Speaker 1: uh I had a UV. I changed my commands recently. So we're gonna run that generate plugin. py. Um we're gonna target a new environmentally friendly host called Greenhost. We're gonna call the plugin DSD-greenhost. We do want to support fully automated deployments, and I'm gonna be in the license file. So default location. Yes, it's correct. Yes, you can write that. Alright, and so it should be there. And so I'm gonna if you can't read this, it's fine. I just wanted to show that you can generate a project. So this is my projects directory, and we should be able to CD into that

Speaker 1: new project. And those are all the files. So that's an empty, a new empty uh plugin. I'm gonna go one step further. Alright, so this is my development environment environment of Django Simple Deploy. And so now I can pip install an editable version of that new Uh plugin. All right, so it is installable, which is good. And now we can run, I believe We can run the the set of tests that comes with a new plugin. So there's 18 tests that come that just basically show that it's working, that it's able to interact with an actual Django project.

Speaker 1: Um so gang. Uh live demos worked Just some caveats. Django Simple Deploy currently expects new plugins to be in the same parent directory as the Django Simple Deploy development directory. It's just for testing work. Um core Django Simple Deploy runs tests based on the plugins it finds in the development environment. Alright, so what next? What's next for this project? Where are we now? Where are we headed? Django Simple Deploy is stable at a post-1. 0 release. This means the API is stable and bug reports should be addressed promptly. The Fly, Platform SH, and Heroku plugins are included in the Django Symbol Deploy organization, and there are stable releases as well. The DSD-VPS plugin is currently under development.

Speaker 1: Designing a generic plugin like this is much different than targeting a path. DSD CodeRED is an experimental plugin. The CR deploy command is one of the few platform-provided tools that actually does much of what Django Simple Deploy does. I plan to bring this to a 1. 0 status in a way that complements what the CR tool itself offers. Wagtail support. I recently implemented support for Wagtail in the platform. sh and Heroku plugins. The Fly plugin should support Wagtail soon, and DSDVPS should support Wagtail at or shortly after the 1. 0 release. So my current priorities, PR. Sorry? PR. This product is only meaningful if people are actually using it. So we all like the idea of this. We've got to start using it if it's if it's truly good. Bring DSD VPS to a 1.

Speaker 1: 0 release. People seem much more excited about a plugin that targets VPS deployments than PaaS providers. There are so many options for how to approach VPS deployments. I'm aiming for one consistent, reliable workflow for a DSD VPS 1. 0. Post-1. 0 work will focus on expanding options that support different deployment approaches. There's also room for a wide variety of uh third, fourth party plugins that support deployment to a VPS. Uh clean up the core plugin interface. There's some rough edges around the core plugin interface. These are mostly almost entirely internal. Developers notice them, but end users should not. Django Simple Deploy needs to support both end users and plugin developers. Improved handling of dependencies. I started Django Slip deployed long before UV came out. I started it

Speaker 1: back when PipEm had some popularity. The current approach works for a lot of projects, but it really should be audited and updated to prioritize current dependency management. workflows. Target additional platforms. There's a number of hosting platforms that would probably benefit from a plugin. Render, Railway, Python Anywhere. Django Simple Deploy generates a log that's really helpful for troubleshooting. But when you write and maintain a plugin, you also learn a lot of things that end users would benefit from knowing. One of the POST 1. 0 features I've been wanting to implement is a friendly summary. It's an HTML file in the log directory that contains a version of the final success message. It also includes links to the most relevant parts of the platform's documentation. This sounds like something that might be focused on beginners, but it actually saves everyone a bunch of time and a headache.

Speaker 1: Consider funding. This entire project has been developed in Sprints when I have time to focus on it over a period of years. I'd love to work halftime consistently on D. GoSymbol deploy. and polish all the rough edges until we have the Django centric deployment tool that we'd all like to have. This talk isn't the funding pitch, so I won't add much more than that at this point. But it's worth noting that we've seen really good open tools evolve in our ecosystem when there's unashamed funding for consistent focused work. So, how can you help? Use Django Simple Deploy. If you need to deploy a project and don't already have a specific deployment plan and one of the currently supported platforms works for you, give this tool a try and see if it works for you. Tell others about it. If you've already got deployment figured out, keep this tool in mind next time someone asks how to deploy a Django project.

Speaker 1: Contribute. There's so many ways to contribute. Tell us what you like to see. Platforms, deployment approaches, configuration options. If you audit a plugin, if you know a platform or deployment approach that's currently supported, consider auditing the plugin. You can read through the deployment steps, or you can simply deploy a sample project and see the configuration choices that are made. Help maintain an existing plugin. Even if the overall plugin structure seems a bit complex, the blocks of code that do the actual configuration work are fairly accessible. Please consider becoming a steward of a plugin that targets a platform you know well or would like to know better. Write a new plugin. Plugin generator gives you a really nice starting place if you have your own ideas. If you want to do this, my doors are wide open. Please let me know what questions you have, and I'll help you get up and running quickly. Talk about deployment.

Speaker 1: When you talk about deployment, please mention this tool and the plugin ecosystem when appropriate. If anything doesn't work, please report it. And write about deployment. If you're a writer, please consider running a deployment and share your thoughts about how a tool-assisted deployment compares to the standard practice of reading through tutorials, guides, and documentation in order to get a working deployment. Again, the goal is not to avoid the platform's documentation. The goal is to read through the documentation alongside a working deployment rather than in hopes that your deployment might work at some point. Alright, so I said I'd explain the picture if I have time. I've often thought of deployment as a trip into the mountains. When people start a project, it's like a long, steady climb up a mountain. When critical functionality starts to work locally on your computer, it's like reaching that first great view on a mountain.

Speaker 1: Then you think about deployment, and it's like that far-off peak without a clear path to get there. If we make a clear path, people will stay with us in the Django world. If we don't, many will simply turn around after that first satisfying view. But they're wishing they could have gone further. Let's build a nice path for them. I'm here through the sprints. Happy to talk to deployment anytime today during sprints or online in the coming weeks and months. Thank you.

Speaker 2: Thank you, Eric, for that uh presentation.

Speaker 3: Thanks. It seems to me and the perfect tools to be used in the Django Gurse workshop. Um have you ever thought about uh proposing kids uh to integrate in the workshop or instruction or if It seems some there would be something that you you suggest to start using hit.

Speaker 1: Yeah. Uh so the question for anybody who didn't hear it, he said uh I believe he said that uh this seems like a very appropriate tool for things like a Django Girls workshop. Yes, uh every project I build, I make an issue called parking lot and I just dump all the ideas that I can't implement immediately. And one of those lines is like look at how this applies to the Django Girls workshop. I believe they deploy to Python anywhere. I have not written a plugin for that yet. open pitch, run the generator and write a a tool for it. But that is the bigger point of that plugin system. Like if we wait for me to write these plugins. We're going to plot along. If I can focus on core and making that interface clean and just resolving anything that comes up for people, we can create a better deployment story for a whole variety of users and a whole variety of deployment approaches.

Speaker 4: I how does um authentication work? I noticed in your demo, no interaction for password, you didn't push, you know, uh on a pop-up or anything. Do you assume that your target platform has a CLI that's already authenticated?

Speaker 1: Great question. So I show it as three steps because that's the nice like shiny demonstration version. The longer list of requirements is you need to be using Git to manage your project. You need to have the appropriate CLI installed. You need to have an account on the platform. Those are the bits that like you have to do before any deployment. Um the pass pro one of the things that makes the pass providers easier to write a plugin for than something like a VPS is they figured all that out. So I am logged into the FLY CLI. Um and so I have authentic authenticated through that CLI and that's managing that part. Current 0. 1 release of the DSD VPS uses like a username password for um logging into a fresh VM on a VPS provider.

Speaker 1: And I'm working on the SSH key part. And then like we can we can build all kinds of workflows. And some of those will fit into one plugin with some CLI args or something like that. And some of them will be more appropriate to just put into a separate plugin.

Speaker 2: Well that's uh about all the time we have for questions. Eric will be in the hall if you have any more questions. One more round of applause.