Wagtail Nest: Maintaining Community Packages Together

Speaker 1: All right, let's get started. Welcome everyone. The talk will be about 25 minutes, and at the end there will be time for Kuhn to answer your questions. So please prepare them and have them ready to place in the Q<unk>A area at the end. Thank you. With that out of the way, I'd like to welcome our speaker, Koen van der Kamp, who will be speaking about the Wacto Nest Initiative. Couldn't take it away.

Speaker 2: Thank you, Storm. Um I hope, and I will first check with Storm again, that I'm visible correctly. Maybe you can respond in the chat because I only see a gray screen. Yes, nice background. Thank you all. So uh here we go. You build a nice website, amazing looking website with Wagtail, and you'll find a third-party package that fits perfectly. And some years go by and you decide to update Wagtail And your problems start when you discover that that specific third-party package isn't maintained no more. It's incompatible with the latest Wagtail.

Speaker 2: This is a very frustrating experience, and I may imagine you or your colleagues had this before. Well, me too. My name is Koen Koen van der Kamp, and I've been part of the Recto community for quite a while. I'm the organizer of several Recto Space events in the Netherlands. Um and uh yeah, there's also this t-shirt about uh including the first one in 2070. I'm a Wagdil core team member, a Google Summer of Code mentor for Wagdil. I'm also a package maintainer for a few packages, and I use Wagdil a lot at work. I work here at 4Digits in the Netherlands and we work with Byten, Django, and Wagtail.

Speaker 2: And I'm proud to say that our clients stay with us for many years. And that also means that stability and maintain uh maintenance is important to us. So over the years I've seen brilliant ideas and useful packages fall behind simply because their authors moved on or didn't have the time anymore. I've also experienced the friction that causes this. I feel the pain. So we, we the Wagtail community, can fix that together. That's what open source is for, and that's where the Wagtail community is for. And that's also what Wagtail Nest is for. So to give you a little background, the idea for Rektil Net started during Rektil space in the Netherlands in um

Speaker 2: 2022 and uh I had a conversation with Dan Braggins, um another core team member. And if you don't know Dan, this is Dan. Hi Dan. Uh and uh then if you're watching this, thank you for all your efforts. And uh Felix is saying hi. Uh Paddington is also still alive. His head is a bit broken, but uh that's okay. Felix is my son. He's six years old and he got this nice uh Pennington from then. So Dan and I were talking about those valuable packages that slowly get behind, but are still useful, and where the original author might lost interest or just had too much on his plate, his or her plate.

Speaker 2: And those packages just deserve community support. So we thought what could uh what if we could give them a new home? And that is how Wagtill Nest was born. We started drafting a short document to define some simple principles, how we work together and keep things open and how we make uh maintenance easier. We needed a way to ensure packages uh stay alive and trustworthy and we needed people to get involved. So a couple of the those principles are uh be nice. Focus on maintenance, maintenance. New features are a bonus, but it's more about not getting behind. We want to release often

Speaker 2: every merge pull request deserves a release. And we test against WagTil, the main branch, during nightly tests. And we automate as much as possible, including the releases. So we hope that through Wecil Nest maintenance uh main maintainers, original authors can have uh no longer have to work alone. So let's take a step back and look at Django and Wagtail. These uh projects take very good care of us. They are backward compatible, have uh depreciation uh deprecation processes. and they provide excellent release nodes and upgrade path. They guide you through the process and they set a very high standard and we can't realistically expect that from every third party

Speaker 2: uh package author. In reality, package always try to catch up with the latest Django or the latest Wagtail versions. Uh and in real in reality they're always a little bit behind the latest and greatest. In real life also, people scratch their own itch. So most package maintainers are people who build a Wacto project and then need something very specific. They had a problem, they solved it, and decided to share the solution with the community. And that's great. Those contributions make Wactor ecosystem thrive. But the burden of maintenance is rarely what they signed up for. They are enth enthusiastic, they mean well, but we all know how life works.

Speaker 2: There are many reasons why a maintainer can keep up and just as many way reasons for a package to fall behind. When that happens, what can you do as a person that uses that package? If your project depends on that package and it no longer works. With the latest uh Wagtail version, you have a few options. You can contribute back to that original package. This is the ideal solution because then we all benefit. Um but this can take time and it's not always time you have. And the second one is you could make a fork and then release your own package, but then The likelihood of having two unmaintained packages is great.

Speaker 2: So my personal advice is just copy over the source code of that package into your own package, into your own project, and maintain it locally. This way you remove the dependency and you have full control. And often packages are small enough and easy enough to understand. and to work with. Whatever you do, don't blame the original author. In Dutch we say you moet een gegeven paard niet voor de back slaan, which literally translates to don't live look a gifted horse in the mouth. And then here you think, what about what about other initiatives for collaboratively maintaining packages like Jazz Band or Django Commons? Well, Just Band

Speaker 2: focuses on Django and has very strict rules. A package must meet high standards before they are onboarded, and contributors must commit up front. This works very well for their packages, but it felt too strict for Wagtil Nest. And Django Commons is a new initiative. It wasn't around when we thought of Wagtil Nest. But it's kind of similar, it also is focusing about uh on team-based maintenance and maintenance and um uses the GitHub as central organization. Both these initiatives are Django focused and WagTil is only a smart part of Django. So Wagtail-specific approach just made sense. Maybe there are other uh initiatives I forgot to mention, and if you know any, uh please share them in the chat.

Speaker 2: And I'm not sure how many people are here. I get a couple of responses. Anyway, we wanted a light and flexible community uh driven approach And I like net Wagton Nest to develop its own life rather than being tightly steered or managed by me or core team. And um I also would like to mention that Wagtail is built on top of Django. And let's put this in perspective for a moment. And the way I came up to do that is uh look at uh monthly downloads. So Django has twenty-seven million downloads each month, which is like

Speaker 2: incredibly high. And uh pick the number for Wecdel now in your mind, and I'll show it in a moment. Wagtail has 415,000. So 1. 5 % of all Django projects also install Wagtail. if download counts are a good measurement. This is my own research. Maybe it's not completely correct. I like to be corrected. But my takeaway is that uh Wacto stands on the shoulder of a giant and the Django ecosystem uh is there for us to use, so use it whenever you can. So also join Django Commons

Speaker 2: or contribute to Jazz Band because that is also part of our community. Back to WactoNest. WactoNest aims to support specific packages, things that complement the CMS itself, while letting developers rely on Django and Django packages for everything else. Um let's look at the current Wagtail Nest packages. I just selected a few that are in Wagtail Nest. So the first one is WactoMath, and it has uh 500 downloads per month. It's not that much, but uh I picked it because it's an excellent uh example. of a project that

Speaker 2: is an itch for somebody and they released it in the public domain and got adopted by WactoNest. So if you need uh nicely rendered Math equations in your project, use WectoMath. The second one I wanted to talk about is WectoFont Awesome SVG. And um it has uh thirteen thousand downloads a month. So This has a l uh another backstory. A long time ago Ecto had a font icon and um then they tried to move to SVG. And um at that time there was also this package like the Font Awesome that uh also shipped the Fontawesome

Speaker 2: font-based um solution and when we wanted to switch away we want to have a drop in replacement for that package. So wait till uh Font Awesome SVG was born. Um let's and the download count is by the way uh three percent of all Wagtail packages So the biggest one uh is Wektel Model Admin. And it is uh 95,000 downloads a month. So 23% of all Wagdil projects still use model admin. Wagdil are two ways of working with Jenning

Speaker 2: models, uh snippets and model admin. and the core team wanted a single way of working. The choice was made to continue the generic fuse, the snippets approach. The model admin package is lifted out of Django Core and into its own package and found a home at the RectailNest. Note that most of these packages are quite mature and they just need the releases to get import fixes and small adjustments to stay compatible with the latest Wagtail. So, uh what's your experience? Which third-party package uh is giving you upgrade trouble? Which ones would you like to see in Wake Till next? And uh maybe you can share that in the chat.

Speaker 2: I'm curious what is used and what is frustrating you and maybe we can work with those authors to get more support to them. Maybe it worked to 2FA. Yeah. Yeah, there are probably some others. If something pops in your mind please share it in the chat I'll uh I'll review the list after this talk. Next, I wanted to highlight Wacktonest tooling. Let's take a look behind the scenes. We actually make how we actually make WactorNest work First of all, uh transferring packages uh to Wagtill Nest is done with

Speaker 2: GitHub repository feature. Um The transfer feature, sorry, I have to say GitHub repository transfer feature. And that makes it easy for original authors to move the package to React on Nest while keeping the history and issues and everything that is in GitHub. all contributors uh still are listed. And once transferred, the original author can help and review and do releases. For them it more or less stays the same. They still get full credit and can stay involved as much as they want. Um because Wagtil Nest is a GitHub organization, we can handle permissions. and uh collaboration. So we distinguish between global maintainers, people

Speaker 2: can manage, release and merge to any package. but we also have uh specific package maintainers, people who manage a specific repository. So the GitHub organization keeps things flexible but safe and not everyone needs access to everything. We also automate uh the publishing. We use GitHub Actions to automatically build and release and behind the scenes This system uh uses trusted uh publisher, the Python package index feature, uh normally to upload to a package to PyPy. um you would store a secret key locally and if that leaks anyone can uh publish fake versions pretending to be you

Speaker 2: With trusted publisher, PyPy can recognize GitHub itself as the publisher, so you don't need keys anymore and uh that risk is gone. Instead of saying here's my password, you say a GitHub can vouch for me. And this is exactly what we need when multiple people maintain a single package And then the Python package index itself, we uh require two uh owners on the package so that we never lose access. Uh then um a word about safety and security. In a perfect world we would give everybody right access and we could uh trust everybody. But we have to keep things secure. And that's why we only selected maintainers can review and merge and release.

Speaker 2: And those maintainers are of course the original author if they want to stay involved. And we also came up with all employees of companies where uh that supply a core team member. For example, all my colleagues at Four Digits or anyone working at Torchbox is a trusted maintainer. These people are people we collaborate with on a daily basis, we know there are professionals, and there's also the element of accountability When somebody represents a company, their actions reflect on the company too, and they will be corrected by that uh company if they misbehave. Note that we don't want WactoNest to be driven by the core team members only, because

Speaker 2: core team members have busy lives and we just WackelNest is a way to share the workload in the community. So contributing, anyone can submit an issue or a pull request and uh I really encourage you to do so do so. You don't have to be an expert. Every improvement, no matter how small, helps these packages to stay alive and healthy. So open up an issue if something breaks or confuses you, review a pull request maybe If you say like I tested this and it works, that's helpful for us to know that an open PR is uh is worth looking at and uh evaluating

Speaker 2: And give the occasional thumbs up if you like a contribution of others. And this is of course true for packages in ReactNes, but basically for all open source packages. your help is appreciated. Next a message to package maintainers, you're not alone. If you are maintain maintaining a recto package, first of all, thank you. You've already done something valuable for the community. And if maintaining it does uh not have your top priority, we get it. Life happens, projects evolve, priorities shift. You can bring your package into Wactal Nest and keep maintaining and developing it just like before. But this time you're backed by the community.

Speaker 2: You don't lose ownership, you gain support. Others can help review pull requests, do small fixes, automate releases, and keep your package compatible with the latest Recto versions. Together we can remove the bus factor. Maybe you feel you lose ownership and that's not needed. You can stay involved and also feel free to credit yourself or your company in the README and we'll just keep that as is. If you like to uh discuss things, join us on Slack. Just say hello. You can go to um Wagdel. orgs slash Slack and then uh you get the invite for uh for Slack and we are on the Wagtel Nest channel.

Speaker 2: So I hope by now you have a solid understanding of what WactoNest is, what any contribution to open source software helps yourself and the community as a whole. We trust a group of people to merge and release and we rely on GitHub organization, GitHub automation, and a trusted publisher. We try to release often. Above all, I hope that package maintainers know that they are backed by WagtalNest. So check out WactilNest on GitHub. And all the packages that are there. And remember, you are the Wagtail community. Together we maintain the ecosystem you rely on. And together we make Wagtail stronger. This was my talk.

Speaker 1: Thank you, Cool. Anyone has any questions , please put them in the chat or in the Q<unk>A area on Zoom. While people are typing up uh their questions, I do have a question. Um How do you handle security reports for uh vulnerabilities in packages maintained by ReactelNest?

Speaker 2: Uh good question. Um we use the uh or we like to onboard packages to use a template and that's the the Vectoil package template and there's a security node in it so the security MD and that points to the the Vectual security team So uh if there's an issue, it points to the security team and the security team probably uh I hope I'm in the security team as well, but uh we'll we'll divert uh the work to the appropriate people. Yeah.

Speaker 1: All right. Thank you. We have a question here in the Q<unk>A. What would you like to see in Nest?

Speaker 2: That's a good question. Um I would like to see uh more packages in it. Just I th I think if we collaboratively uh support those packages and uh they they're more stable, they are more likely to be used and there's less friction for all of us. So uh more packages uh definitely. Uh I also would like more contributors, people who can maintain those packages. Maybe there's that yeah, that's maybe broader question, but uh maybe there are things that people like to see in in Wagto in general and uh starting with a third-party package is most of the times a good way to

Speaker 2: uh start that ball rolling. So maybe the question would be what do we all need in Wagtail and isn't directly a core feature because I think Wagtel Nest would be a great home for that too.

Speaker 1: All right, we have another question here in the chat. There are packages under Wagtail and the Torchbox GitHub organizations. Any thoughts on moving some of these to WagtailNest? For example, Wactil Footnotes and Bactil Autocomplete?

Speaker 2: I think it kind of depends if they are maintained because uh I think Torchbox has a good track record of maintaining their packages and if that's the issue then there's no need to move them. They also uh have already their organization with PyPy and um No bus factor, so I wouldn't worry too much about using packages for the from the Torchbox namespace. Um but if they like and it and they like to open that up, uh they are welcome.

Speaker 2: In

Speaker 1: absence of uh other questions, I have um a question I'd like to ask. How can I help out? How can I become I mean say a maintainer of a package that I use often

Speaker 2: Yeah, um of course everything in on the internet is based on trust and we've seen recent uh things happening with packages uh in uh uh the note world and the the javascript world where uh social engineering is used to inject code into the package. I think that most packages in the Wagto domain are quite simple to understand with Python being a high-level language and not too complex things that are done there. So uh I think this uh Risk is low, but still if we need to onboard somebody we don't know, it's based on trust, so just start with contributing just start with making these PRs and uh

Speaker 2: contribute and become an active member in the community and if people know you and uh they trust you, then you're likely to uh be invited as a maintainer if you like to do that. So uh yeah, I think that's the way. Anyone uh who is a maintainer already can invite somebody. So you can just go to the current maintainers.

Speaker 1: All right, thank you. We have one for a questionnaire in the chat. How can we help pair-up folks looking to get started with something in the Wacto ecosystem with the people who've created packages but are having a life happening? says the fellow with a 13-year-old

Speaker 2: kiddo. Uh I'm I'm not sure if I'm uh understand the cor uh the question. Like

Speaker 1: Like it's more about connecting people. How can we uh bring these people together?

Speaker 2: Uh yeah, I would say hang out in the Wagtill Nest channel. That's not too busy. These kind of events. Yeah, you're welcome. I just joined Tim Allen. These kinds of events and also real life events, if you have the opportunity to join, do so. And yeah, I think that's That's a great way to start and um collaborate, be open.

Speaker 1: I have sorry, I have one more question uh that I would like to uh to ask. say that I have a package that I need help with um maintaining who would I reach out to to get this transferred over to Wector West? Could that be you or someone else?

Speaker 2: Yeah, probably me. But uh anyone who's already on the on the contributor list in Wagdle Nest, uh you can see in the GitHub organization who's uh involved And I think anyone can help you transfer to Wagtail Nest. Like I said, we use the WagtailNest transfer of the transfer feature. So You can transfer to Nest and we can just accept the the transfer. Yeah, that's quite easy to do. And uh probably good to first chat about it

Speaker 1: All right, thank you. Any more questions?

Speaker 2: Seems a wrap.

Speaker 1: All right. Thank you very much for this talk. You're welcome to everyone else. Please enjoy the rest of the conference.

Speaker 2: See ya.

Speaker 1: Bye-bye