The state of Wagtail 2022 - Tom Dyson

Speaker 1: I would like to bring forth our first presenter, Tom Dyson of Torchbox, who will be talking about the state of WegTil 2022. There's nothing really groundbreaking in this book. Has anyone seen it? Has anyone

Speaker 2: enjoyed it? It's uh it's written in a slightly annoying style like most American business books. But there are a few ways that it's helped us become a more effective organization. One of those is around metrics. The author tells us to imagine you're on a desert island somewhere. All you have is a piece of paper with a handful of numbers on it. These numbers must allow you to have an absolute pulse on your business. What are all of the numbers that should be on that piece of paper? For a business, these numbers could be incoming leads or client satisfaction or staff happiness. For an open source project, the metrics aren't so good.

Speaker 2: One metric we've used is GitHub stars. The last time I gave this talk in person, we were celebrating 7,000 stars. And today I'm happy that we've we've almost doubled that. But is this growth fast enough? Do GitHub stars even matter? It's hard to know, but the fact that they're increasing and at a faster rate than before is Comforting. And GitHub styles are definitely a factor that people who choose technologies take into account. But it's not the only metric that we track. I'm afraid this this the numbers on this might be too small for some of you at the back, but I'll read out some. The Ragtail Core team agreed to focus on the following numbers.

Speaker 2: New issues, closed issues, opened and merged pull requests, contributors, and star count. Every week the call team meets and we have a fixed agenda item to review these numbers We have a semi-automated process that extracts them from GitHub but then requires someone to post them in. Actually I think that manual part is quite useful because it uh the at the at the moment at the point of putting them in you think about the numbers and and what they might need. I'm really pleased that we have these metrics, but I'm not sure that they're always the right ones or that we're dealing with them in the right way. Obviously More GitHub stars is better than fewer GitHub stars. But what about pull requests? Do we want pull requests to go up or down?

Speaker 2: As with most statistics, you can spin them both ways. We could feel like a high number of pull requests meaning that we have an engaged, enthusiastic community who's constantly contributing stuff Or we could feel like we're not putting enough effort into maintenance and supporting the people who are submitting the requests. Similarly with our issues, our issue count is slowly growing. Do we celebrate that because it means we have lots of eyes on the different edge cases, the different things that can go wrong in software? Or do we do we worry about it because it means we've got ugly code? Carl has been doing some interesting work on this, pulling all the GitHub stats into a big database which is going to allow us to build more sophisticated and useful queries.

Speaker 2: For example, what's the average time between pull request, submission, and first response? Or more precisely, has anyone waited longer than a certain number of days or weeks for a response to their pull request? I know how this goes on the other side. I've often submitted small pull requests to other repos, sometimes just typos on the README. And sometimes those PRs are ignored. Of course, that's fine. They don't owe me any attention. But But when other open source maintainers respond quickly and with a friendly message, I feel very motivated to continue contributing. I think we're doing an okay job here, but I don't think we're always doing a brilliant job.

Speaker 2: Another one is the average response time for support questions. And for support we use mainly Slack and Stack Overflow. This is a harder one to measure, but I think we could find a way of doing it through scraping and API access. My impression is that the White Talk community is pretty strong. There are a handful of people, and some of them are in this room, who do an amazing job of supporting people on Slack and Slack Overflow. But again, it would be good to set a target for the average and maximum response times and keep finding ways of bringing these numbers down But the metric that I think is completely uncontroversial is the number of contributors. Last year we set ourselves a target of 500 contributors

Speaker 2: before the start of 2022 and I'm happy to say that we we hit that target a few months early. But if we look again at the figures, you will see that our growth is pretty linear. So this is showing the weeks that have gone back the last 20 or so weeks and Most weeks we gain between 0 and 3 new contributors. Every now and then, like halfway along in 6th of April, we had a spike. We had eight new contributors one week And I hope we'll see another one after the sprints that we've had in the last two days. But then it settles down again. For me this is the most important metric because it shows the health of the community and also because

Speaker 2: larger number of contributors should lead to more rapid improvements. So, how can we gain more new contributors? This is a question that we ask ourselves in the World Record team and I'm sure all open source projects think about the same question. In our team we've come up with four main answers to this First is triaging. So this is the uh the initial work that happens when a when a pull request or or an issue comes in. And um it's it's Often quite a simple task of identifying what sort of issue this is and whether or not it's something that could be easily handled by a first-time contributor. Often it's tempting for the more expert developers to just deal with these issues.

Speaker 2: But Increasingly we're uh we're asking ourselves to to resist handling the easy things and instead tag them as good first issues. and perhaps provide a description uh that will help someone who's coming into the community to tackle. The second is sprints. So for the last few days here in Ireland there have been a group of people working on uh working on white other issues and and and the concentrated and accelerated way And similarly with these sprints, uh it's it's tempting for people to uh for the for the for the experts to work on solving these problems, but increasingly we're trying to focus on helping new people in the community get over the hurdles of creating pull

Speaker 2: requests, sometimes their very first pull requests. The third one is around developer environments. Finding ways to simplify the setup. So getting to that that allowing people to start solving the problem once they've done the boring tasks of uh of creating a local environment I think we've done some good work here. For example, we have the Docker environment, Wagtail, Docker Wagtail Develop. Maybe some of you experience that. That was created at a previous screen. For the one-click GitPob setup, which I particularly like, so this is this creates a complete developer environment for White L in your browser in the cloud. And uh and Boom has been been done

Speaker 2: very well on this. And the last one of our list or is uh investing in programs like Google Some later. And others. There are there are similar projects. Out each year is the one that we're interested in. And this is about Creating, I think increasing the diversity of the contributors that we have in our in our pool at the moment and expanding it internationally. Before I leave the subject of metrics, I want to address the question of accountability. This is one way that open source projects are very different to commercial environments. Our community is made up mainly of volunteers, some of whom are supported by their employers, but most of whom work on Wagtail at least partly in their own time.

Speaker 2: This might be because they they find it enjoyable or because it's an interesting technical challenge or because they want to build their skills. But generally it's for altruistic reasons. As a group of volunteers, we can assign actions to each other and sometimes we might. For example, in our WebDo4 team meeting we might say everybody in the team should triage three actions before the next meeting. But if people don't carry out those actions, there aren't any consequences for them. Of course, this is the same problem that all voluntary organizations face. In general, there's enough goodwill and energy to keep things moving. But I think the the lack of accountability in open source and voluntary organizations is one of the reasons that big open source projects like Wagtail

Speaker 2: Need commercial sponsors to succeed to provide kind of background of that graphical team. Luckily, Michael has many examples of this There are lots of different ways that organisations can spend money on open source with results that are mutually beneficial and which can help to create accountability. Again, some of the organizations on this list are represented in this room. Torchbox , the company that some of us here work for, that creates WhiteL And is investing more and more. I think we've doubled or even trebled our investment in Whitefeld in the last year. Companies like Four Digits, Fabric, Lab Digital, Hybetha, Rodacast who sponsor or run conferences like this and who

Speaker 2: give some of their time to employees to work on the products which in turn are useful for their clients. And at the bottom of the list, people like Mozilla and Google and Multifool and Ugov who directly sponsor features, which uh which is kind of progressive and generous, but also makes good sense for them because uh Because Witel is a very important tool for the big websites that they build and it's in their interest for it to continue to improve. We need to keep building these sort of commercial relationships, but it's not a task that I think many open source developers would relish. I think it's an important part nonetheless of creating a sustainable community. And then the flip side of accountability is burnout. This is something that we hear more and more about in the open source community.

Speaker 2: Contributors who have given many lives, many years of their lives to projects, but then give up in exhaustion. Sometimes because of rude people criticizing decisions or demanding changes for their very specific use cases. Thankfully, we haven't seen much of this through the mic tab. Uh just Wacker doesn't seem to suffer as much from angry entitled people as many other open source projects. Um maybe you'll maybe you'll tell me I'm wrong and you've you've you've experienced this, but I'm I'm grateful that I haven't really. I don't know really why this is other than that Python and Django seem to attract kind and accessible people. But I do think that it's it's important that as Wagtail grows, potentially outside the

Speaker 2: the the Django and Python communities, that we maintain a culture of positivity and support. Okay, that was the the big picture stuff. Next uh for the next I want to cover six headlines from the Wagto project from the last year. And there are lots more than I could have chosen from and I'm sorry about your top six, but uh these are the ones that felt most significant to me The first is White Tail 3. Big release for us for many reasons. One, well maybe the two of the top things that that many of you will have noticed will be the numbering change. So we have adopted semantic versioning. This means that we're going to use the big number for

Speaker 2: big releases, particularly that have um backwards incompatibility. uh constraints um all that have significant editor UI changes. Um and this does put us in the slightly strange position that uh having gone four years between White Tail 2 and White Tail 3 We're actually going to go from Wagto 3. 0 to Wagtail 4. 0 in three months because uh the next release is also gonna have some significant UI changes. But um as as Matthew said when he was uh outlining his case for making the switch to semantic versioning, this sort of uh weirdness is is uh is kind of what you have to What i i i it's it's it's a weirdness that you have to adopt, you have to be willing to adopt with somatic version. Um the next is that things look different.

Speaker 2: So we've we've got some new colours. We have uh some of you may may feel a bit sad to see the uh the the the white tails green disappear and and be gradually replaced but um uh we're confident that it's uh it's a kind of a fresher more modern and more accessible And then there are cool features like a splitter, the text splitter, which was actually concept of this at a previous WorkTale space and now that's in Worktail 3 and image duplicates. just kind of gradually chipping off the the subtle annoyances that uh editors might face. So Wactor 3 did release something for us to be proud of in the last year. Another one is that we now have a full-time community manager and I'm sorry she's not she's not here today.

Speaker 2: This is Megan Voss, she's based in the US Um she is uh um uh a writer and an educator and um she we we became aware of maybe when she started asking really interesting questions on the Wagto Slack, how we can build her own sites on Wagto. And we were I guess a bit inspired by to for this role by Divio and Daniela, who were who Daniela Proshido who has been at many of these events. and did an amazing job as community manager for Judio and Django CNS. And I think it's uh it's an important move and quite a big step for us to have someone who's who's going to be full-time thinking about how can we grow and support our community. And

Speaker 2: uh I'm sure you'll be we'll be hearing a lot more from Megan in the next year. The next is Google Summer of Code. So uh I guess most of you are familiar with this project. It's something that's been running for I don't know ten, fifteen years. Google give money to students from around the world to work on open source and um it's it's not a huge sum of money but it's a it's it's a meaningful amount and uh and it's becoming a really important way of getting people into open source. Last year we had a lot of success with this. We went under the umbrella of the Django project. and uh we had we we had three projects. In fact Google awarded us two and then we sponsored a third one and those projects were really successful. And um we had uh Tibyan

Speaker 2: in Sherman and Aldan. We all worked on really interesting features Tidia and Naldan both gave talks at PyCon US, and their talks are in the in some of the most watched talks at that conference Tilean's now working full time on Django. Um Shohan is continuing to develop cool features. It's uh it's you know the the process of of of mentoring these people is really rewarding but then but then seeing the outcomes is is even more exciting. So this year we've gone a step further. We're no longer going under the Django umbrella. White Tail submitted as its own project to Google Tumblr Code and we had an amazing amount of interest from candidates. We submitted I think six and we were awarded three.

Speaker 2: So three projects with mentors and sub-mentors and applicants. And uh there was one other that we just thought we we we were so excited by the project that we wanted to sponsor itself. So GoogleTalk is is Paying the same stipend that the Google would, but it's going through otherwise exactly the same process with methods and the same process. So uh we're gonna have by the end of this uh Windows Py Contrast support. So this is a big step for the continuing focus on accessibility in Wagtail We're going to move the editor's guide out from its section in the docs into a standalone project. So creating a really rich and useful document for your clients, hopefully for the editors.

Speaker 2: A gnarly one is the toolkit for stream field data migration. So you you might have experienced this already, the work that you have to do if you're especially if you're taking content from kind of more traditional constructed data into Streamfield and then finally uh a a a a thorough unification of the UX so we're going to start on this and we're going to deliver this through the whole of Brightel. So this is It's Google Summer of Code. Summer is a is a bit of a tricky word because actually for for many of the participants it's it's not summer but it's uh it's a European summer, it's starting starting soon and uh you'll be seeing a lot of the output of this soon. Okay, I'm near the end of my list. We created a vision for Wi-Fi. Hopefully some of you saw this. We presented it at uh one of our uh what's new in whacktower webinars.

Speaker 2: Um I won't go through it all now but um uh going through this the process of creating vision is is really interesting and um challenging and it's hard to to focus on a few things and then to express them in simple terms but I think we've we found that process really valuable and we've come up with a document which I feel stands up well. It's just a one year vision because uh because it's hard to think five years ahead and fit. technologies like this. But I really encourage you if you haven't seen it already to look at it. Here's the URL. It's in a sort of cartoon style. And we've had some events in the last six years as well. So in the last year, I mean. There was an in-person event in Cleveland for the Rector Space US. Uh PyCon was we were was Wacon was very highly represented. We're all here now at Wacospace

Speaker 2: Netherlands and we've got DjangoCon coming up. And um you know a year ago it was kind of hard to imagine that this should all be happening again and I think uh It's fantastic that listen, real life events happening again, that work toast are very well recommended. And finally on my list is Worktail Builder uh which is um a project that we've been working on a torchbox and uh which I'm very excited about but I'm not gonna say any more about because it will spoil uh talk just about to happen But it is, in my opinion, one of the uh one of the big feast of news in White Shelf. Right. And after all that, I'm going to end on uh maybe a slightly gloomy question Will wait till die? I think it's uh uh

Speaker 2: I think it's important to acknowledge that technology moves on. I guess I am one of the um the oldest person here or certainly in the top few. And so I've seen the the birth and death of a few different technologies and I'm sure many of you have too. Uh Torchbox The first dynamics stuff we were building was on Pearl, then we really got into Cold Fusion. And then in late 2000s Torchbox adopted Drupal , which is a really successful CMS, thriving open source community, especially if you're in the non-profit sector where we're focused. And it was very good for Torchbox commercially to become a sort of leading Drupal agency. Similarly, for digits

Speaker 2: are hosts were really big in the Plone community and I think some of the ways that they they learned how to run events like this and to be such kind of excellent open source participants was through their through their involvement in the Plone community And but in many cases those those those technologies it's not it's not true to say that they've died but clearly they're not uh then they're on the way. And there there was a there was uh a phrase in in Drupal It might even be like a a subheading like a like a slogan, which is come for the software and stage of community. Which is a nice it's a nice idea And it's something that resonates. I think it's a dangerous idea. And it's not one that I'd like to promote for Wagto. Because I think that if we're all coming to Arnhem, if we all come to Arnhem again in ten

Speaker 2: years' time, it shouldn't be because you're all lovely and because Fortigets puts on a big party. It should be because Wagtail is still the best tool for us to build websites for our clients. And so then to help ensure that Wagtail is still the best tool, we should ask a different question. Not will Wagtail die, but why might Wagtail die? Here's some some ideas I've got, some some reasons that you could give that that Wagtail might die. One is that Python could lose favor. So uh this is something Python's really helped us. You know when when when we launched WhiteDell twenty fourteen, twenty fifteen, Python was relatively niche and then you know it's exploded since because of dead science and machine learning

Speaker 2: and because people just recognize it's a great language. But it's possible that Python itself will be supported by cooler, newer, faster languages. Django we could store, so um maybe there's already a sense that that the the development of Django itself has uh has decelerated. There aren't as many exciting ideas in Django. It could be that everyone moves to hosted services in the same way that probably many of you don't run your own email servers anymore, but you use Gmail or Office for your organization, similarly people will feel that they don't be want to be running web servers for their for their websites. Or it could be the websites themselves die that um just just like people But people are um rarely run their own blogs now, they just tweet or Instagram

Speaker 2: or or or use medium. The same thing could happen for the big one. conditions. Um so those are those are some reasons. I don't really believe in any of those. I I don't think Python will lose favor, at least at least not in the next ten years. Um, I think it is possibly true. My take on that is that Jagger has really solved most of the important problems that it set out to solve. And And that could just be a good thing. I mean it's it's a it's uh it's what some what you might describe boring but stable technology. And I think that actually there'll be a move against some of these hosted services uh as people become more conscious of uh the Privacy and and data ownership. And I think actually here in in in Main Lab Europe you would be ahead of us in the UK and the US on this

Speaker 2: And I don't really, but I think I I can't see websites disappearing. And in fact, again, there seems to be perhaps in some cases a move back to traditional websites. I'm not denying that any of these things could happen, but I I don't really believe in them. I I guess the the reasons that that are generally die are probably hard to predict. But the one that I think is is most likely is that something simpler will come along. And and I guess one of the reasons I think this is thinking back to when Magtano came out. When Magtot came out there already was an established successful Django CMS called Django CMS. and uh had many features, it had uh uh a lot of users, a lot of contributors, and um and Wagtail actually Uh you know, it didn't have many of the things that Django CMS. When Mikefield came out didn't have Streamfield.

Speaker 2: Um, it didn't have uh uh internationalization, it didn't have an API. I think one of the things that appealed to people with with Wagtail was that it just felt lighter and fresher and easier to get into. And especially as a as a Dangerous developer, it felt like It felt more natural. I don't I'm you know I'm speaking for everybody else, but uh that that that's my impression, the reason it was adopted despite that. And and although now as Wactor we might feel that We sort of created this big motion around us because we have so much attention to detail. I mean when I when I think about some of the PRs coming in now around something that was around yesterday around accessibility on tool tips on on relative dates. You know these are not problems that you think about when you when you're creating a new CMS but they are important problems that and it's because it's then endless feeling

Speaker 2: like chipping away and making things better And when you do all that way, you think, how can anyone compete with this? Because uh because you know it's going to take them years to get to the stage where they're worrying about accessible tooltips on a really good base. Coc, the reality is that people are always attracted to things that are similar and easier to use, and up. And I guess that's where I want to end, is that as well as all the interesting work we're doing on building new features and growing our community and uh and and finding new ways to build cool websites with Wagdell And we continue to focus as well on simplificity and speed and the joy of first experience. That's it for me. Thank you very much.