Maintaining Demystified

Hi everyone, my name is Katherine Michael. I also go by Katie. My talk is maintaining demystified A few of my affiliations. I have been a part-time Pin X maintainer and release manager for a few years. I've been a Python Django React consultant for the Wharton School. I have taught some introductory Python programming through Stanford. I'm a member of the DEFNET board, and I am a DjangoCon US website chair. I'm going to talk about the fundamentals of maintaining, a case study about maintaining the DjangoCon US website, a case study about maintaining PINX, and some lessons learned. I have an assumption that you already understand the fundamentals of using GitHub and the fundamentals of contributing.

If you need more information about these subjects, I have a talk called Get a Jump Start on Collaboration and Code Review on GitHub, and there are several versions in my GitHub account. Firstly, it is important to realize that maintaining is not only about code. It is also about people, community, process, communication, and kindness. Put yourself in the shoes of the contributor. Make contributing as frictionless as possible. One way to know if you want to contribute to a project is to use the GitHub Open Source Guide. There is a checklist before you contribute. Maintainers can use it to improve their projects. Some signs of a healthy project. There is a welcoming atmosphere.

There is recent activity. The responder The maintainer responds quickly. The maintainers reduce the issues and pull requests. An ideal process. The future contributor finds the project agrees to the code of conduct, reads the contributing file, uses the docs and issue and pull request templates, creates an issue or pull request, and the maintainer responds quickly The first step is for the maintainer to create the community health files. A README gives general project information. A license tells you the legal terms under which you can contribute and use the code. The code of conduct is a set of rules

outlining the expectations and responsibilities of contributors, and the contributing file gives information about contributing. Documentation is very important for the future success of a project. GitHub did an open source survey in 2017 and they found that documentation is highly valued. often overlooked and a means for establishing inclusing inclusive and accessible communities. I've made a couple of diagrams that I hope will give you an idea of the process of working locally by a contributor versus a maintainer. The shared repository model is normally used in a GitHub organization.

A maintainer does not need to use a fork because he or she has write permission to the repo. The maintainer can clone the shared repo using the shared repo URL. The maintainer can make his or her changes probably in a branch and push the change to the shared repo. The maintainer can create the pull request directly in the GitHub organization. The fork and pull model is normally used in a contributor's account. Because the contributor does not have write permission to the GitHub organization, he or she needs to fork the repo into their user account where they have write permission. The contributor forks the repo. The contributor clones the fork using the fork URL. The contributor can make his or her change, then commit the changes to the fork, probably in a branch.

When ready, the contributor submits the pull request to the GitHub organization. The compare branch should be the contributor's branch. The base branch should be the branch into which the changes should be merged. The creator creates a pull request title, maybe a description. The contributor clicks on create pull request. Now let's switch to the maintainer perspective. When a contributor creates a pull request, the maintainers of the repo will receive a browser notification or email to inform them that there is a pull request. Follow the link to the pull request page in the browser. Review the information about the pull request. You can see the title and description and click on the files

changed link to see all the changes made. Below that will be a link that says command line instructions, a merge button that you can click to merge in the browser when ready. There are some situations when you need to fetch the pull request branch in your local development environment in order to run the code or run a test or work on the code in order to merge it. When you click on the command line instructions link, it will open a set of instructions for how to review and possibly merge the pull request in your local development environment. If no change change is needed or the change can be made in the browser, make the change in the browser and click merge. If no change is needed, go back to the browser and click merge.

If a change is needed, ask the contributor to make the change or push the update yourself, then click merge. If you need to run the code locally and you follow the entire instruction set, you will manually merge the pull request branch locally into the branch it is intended to be merged into. and push it to that branch on GitHub. I do not do that. I only follow the part of the instructions to fetch the branch. I ignore the instructions in red. If I verify that the change can be merged, I go back to the GitHub pull request page in the browser and click merge. For one thing, the main branch is often protected. Also in the browser you can simply push a button to revert the pull request.

The workflow we have been using is basically GitHub flow. It consists of merging pull requests into a main branch. Some projects use both main and develop branches. In this situation, developers merge their work into the dev branch. Eventually the dev branch is merged into the main branch. If the main branch is deployed, the changes are live. Some recommendations A very common license is the MIT license. The MIT license is non-copyleft and gives the user a lot of freedom in the use of the code, including making the code private. A copyleft license usually requires that the code remains open source. For this reason, copyleft licenses can create a business risk.

Ethical licenses are a very interesting subject in my opinion, but sometimes the terms of ethical licenses are difficult, difficult to enforce. GitHub has an excellent resource called Choose a License. In the appendix is a table with a comparison of licenses. One difficult aspect of software security is that the software supply chain is everything that makes up your code, including the GitHub organization, the project, and the people who have access to it. Including the dependencies and external hosting. Therefore, a project needs a security plan with several parts Some examples for the individual level, two-factor authentication, limited access, for the organization level, scanning of secrets and dependencies for vulnerabilities

For the project level, new releases incorporating the latest security patches in a project security plan with information on how to report vulnerabilities. If you want to start measuring your project's health, there are special metrics. Chaos, for example, is a collection of metrics that you can use to create a health plan for your project. A list for future reference of more useful GitHub features, issue and pull request templates, and global community health file repo. For project management, project boards, milestones, issues and labels. Communication methods, wiki, GitHub pages, JEKL, Discoverability, Labels, Taging, Topics, and Blog posts.

A case study, DjangoCon US website. DjangoCon US website has been an opportunity to learn the fundamentals of maintaining and grow a community of contributors. I spent a lot of time looking through GitHub and I realized that many conferences are looking for volunteers who can maintain their websites. So I got involved with DjangoCon US. This is a picture of me with some of the other organizers. I wanted to take my GitHub knowledge to the next level by learning how to be a maintainer. So I asked the DjangoCon US chair if I could be a maintainer of the website. She said sure and invited me to be the website chair, which I accepted. One of the strategies that helped me learn the process of maintaining was that the other maintainer left the easier pull request for me.

I learned how to fetch a pull request locally, run the code, push an update from a fork to a pull request, merge a pull request locally, and push a pull request to a main branch. Because a JUKO website is very easy to use, I was able to learn the basics of being a maintainer. And we were able to drastically increase the number of contributors from 23 and 2016. to 65 in 2020. I created the new contributing docs and I helped some beginners make their first contributions. And a few lessons learned from DjangoCon US website. It is ideal to learn in an environment of psychological safety.

Fortunately for me, I could not have stumbled upon a nicer community. It's okay to have a beginner mindset. Make mistakes, keep improving. A beginner perspective is valuable. Project veterans may not be able to see the project through the eyes of a beginner. A beginner can improve the project for the next person. For example, create installation docs. Achieving 10x results. People often talk about the concept of 10x results. It is important to remember that some of the skills that create 10x results Do not involve writing code. A few examples. Maintaining the docs, publicizing the project, mentoring collaborators. When you learn a new skill, you have the opportunity to teach other people and impact the world.

After I learned how to maintain the DjangoCon US website, I created a talk about how to do it. It is one of my goals to help other women be maintainers. I gave the talk called Get a Jump Start on Collaboration and Code Review GitHub for the first time at DjangoCon US 2017. A case study PinX. Pinx has been an opportunity to maintain a complex Django library, become a Python package release manager, and learn how to deal with Scope Prep. In the fall of 2017, based on my experience as a maintainer at DjangoCon US 2017, I was hired to work on Pin X. Pinax is an open source library of starter projects, reusable Django apps, and themes for building websites.

A little history about Pinax. When I was hired, it had been about 10 years since the idea of Pinax was born. Django had been open sourced in 2005 And in March 2008 at PyCon US in Chicago, some Django enthusiasts started working on Pin X. They were creating Pin X to solve their own problem. They found themselves reusing some of the same code patterns while creating websites with Django. So they started to abstract these patterns into reusable Django starter projects, apps, and themes. They wanted to create a reusable web development library that would make choices and compromises. This library would allow them to focus on the features at the top of the stack.

That way they could go from website idea to realization quickly rather than reinventing the wheel. So what was this data pinux when I was hired? It's quite common from the beginning until now for people to discover Pinax and say it's everything they ever dreamed of. On this slide is one of the very early tweets about Pinex. Someone says Pinex is every idea I've ever had By 2017, Pinx had grown to be a large group of professional quality, interdependent Django projects and apps. including starter projects with Pinx apps pre-installed and a Pinx CLI to install them. And sophisticated testing packaging and continuous integration configurations.

The Pinox GitHub organization alone has around 80 repos in it. This slide includes many of the more popular ones. But sustainability was lacking. Many of the original authors had moved on. Without a strategy to make Pinax easier to maintain, the maintainers began to suffer burnout. In addition to the GitHub organization, Pinx now had a global doc site and a Pin X slack channel for community and support. There were also app specific docs and individual repos. Now I'm going to talk about GitHub and the local development environment and how Pinax works. In your local development environment, if you are starting a traditional Django project You will need to have Django installed and you would normally be using a virtual environment.

Run the Django admin start project command in the terminal to start a project. The project contains the global settings and other configurations for your site. The project template files populated in the directory come from within the Django package. You would then run an additional command to add one or more apps to your project. Each app contains a special functionality, for example a blog app, for a blog part of your website. The app template files populated also come from within the Django package. With Pinx, you can install Pinx CLI, which is a command line interface. You can use commands to get info about Pinx, such as a list of starter projects or apps

Pinax starter projects are basically custom Django projects. They already have some functionality built in and by using them you do not have to start from scratch using a traditional Django project. The same with Pinax apps. You can also use Pinx CLI to install a Pinx Starter project instead of the traditional Django project. There's a special Pin X CLI command to do this. Pin X start Pinox starter project and your project name. When the Pinox CLI runs this command, the Pinox CLI uses the same start project command used to install a traditional Django project But Django calls it from the PinAx CLI codebase. The starter project URL is passed in as a parameter.

The URL leads to a projects. json file that provides the tar file address in the Pinx Starter Projects repo. Pinax starter projects are in one repo. Each starter project is in an individual branch. Each starter project contains the relevant Pinax apps to be installed from PyPI, the Python package index. Alternatively, Pinx apps can be used independently of any Pinx starter project. You can find them by searching Pinx on PyPI. A typical release, we update the test matrix, add new features, implement new best practices, fix deprecations, improve documentation.

A few important details about maintaining Pinax apps. We use GitHub Flow Workflow. For semantic versioning, we use Calveret release level, which is year. month We use SEMVR at application level, which is major. minor. patch. Each Pinx app codebase lives in a GitHub repo Each app repo has professional level configurations used to keep the code up to date. CircleCI is for continuous integration. Setup. py contains packaging configurations. TOX is for testing the Python Django versions matrix. When we do a release, we update the test matrix to support the latest versions of Python and Django.

In my release plan I document some test matrix boilerplate to copy and paste into the app files. The process is that I clone the Pinax app repo locally and create a new branch I update the test matrix in the branch. There is a tool called Py Imp that you could use to install multiple Python global versions locally. When TOX is run, it can use these global versions to check the compatibility of the Pinx app code against the new Python Django versions. When there is an incompatibility, talks will show you the error in red. When all of the tests pass, it will show green at the end. While you are troubleshooting to fix these incompatibilities, the Python Django release notes explain all of the changes made to the Python Django code

base in the release. You can use the release notes to understand what updates to make for compliance. When all of the test matrix and other updates have been made, You will want to update the SEMVR version in the setup. py, as well as the versions throughout the setup. py metadata and readme. Update the changelog in the README as well. There are some of the various, these are some of the various pieces of metadata. Then go to the tags area of the repo, draft a new release, include a link to the changelog. Run a process locally to package the app. I have documented this process in the. github repo in the release file And published the package to PyPI.

In July 2020, I oversaw the completion of a major Pinax release It included approximately 28 applications and notably drop support for Python 2. 7. It was a major milestone for me personally and professionally. I initiated the release, managed the process from start to finish. Created the release plan, supervised the work of others, updated 10 applications myself, merged all pull requests And tagged and published the packages. Some lessons learned from Pinx. I have tried to solve some of the critical problems of Pin X. The goal in general has been to implement changes to make it simpler and self-service.

As a result, newcomers, contributors, and maintainers could help themselves and therefore create a culture that could sustain itself. Some major improvements I've led. Documentation of tribal knowledge, consolidation of docks in one easy-to-find place. Standardization of configurations between projects, creation of detailed release and maintainer docs, creation of community health files including issue and pull request templates And a significant reduction in the number of issues and pull requests in order to get up to date with new issues and pull requests. One of the most important lessons I've learned from PINAX, sometimes it is counterintuitive. Sometimes it is important to stop and communicate your progress and plans to the community and ask for help.

For example, because of my blog posts, I received help that was crucial in completing the latest Pinox release Burnout. There is a real danger of burnout in open source. It is a sad subject. Code authors and maintainers are generous. Sometimes they work too hard And unfortunately, sometimes they have to deal with the negative attitudes of code users. I'm not sure Pinax is going to be healthy and thrive again, but I've learned a lot from Pinax. I want to talk about some ways to make a self-sustaining community for the maintainers, contributors, and users. Ways to reduce the scope, mark repos as deprecated, archive repos, disable issues.

Communicate that the code is maintained sporadically. You can implement additional automation to reduce the workload. Here are a few places where you can find automation tools. GitHub Actions, GitHub apps, Probot. Probot is a place to find novel ideas about what could be automated. Some Python-specific automation tools. In addition to PyEnv and Talks, there are coverage. py, iSort , Black, Restructured Text, and many, many others It is a problem when only one person has access or knowledge of a project, for example, repo access or PyPI access. If the person leaves the project and does not hand over the access and knowledge

It is difficult to continue. It is better, in my opinion, to have a policy of centralizing access and giving access to others Another flavor of this problem is when a maintainer creates a unique solution to a problem and is the only person with access and knowledge. If the person loses interest, it is difficult for others to continue. In my opinion, it is better to avoid very unique solutions. It's important to remember that it is the responsibility of the maintainer to secure the code. I say that because when there is a risk of burnout, it is possible to give additional limited access to contributors. You can give selective permissions, for example, only give access to an individual repo.

Protect branches, for example, no one can delete an important branch. require pull request reviews. For example, more than one person contributes to the review before merging. You can use status checks and in the worst case you can revert a pull request if needed. A few final thoughts. It's inevitable that you will have moments of frustration and get stuck. Here are a few places where you can find help. Google, Stack Overflow, GitHub Docs, Git docs, Atlassian, and GitLab docs. Brett Cannon, a Python Core Dev, has talked about the open source social contract.

Open source maintainers owe the community quite literally nothing. The PINX authors didn't have to open source their code and maintain it. They didn't have to answer the countless questions they answered. It is important to remember the generosity of open source code maintainers. There are also many perks of maintaining. It is an opportunity to learn technical skills, find new professional opportunities, attend conferences. Meet smart, interesting people building incredible apps, and travel to incredible places. At Pi Cascades 2020, for example, I had the opportunity to eat dinner with Guido The creator of Python Programming Language. He told us some stories about the early days of Python.

It was a magical evening. I directly attribute these kinds of experiences to open source. During GitHub Maintainer Week, I saw this tweet from Maffan. His contributions were critical in helping me maintain to complete the 20. 07 release. Seeing the difference my work has made to him and finding out that he considers me to be a very good friend is very special to me What he said immediately made it worth it. I suddenly realized that at the heart of open source are the relationships I form with people, which I'll have for the rest of my life. I want to give a special thank you to Ricardo Diaz Rincón for reviewing the Spanish version of this talk. If there is an error, it was made by me.

The Spanish version is scheduled to be broadcast at DjangoCon US 2021 on October 23rd. Feel free to contact me for more information. Thank you, everyone.