Patterns for Extensibility

Speaker 1: Good morning. Thank you for coming to my talk. My name is Patrick Altman, VP of Engineering at Eldarion and core developer on the Pinax project. Well, there we go. Um So there's currently over 60 open source projects in the Pinx ecosystem. And this morning I'd like to discuss how we've added extensibility to many of them. And I cannot figure out how to make these advance. Guess it's gonna do it its own and so on. So why should you care about extensibility? Number one, it's the foundation of reusability. When we're writing Django

Speaker 1: apps, we want them to be reusable. Second, we have a chance to delight our users. We've all experienced this, I know at least I have. Django is a great example of this, whether it's extending the admin, using class-based views. Or custom use, you know, using a custom user model. And lastly, we have a chance to make the world a better place. I know it's a pretty grandiose statement, but really, if we reduce the need to reinvent the wheel, we increase the time and cognitive capacity that is available to spend working on new and interesting problems. The community benefits and becomes more productive and even more innovative, resulting in more value creation for and in the world around us.

Speaker 1: Just how do we go about creating extensibility? There's a lot of different ways, but I want to focus just on three today. The first is is what we call hook sets. And this is a term coined by Well. It's a term coined by Brian Rosner. He created uh hook sets in Django user accounts and was it was was driven by a a client need Where the use of signals didn't quite support the requirement, at least not as cleanly as we had liked. And it was simply uh it's simply a class with methods that can be overridden at the site level Then we have class-based views. Russell Keith McGee

Speaker 1: discussed this at length on Tuesday, and they're a great way to modify and extend behavior of an app inside a site. And lastly, plugins. Plugins permit loose coupling between the plugin and consumer. They allow for rapid expansion of functionality due to the ease of writing individual plugins. And they're used mostly to extend a single app or a website. They're less about reusability. So Django User Accounts is probably one of Eldarion's most used apps out of Pin X. It provides a ton of functionality that we've all come to expect from web applications, things like sign-up, login, email confirmation, password resetting, and more.

Speaker 1: But a lot of these rely heavily on sending emails. Up until hook sets, um, we were really only able to modify um sorry. Up until hook sets, we were really only able to modify the content of the email templates and not really change how the emails were being sent. The client in this case wanted to control the content of the email through a third-party website, third-party service that he had subscribed to, and it would require us to make API calls providing the context context for those emails. So the solution was to create a hookset method in um

Speaker 1: hookset methods to override how the emails are being sent so we could implement this into site. So let's take a look at some code if I can make these advanced. So the default hook set implementation that ships with Django user accounts basically just pulled out all of the uh blocks of sending email into individual methods in this in this object in this class. Then we have a hook proxy and we use this hook set object, the instance of the hook set proxy, to wrap the pulling of the settings. And that setting is what allows us to override. the default hook set so we can create a a hook set in our application and our website that allows us to override how the emails work.

Speaker 1: So here in our conf module, you can see we use Jana Slide L'sjango appconf for application settings. It's a fantastic way to encapsulate all the settings in your reusable app. Highly recommend it. So we ship with uh default hook set configured as you see here in this hook set property of the comp main conf module. But to override it, it's as simple as instant you know creating your own class at the uh and inheriting from the base hook set. and putting it in somewhere in your site and then setting the account hook set

Speaker 1: setting in your settings. py Here in the models of the Django of your Zengo Django user accounts, um, you can see in this sign up Uh class in the signup model we have a method called send, and there used to be a block of you know your typical send mail uh code here and that was replaced with this uh hook set uh method to send invitation email. And that's what keeps it that's what allows us to override how the email is sent. So for the client we ended up just making API calls and replacing the default hook set.

Speaker 1: So moving on to class-based views, we had a client with specific needs for our signup, for the signup process, as well as complexity of passwords. I really apologize by these slides, guys. Um Anyways, the uh he need the client needed support for profile creation and coupon codes right upon upon sign-up, and passwords had very specific business rules like containing certain characters and whatnot that out of the box are really supported by Django user accounts. So we had to extend the signup view and override some of the forms.

Speaker 1: To modify them to collect and process this extra data. But we didn't want to rewrite, have to rewrite the entire sign-up view. And the forms that dealt with the passwords, all we really needed to do was add a little bit of extra validation. Oh why keeps going back on me? So as you can see here in the view, this is the views module of the this project. We are pulling in the signup view from Django User Accounts. subclassing it and then overriding just a few methods, um two that deal with user creation and one um that's called It's an after-sign-up method that gives us access to the form. And

Speaker 1: we use that to basically take the created user, create a profile, and process any coupon codes. In addition, we're pulling in the custom signup form from that has the uh password, the secure password mix-in that we'll see here. And really all we needed to do to change these three forms that deal with that deal with uh passwords is to create a mix-in and to encapsulate in one place the business rules that we had that were around validating that the password was um the password had was up to his complexity requirements.

Speaker 1: and mix it in with the other three forms without really changing any of the the forms themselves. Except for the sign up form, obviously we had to add some additional fields there. Then we tie it all together in the URLs pie. We override routes provided by DU by Django user accounts for the uh change password and password reset views. We're using the stock views, but we're pulling, we're adding, we're overriding them here so we can use the cust the forms that we overrode. And then of course using our custom signup view. And now onto plugins.

Speaker 1: So Eldarin has a site that's really in early development, KPI Tree. It's a tool for teams to track metrics, scorecards, and key performance indicators. Core to this app is collecting metrics. Some of this is some of this will be manual and there's a lot of metrics. You know, some of this is manual like you know we'll come to the site and fill out a form or respond to an email, but there's a lot of metrics that are generated from systems, and so it makes sense to automate the collection of those Um geez. I'm about to throw my computer across the room.

Speaker 1: So yeah, so for the automated metrics, um we wanted a way to easily write

Speaker 2: The open source apps uh project.

Speaker 1: Jeez.

Speaker 2: Good morning.

Speaker 1: There's my rehearsal from last night. Funny enough, when you're rehearsing this, it doesn't automatically advance like this. Okay. I believe I was talking about the automation. So we needed a way to easily write integrations for the automated collections. We'll likely end up writing the bulk of these, but we wanted to expose the ability for anyone to contribute and add to them. We're going to open source this package

Speaker 1: so people can submit their own. And you know, to encourage this, we need it to be dead simple. This is nothing new. Sites like Git Century offer this. GitHub, you can send pull requests to extend a plugin. Um and the idea is you should only have to implement a single class to conform and conform to an API and not have to worry about any of the internals of the host system. Um so there's really just two parts here that are that make this plugin system work. Um the first is uh Uh the first is the registry, which is really just a um a small wrapper around a dictionary.

Speaker 1: And you can see here we create a an instance and then delete it to create a singleton. And the important point of ref in uh the point the important point here is um this registry object is a single point of interface between the the internals of the website and the plugin system. The second half of this base dot pie module that's implementing this whole plugin system are is a base plugin class. In some languages it would be an abstract base class, but important to note here, you can see it has this thing called a meta class , which is this the the registrable class at the top. And the really cool thing about this is

Speaker 1: any definition of a any def definition of a new plugin Will automatically register itself. It's one less thing to uh worry about in configuration on our side or for the plugin developer to have to get right. Um so here's an example of a plug-in that I wrote for um for our use. Um we use it to integrate with user voice specifically to pull in support metrics from Gondor. So this is about all there is to implementing a plugin. Obviously I've cut out the implementation details because they're not important and the slide's only so big. The idea is you just implement the these methods to interact with, in this case, user voice API and not have to worry about the internals of how it's consumed.

Speaker 1: So here's an example of how it's being consumed internally. This is a models module of one of our internal apps in the project. The auto collection is um simply a model with that the user will create an instance of And as such, we'll select a provider, in this case, like user voice. The choices are populated from the registry. There's a method on there that will group all the choices together. And the inputs The user input that the plugin needs to execute, whether it's OAuth tokens or or whatever, and the plugin developer is responsible for telling it what it needs.

Speaker 1: And this it will actually validate against the plug-ins inputs method to make sure that all the required inputs are there. And then And I cut a lot of code out here for brevity, but in the process method, we're going to call and fetch the data and then create metrics internally. And the beautiful thing about this is The plugin developer doesn't have to know how this is being consumed and KPI tree doesn't have to know what plugin it's talking about. It's a very clean separation. And earlier I mentioned when about the metaclass and how that works and uh things get registered automatically when things are defined and imported.

Speaker 1: Well To import them , we have we use a startup. uh pie module in all of our projects that's going to quickly change now with Django 17. But for the past two years we've wanted to organize all of our startup code in one place. And not have to do things like import things into URLs Pi relying on the fact that Django autoloads that. So and All of our projects we have at least this run method and the auto load method to run the auto-discover for the admin and to load all of our receivers modules where we store things like signal handlers. Instead of putting signal handlers inside models, we put them in our receivers pie

Speaker 1: to keep that all organized well. So for the KPI tree project, we added this function called load submodules And all it's going to do is loop over this KPI tree integrations package and import all of its modules. In our case, they are plugins. And that's what, and then when that happens, they get auto-registered. So in order for this run method to execute, we have to modify the two entry points to our application. And that's the WISGI module. You can see here we m import KPI tree startup and call startup run, we do the same thing in the manage. py. So as of Tuesday, we've had over 5,000 stargazers on across

Speaker 1: those 60 uh Pinx projects. And for the ones that are packaged, for example, we have startup project, starter project templates, but for the ones that are packaged, over a million, close to a million downloads. And I say this because you know with hook sets, class-based views, and plugins , it's helped Eldarian make apps more extensible. And in doing so, we've seen greater re- re-reuse. And as evidenced by these stars and downloads, I like to think that we're delighting our users, at least some of them. And after three plus years of working at Eldarion, I can honestly attest that our immediate world has gotten better, both for us and our clients, because

Speaker 1: I've seen that we are building more things and we're building them faster, lots faster than where we were three years ago , both for ourselves and for our clients. So I've covered a lot here really quick. Um this wasn't really designed to be a tutorial, but just to kind of get things get you thinking. I'm around the rest of the afternoon. If you want to talk any more, talk any deeper about any of these things, just find me and let's Let's connect. So with that, are there any questions? Thank you.

Speaker 1: Okay.