Mighty Model Managers by Shawn Inman

Speaker 1: Come on, no, yeah, yeah, yeah.

Speaker 2: Alright, so thanks everybody for coming out. I really appreciate it. Um this talk is called Mighty Model Managers. As he said. Um, so first of all, I have to give you all props. It's pretty full in here. It's uh 440 and it's beautiful outside. And you're here to talk with me about uh part of Django Core that's been around for years. So you are the true believers, so give yourselves a hand. Um So we're going to be talking about model managers, specifically creating model manager subclasses and how that makes our lives much easier. And it really does, so get excited. So first off, why are we here? Not like in an existential sense. I mean like why are we here as makers of software? What are our jobs? And for me, I think that really making software is primarily about the process of managing complexity.

Speaker 2: And so depending on the case, you know, sometimes we're managing complexity on behalf of our users or the complexity in a business process. But a lot of times we've got to take a step back and make sure we're doing it right and that we're managing complexity for ourselves in the applications that we make. And we can probably all agree on that, at least to some extent So why listen to me talk about managing complexity? What do I know, right? Well, this is my family. So my wife and I have four amazing kids. Uh they're ages six and under. I love it and I love them very much, but I learned a lot about managing complexity along the way. I uh also I I work for a uh a company called eFellow. We work in healthcare, which has also like got its own complexity, but that's not really as cute as a picture of a bunch of kids, right? So

Speaker 2: oh so why do I want to talk about model managers? Well the the thing is that they're part of the core framework. The documentation is spot on. They're really useful and I feel like they're overlooked in the community. I have A bunch of friends and you know we talk and I feel like it solves a number of problems but people really aren't looking at them. I think it's one of those things you just get it working and sometimes you step away. Okay, so um my goal is that we can talk about this stuff today and you'll be inclined to take at least one thing away from you to implement in one of your applications. We're gonna get there, I promise So there are a ton of benefits to uh to using managers. There improves semantics, uh, readability, encapsulation of logic, ease of maintenance. There's a lot more, but this is a short talk, so we'll keep it quick.

Speaker 2: But um so a show of hands, how many here in attendance use model managers in your code today? Okay, so a whole bunch of these. That's great. So for anyone who didn't raise your hand, there's a little bit of a trick question, so sorry at your expense, but if you've used the R ORM at all, you've used model managers in one way or another. So this simple line right here, animal. objects. all. This call to objects. Objects is the default model manager that Django gives you right out of the box So it's right here is providing us a query setback and we're saying give me all. So I'm getting all instances of my animal model. Pretty simple. If you want to see it a little more explicitly, you can actually code it right out. This is what's happening for you behind the scenes. So when you define animal, you could put objects equals model

Speaker 2: stu manager. You wouldn't accomplish anything new, but it would be really explicit So what's the job of a manager? Is that the only job is to give us a query set? Not really. So the first job of a model manager is to control the query set that you get back when you want to inspect it. But Aside from that, the second job is to provide table level functionality for the model. So if you think of it this way, if you wanted to implement functionality on an instance of a model, you'd create a method and you'd put it on that model. If you wanted to provide table level functionality or something that operates against all the instances of a model, you would create it on that model 's manager instead So, as is usually the case, uh if you start tweaking things a little bit, you can get some really interesting results

Speaker 2: So to demonstrate that, let's talk a little bit about a sample project. So we get together with our team and we decide we need an application for keeping track of the people within our organization It's really simple. It only needs to remember two things about a person, their name and their role within the company. So you say, no big deal. You're either an author or an editor. Sorry. So you say no big deal. You head back, you code up this model right here. Everything is good to go. You toss it into the admin and now we can we can start tracking these things. To query that model real simple like we talked about, just person. objects. alf and then you get your results set But the semantics of this are a little weird. We were all used to seeing you know model. objects.

Speaker 2: all all the time. It's commonplace to us, but people aren't objects, people are people. So if we want to make that slightly more semantic, it's a really simple change. All you have to do is as we said before you can still use the default manager that comes right out of the box with Django. Just rename it. So from now on, if I want to query in a more semantic way, if taking my list of person objects, I say person. people. all. Again, same result, just a little bit more semantic along the way. This is a small change, but honestly readability is important. And I think this community gets that in general. We have a lot of emphasis on code quality, you know, with Pep 8 and a number of other things. You don't have to take my word for it. Ask Trey. Trey did a whole talk about this yesterday. I'm sure he'll be happy to tell you. Maybe sir.

Speaker 2: So another thing you may be thinking like is this does this really matter? And it does, and I think that I can convince you you probably wouldn't leave your model looking like this in the Django admin. Persons is not a great way of representing my instances of person. You would go to your model, you would add the verbose name for it, and all would be right. You do it that way. The thing is, just because the end user isn't seeing the change or the way that we query our models doesn't mean that that's not important. So we talked a lot a lot a lot about semantics and some fuzzy stuff and and not everybody may be convinced that it's worth it, but there's some much more concrete benefits to be had. The encapsulation of business logic is really important, especially as your application starts. to grow. So say

Speaker 2: someone comes to you and they ask, I need to get a list of all of our editors. How can I do that? So you say, well go ahead, person dot people dot filter roll equals e. There's all of our editors. No problem. The issue is that that query most likely is going to be replicated throughout your view code all over the application. If you need to create a report of all the editors, you're going to write that query If you need to send a mail message to all of your editors, you're probably gonna write that query. It's gonna be sprinkled throughout. And it it's it works, but you kinda wonder like, is this the right way to do it? Bernie does not think it's the right way to do it. So we can do a lot better than that. So what we'll do is we'll set up a manager subclass specif specifically for editors.

Speaker 2: So there's there's two quick steps to that. First we'll subclass manager and only thing you need to do is provide a get query set method. So in here you can see we're just we're putting the filter right inside the manager so this is all wrapped up Then we just have to reference that manager in our model. So we can leave the existing model and the existing manager intact and we'll set up a reference called editors to our new our new model or new manager So now the query process is a little bit different, but it's really clean. Again, readability. So person. editors. all is pretty pretty slick. You know, it's very obvious to anyone who's writing the code, new members to the team, everyone's gonna understand exactly what's going on with that So what if we wanted to say I needed to replicate that process for authors? You know, the editor is not the only person that I care about.

Speaker 2: That's really straightforward. So I'll create another manager. I'll call it author author manager. Reference that in my model just like this, and it can live right alongside the other two. There's really no limit to the amount of manager. that you can have. Again, same process to query, person. authors. all. So this is good. This is familiar. Everyone's going to get this who's using your application. There's some drawbacks though to this approach. So because we are getting a query set back, when we say person. editors. all, you're getting a raw query, a Django query set, the base query set set. So there's no customer operations. You don't have any way to inject custom code there. The other thing is we're creating a subclass every time we want to have one of these little handy shortcuts and that's going to start feeling pretty heavy after for a while, especially if you need to uh do an intermediary step for all of those.

Speaker 2: So it'll work for now, but there's probably a way that we can make it this better there is so what we can do is we can instead subclass query sec so now I have two parts of this I have a person query set and I'm just going to expose two methods, authors and editors, and that's where I'm going to do my filter. The query set is filtering itself on the roll. I'm going to create a new manager called Person Manager and I'm just going to set up pass-through methods to those two. I'm going to have authors and editors and all it's doing is calling It's calling get getQuery set. authors or dot editors. I can throw away my other editors now. This is going to replace them. So our our model now It's gonna look like this. We're gonna have an our person. There's only one manager, it's called people, and it points directly at the person

Speaker 2: manager So now this is what my query interface looks like. A little it's slightly different, but it's better. It's getting better. So I have the ability to say person. people. editors. I get my same results as before. Person. people. authors, same results as before. And since now I've I've uh subclass query set, my custom operations are now here so I can chain these together. I couldn't do that before So I could say person, not people, not authors, editors, which that doesn't make sense yet because you can only be one or the other, but we'll show how that comes into play in just a few So now the team gets back together and they say the app is great, but we need to be able to track whether a person is full-time or part-time. No worries.

Speaker 2: I'm just gonna update the model. So we add that employee type attribute. time or part time. No worries. Now we'll head back to our query set. We'll add some new methods for full timers and part-timers there, just doing the filter, again hiding that away. And then We'll next go over to our manager and add those supporting methods there as well. So now when I query against this model I have person. people that's full timers and I'll get my full-time employees just like you would expect. But also I can change. So I can say full timers. authors to get my full-time authors or part-time or set authors to get my part-time authors.

Speaker 2: But as I was going through this, I started learning about this, I thought there's a lot of repeated code right we're just this essentially our our manager is acting as a pass-through there's got to be a better way to do this so as of Django 17 some of you may be familiar there's act there actually is a better way to do it What we can do is we can ask Django to extract the manager straight from a query set that we provide, and it'll do it based on whatever methods that the query set has on it. So what we do to make that work is we'll say people equals personquery set dot as manager and now we can delete the manager entirely. If you're not doing anything else, just get rid of it. So now we're only gonna have a query set with the methods that we're looking at.

Speaker 2: Nobody talks about strong bad anymore. I love strong bad. This is still so good. So again healthcare healthcare is uh it can be really really tricky there's a lot of things that I ended up doing uh to support our application that you know we had to add things like soft deletes or um you know different status fields or things that you want to filter away. I feel like this comes up all the time So managers are a terrific option at preventing mistakes that happen in your code base. And I I like to joke about this as a way of preventing or protecting future you from present you because half the time it's my own code that I wrote that I forget to something. So the team gets back together and they say deleting uh

Speaker 2: deleting the person instances from a database is just is way too heavy. We need to enable a soft delete and you say yeah I I got this. So you you add an is active field to your model uh just a boolean field but it's so easy to forget the filter on that every time that you run a query you have to remember to add is active to to the chain of of filters or you're gonna end up with bad data or leaky data. So anybody here ever done something like that before? I do that all the time. And I my background, I used to be in banking and now I'm in healthcare. It's like two really bad places to make mistakes like that. So uh anyway, so depending on your industry too, it can be harmless inconvenience or it can be a major compliance issue. depending on what the situation is.

Speaker 2: So one way we can do this is we can leverage all this encapsulation that we we have built into our application and really use it to our advantage. So I'm gonna add a private method here called active. I'm gonna assume that someone who's querying against my model doesn't need to use this active function um this active method explicitly. They're just gonna they're gonna get active records by using my other methods that I exposed. So all I can do or all I need to do is internally within my full timers, part-timers, etc. I'm just gonna chain in the call to active right in there. So now by default, when I uh query against this model, I'm only going to get active users. Again, no one had to do anything different. I didn't have to go rewrite all of my queries against the person model to add this active

Speaker 2: uh to chain it in there, I could just do it behind the scenes, which is phenomenal. You can do this with things like start dates for you know employee records or there's there's a million different applications for it. But really anywhere that you need to hide something that you you know you can you can sort of encapsulate that scope creep over time without causing pain. So um so yeah we're gonna include only active records by default There's only one caveat to this approach here. So you may notice we're not overriding the default query set that we're returning So if we were to run a dot all, so person. people. all, we would get inactive records. And that's by design here. You don't have to do it this way, but there's one thing that you have to really keep uh keep in mind is that when you assume control of

Speaker 2: the the the fault manager for a model, what you're doing is you're saying I want to control the instance or control the query set that gets ris returned from my model under all instances Most of the time, you're just referencing that from within your application, and that's exactly what you want. But it also counts for things like if you're running a dump data, for example In a dump data scenario scenario, you may want everything. So you every use case is going to be slightly different. So your mileage mileage may vary. It's just going to depend on how you want to use it. The other thing to note is that there are specific methods that you can call or that specific attributes that you can set on a method within a query set so that when you call as manager, they don't necessarily come over by default into the the manager. So

Speaker 2: for instance, no private methods come over by default when you call as manager. And there's also uh there's another attribute called query set only. And if you set that it's It's pretty obvious to name, but query set only will prevent those methods from coming into the manager. So if you need to hide something, you can certainly do that. Test coverage. Um so testing is is outside the scope of this talk, but it's pretty easy to see how this is a big win for uh for test coverage in general from within our applications. So uh really what boils down to you only have to test the query once inside the manager you don't need to test the query at 40 different touch points from within your application so it's a big win there

Speaker 2: Um one thing also I want to note, there's another talk that's coming up directly after this, and it's actually in this room. Uh it's called I Didn't Know Query Sets Could Do That, and it's actually a much deeper dive, but I paired back the scope a little bit of this talk so that I didn't step on that talk, but he's going to really dive deep into using F objects and Q objects and how you can write raw SQL queries. and do really complex things with query sets. It pairs really well with this because I know what you're going to do is you're going to do all those complex things and then you're going to encapsulate them inside of the custom method so that you save everyone else the pain. So hang out here. Also, I want to give a quick shout out. This talk was really heavily inspired by the documentation for model managers that it that's out there. Anyone who's contributed to the docs, they're doing amazing work.

Speaker 2: When I was first getting into Python five years ago, the real reason that I was drawn towards the Django framework was just the quality and the the community support around documentation. I think honestly that it's unparalleled. And so like give those folks a hand

Speaker 3: Hi, thank you very much. Um The the the bit at the end with the dot all uh got me thinking, could you could you put could you override the dot all method on your query set manager or query set Git query sets and my managers mixed up. Yeah. And put a keyword argument in it that defaults to say true. So I'm thinking uh you know your is active equals true and then use that in a filter underneath. Would there be any gotchas or scary side effects of overriding. all

Speaker 2: It's a good question. Um I I can't say it with an authority, but I do know that what you'd want to do is override the init of the query set in order to to get what you're looking for. And so you can't there obviously it's Python object so you can pass. I don't see why that wouldn't work, but I was doing some experimenting with overriding it for this talk, but it just got kind of hairy and I wasn't sure if there'd be beginners in it. I don't want to get crazy, but I do believe that yeah you could pass a card to it if you wanted to. Only thing is is that it's a little um little at odds with It would just have to be a really known quantity, I guess, for anyone who's gonna query against your model. The whole goal with this is to make it, you know, really straightforward.

Speaker 3: Yeah, I I was just thinking of the use case in which 95% of the time you wanted active things and you could count on someone to specify, I want an inactive record one. that came up.

Speaker 2: Yeah, yeah, you certainly could. The other thing that I've seen too is that you can actually make so I I made active a uh private method. here. But I've seen it as a public method. And then you also expose a public method that really includes everything, but you make it some obnoxiously long name. So if someone's doing it they're totally opting in that I want I want everything including bad records and deleted records and everything so there's a lot of different ways to approach it. I think it's a such a generic concept. It could go either way. Thank you. Thank you.

Speaker 4: Yeah, hi. Have you ever uh tried using model managers to do other things besides filtering? Uh I'm thinking just as examples like uh logging or initiating hooks Or putting things in caches or anything like that?

Speaker 2: Yeah, absolutely. That was actually one thing that um I was thinking about talking that I didn't cover, which is it's a really great way of caching because you're um Because we're sort of doing this composable idea of chaining together all these filters, you can do really it's almost like the Russian doll caching uh idea where you can cache really small bits of queries and bring them all together. So absolutely it's a really good way of doing that. We experimented with this within the healthcare thing about using this as a way of logging access to certain models It's a little bit hard to wedge the user context into a query, but absolutely you can do a lot of things that are not necessarily query related in there because it's just a method. So certainly. Mm-hmm.

Speaker 5: Um so correct me if I'm wrong here, but overriding the objects property on a model. um to say like persons equals default manager. That still leaves the objects um as an accessible property on that model, correct?

Speaker 2: It does not. So when you uh what happens is Django will look for the first instance of a manager that it encounters. and that becomes default. You could still leave. So if you're worried about backwards compatibility, you could have two references to the same manager, one called objects, one called people, and they could live next to each other. But dot objects would not be available.

Speaker 5: Okay, so you're saying I have to explicitly call that out?

Speaker 2: Yes.

Speaker 5: Okay.

Speaker 2: Yeah. Django won't automatically give you it if it finds the yours there. first.

Speaker 5: Okay, thank you.

Speaker 2: Sure.

Speaker 6: My question kind of piggybacks off of that. If how does that play with uh if you use people instead of objects, does that play well with Django Rest framework?

Speaker 2: I you know I couldn't I couldn't answer that. I know uh I would guess yes because the what the the ORM does is it does a dis is a kind of what I was just saying, it does discover what instances of a manager class or subclass are present in your model. So it's probably leaning on the ORM to do that, but I honestly I would have to test that. That's a good question though.

Speaker 7: Um I just really liked how your your names worked out and those filter methods and stuff. And yeah, uh anyway, I was curious, uh did you have other clever um ways of naming things with uh when you're like doing things where you're adding additional select related um to to something. Uh like so uh K yeah you know a really common example would be listing comments on a page and you want to do some user profile information with it. each comment you'd do a select related and it'd be a common thing you'd want with a lot of queries.

Speaker 2: Sure. So you're saying uh sort of query the same data two different ways just with two different managers. Like one would include additional data, one would not.

Speaker 7: Correct. Yeah. So you

Speaker 2: could certainly do that. Yeah. I mean so we sh as I mentioned you could have really an infinite number of managers living alongside of each other. So um you know if you were clever about it you could probably even uh like leverage a lot of one base query set and subclass you know subclass your subclass to include the additional information in there as well so yeah you could definitely

Speaker 7: Okay, yeah. So I was trying to make it readable.

Speaker 2: I have not personally done that.

Speaker 7: Okay. Uh

Speaker 4: round of applause for Sean Edmund.