graphene-django

Welcome all. I hope you're enjoying your DjangoCon US 2021 so far. This is my fourth time attending Django Con and my third time having the opportunity to speak at it with you, so I'm really excited to be here once again. My name is Dane Hillard, and I've been working with Python and web applications professionally for nearly a decade. Before all that, my first web project was a CSS theme for my live journal. My Twitter handle is Easy as Python, which you can see in the bottom right of these slides throughout the talk. Please feel free to keep in touch with me there. So this talk is graphing Django or How I Learned to Stop Resting and Love the Graph. If you're unfamiliar, the title of this talk has been lifted from Doctor Strange Love or How I Learned to Stop Worrying and Love the Bomb.

This is a Kubrick film starring Peter Sellers and Peter Sellers and Peter Sellers to name a few. So where the film spoofs the Cold War zeitgeist, uh this talk's title kind of spoofs only for catchiness sake. This is not in all actuality an anti-rest talk, but it is a pro-critical thinking one and something to hopefully entice you to think about GraphQL. So though the Python community hasn't yet fully kind of stood behind or aligned on type hints for a variety of decent reasons. The intentions of type hints are generally pretty well understood, namely

safety and productivity. And Type systems can and do introduce safety and introspection that help us write more code correctly, more quickly. And types let us know what to expect as we author code. And with our tools building and reinforcing confidence in the types all the while. So we in the Django community leverage types in declarative programming fairly regularly. We can find it in the definitions of our class-based views, our models, our model forms, Django Rest framework serial. all that stuff. And type systems

coupled with declarative programming, coupled with static analysis, end up making for a pretty powerful tool belt And it can it can start to feel like code is writing itself. But type systems aren't a sort of a magic elixir either. And even full type agreement doesn't always equate to code correctness, right? So systems aim to reduce or eliminate classes of errors through abstraction, convention, and consistency. And the more we can kind of successfully compose these systems and integrate them into our into our work to eliminate classes of errors, the more we can focus on the actual business value, the core domain value that we're trying to deliver.

So Django's share of the stack is different for each of us here, I'm sure. But it's common for Django to collect data from a service further down in the back end, maybe from a third party. or to act as an API for a single page app maybe further in the front end. And you know maybe both. Uh in in these junctures um between these services uh or between these applications are kind of these infamous integration points, right? And they're frequent sources of bugs and outages. And These bugs fall into many classes, but type errors are one of the really common ones.

However challenging it is for types to agree within a single system, types agreeing across systems is that much more challenging. And so we found ways to reduce and eliminate classes of these errors through the data transport that we choose to use, the medium that we choose to use, like JSON. But this only works kind of at the primitive data type level, numbers and strings and lists and so on. And anything even more even mildly more complicated than that. doesn't really have a strong type representation in in JSON. So strings do not a type make, right?

Pets just have a string representation here, that's not a very rich representation. And if we decided we wanted to enhance that and include their species or their breed or something like that. that you have a a type hiding in there, but in JSON you can only use primitives to communicate that So even when we work hard to ensure consistency between a service and client, we also typically end up requiring some amount of duplication between those systems to ensure that both ends understand that common language between them. And if either one gets out of sync, it can spell disaster too. So we find ways to mitigate this through versioning and duplication

and backward compatible changes sometimes backward compatible changes anyway, uh just to really emphasize the very scary scare quotes there. So GraphQL uh is a is a technology and a practice that seeks to reduce classes of type-related errors. Both within and across systems through a type system with a published centralized type schema. So It also departs from REST in many ways. The first is that resources are defined by the type schema and located using a query.

rather than by the URI. So a resource doesn't have a unique URI in GraphQL. And introspection allows you to actually validate that a resource exists without executing any queries, which is kind of an interesting change from rest. And then Parameters are also part of the querying and they're not part of the URI either. And here introspection allows you to validate that those parameters are valid. Maybe not the value you specify, but the name of of a parameter without executing a query. And then data writing is done through mutations rather than HTTP verbs. And then maybe most interestingly to me

is that the query and the response share a shape. So you're in a way declaring the outcome that you want from the query, and the response will match that desired outcome. So Graphene uh is a is a Python implementation of GraphQL schemas and types. And Graphene Django integrates graphene further with Django by providing data model introspection and Ultimately resulting in a mostly declarative authoring experience that feels really similar to writing models and forms and class-based views and so on. So

This is probably best demonstrated by example a bit, and I really want to emphasize how fast you can get from zero to one with this. So the next few minutes we're going to go through some some code here and hopefully you're watching this talk in a way that you can pause the video. Ideally I'd like you to follow along and see how quickly you can you can get from zero to one like I said. So The first step, of course, is to create a Django project. So you would normally create a directory for your project. Let's call it Graph. And you would create a virtual environment using your favorite method. And within that environment, you'll install Django and Graphing Django. And then you'll use Django to start a project called Graph.

And then within that project you'll start an app called Library. And the library app is just sort of this books and authors model, sort of a timeless classic rite of relational databases. So you'll have an author class with maybe a family name and a given name, and then a book class, and a book has a title and a subtitle optionally. Then a reference to that book's author, uh, and then maybe the date that it was published

So then once you've actually I need to go forward one here. So once you've Created those models, you'll need to add the library app and the graphing Django app to your installed apps for the project. Then you can add this graphing setting that has a schema key that points to the dotted module the dotted name path and you'll create this shortly. Ultimately that's what defines the different types in your schema And once you have that app, library app installed, you'll also need to make the migrations for those data models and then migrate your database.

And then you need to have an endpoint that you can use to query the data in your graph. So you'll add a URL to your URL patterns for the project and it's common to put it at slash GraphQL. And you'll use this graphing Django view called GraphQL view. And you can see here that it's passing graphical equals true. And graphical is a user interface that can also be served at that same URL so that you can sort of manually develop some queries. Uh in a user interface in addition to sending queries to that endpoint.

Now here's the big part and I wanted to just give all the all the code at a high level here just to give you a sense of the the space but we're going to go through each of these things step by step as well And this is the part where you're really getting into graphing and setting up the GraphQL API around your data. So the first step is to import graphing and then from graphing Django you're going to import this Django object type And this is so so GraphQL is an amalgamation of different types as we've been talking about. And in graphing, you

map types to definitions of those types with the different fields they have and what types those fields are. So a Django object type is a helper class that allows you to create a type out of one of your existing Django data model objects And then of course you need to import the models that you're going to create these types from. So you import the models from the library app that you've created. So now you can actually get into creating those types. So you have some classes that inherit from Django object type. And then for sort of the simple case and and probably scaling up to a decent size project, you can probably get away with

Just having these types mapping directly to one of your Django objects. So you can do that with this meta class, and this is a pattern you might be familiar with if you've ever overridden things in the Django admin or Defined custom ordering for your Django models. So within the class meta, you specify that the model is models. author or models. book. So each of your models ends up having a type in the in the graphene or GraphQL schema And now for the bigger chunk, you have all these types now kind of floating around, but you need a way to bring them all together and make them queryable.

And so GraphQL and Graphing don't know how to query those out of the box. You have to tell them how to resolve a query to some data. And for this books authors library app, maybe we want a way to get a list of all the authors, a list of all the books. And then maybe a list of books by a given author by last name , by family name. So For the simple case where you're just listing all of some type, you can specify that authors is a graphene list of author type. And this might look familiar if you're keen on type hints.

So it's a list of a type Really what it's saying is that this will end up being a list of data that fits the author type schema. And same for books, it's a graphing list of book types. And then when you get to books by author, that's the more interesting one. So it's still a graphing list of book types. However, you want to be able to filter that down to some subset of all of the books. So here we're going to pass an additional field, additional argument called author family name. And that is a graphing string type. So that means that this query will expect

an additional argument and it will use that in some way. So now you've you've kind of defined the types that are available at the high level of this query. You have several lists of different types of objects Now you need a way to resolve those queries down to an actual Django query. So for each field you have here, authors, books, books by author, you're going to create a corresponding method called resolveAuthors, resolve books, resolve books by author. And again, for the the simple case where you just want a list of all the objects, you can just return all the objects.

So models, authors, objects, all. Um sorry, author should be singular there. Models author, objects all. So um Then again for the the more interesting one where you want to filter down books by a given name, you can see that the resolve method there takes an additional argument And all of these take this info argument, which is rarely used in my experience. It gives you some additional context about the GraphQL schema if you need it. And by specifying author family name as a graphene string

argument near the top of this class, you're telling the resolve method that it will also take that additional argument. So the resolver takes in an author family name, and then you can use that to filter down your query set. Models book objects filter, and this time we're going to filter it by the author's family name. to match the corresponding specified author family name. So this is the like a through filter for that foreign key Alright, so now we've got this query, we've got all these types, we've defined how they all fit together And that kind of represents our GraphQL schema and our graphing

schema. So at the very end, you can see that we need to kind of put that all together. Using this schema variable, and that's what you referenced in your settings file. So You put this all together with graphing dot schema and you specify the class that represents the root query, if you will, which is that query class you created. And this should all be in a schema. py module in the root of your your Django project. I failed to mention that at the beginning.

So, although Django Graphing supports write functionality as well. What you have here is is a read-only setup if you will and it is enough here that you've created already to have a fully functional read-only GraphQL endpoint. And at this point you can add authors and books using your typical favorite approach, whether in the admin or just adding some things manually in the shell. And you can once you add some books and some authors, you can start trying out the querying capabilities here. So hopefully that didn't seem like too much work. And it should seem like even less work if you remember that

the all those initial steps you did before creating the graphene stuff. is stuff you would have to do anyway for a typical Django project, right? Setting up your data model and your views and your URLs. So If you if you now run your application and you visit the GraphQL uh page in your browser you'll see that graphical interface. And I'm gonna switch over to that live here for just a second. And You can see that it has auto completion. I'm gonna create a query here.

And I want a list of all the books. And for each book, I want to see the title and I want to see the author. And for the author, I want to see just their family name. And if I run this query, you can see that I get a list of all the books. or a list of some books and I see for each book the title and then the author and in the author I only get the family name back So you can see that the shape of the query here matches the shape of the data that's returned And

you get auto completion for most everything. So you can see that books and books by author are valid top-level queries. And once I descend into that, if I start typing title, I can see there's a subtitle and an ID available as well. And if I want the author, it further. further completes what's available on the author. And what's great is that there's this docs tab on the right and you can sort of explore this entire tree yourself. So you can see that there's an authors which is a list of author type, there's a books, which is a list of book type, there's a books by author

which is a list of book type, and it requires an author family name. argument and if you click any of these types you can also see which fields are on those types. So a book type has an ID, subtitle, title, author, published date. So Graphing did all this for you. You declared some things about how these types fit together and it sort of introspected the rest from your data model. And I think that's really powerful and really interesting. And frankly, it's exciting to use this and to see. see how much

feature there really is to this as you're trying to develop queries and explore your data system. So um Like I said, I'll switch back to presentation mode here. Like I said, you have autocompletion, you have full documentation of your data types. and the query, you can see how the query shape matches the response shape. So all of this is is sort of really showing the power of GraphQL and the power of types in introspection. And there's also consumer implementations of GraphQL in JavaScript,

and especially in Reactive JavaScript frameworks like React or Vue. There's there's kind of this natural pairing now of consuming a GraphQL schema with one of these JavaScript frameworks and starts to get at building these really rich interactive web experiences. So uh GraphQL is really useful, pretty fun to use. It isn't all powerful though, and certainly not right out of the gate. So There are a number of things that won't sort of magically go away by using GraphQL over REST. So Type inference, although GraphQL's schema provides type safety and you can introspect and even um

inspect that that type system uh whether manually or in code neither the query nor the response really carries type information directly by default Right, the queries specify the fields, but they don't necessarily specify the type. You can include this type name field that gets generated. in your in your query and the response will contain the type but it's just the name of the type and there's still There's still a bit of wishy-washiness to that because you're just representing it as a string. And it also bloats the response.

If you have a big list of those objects, every single one will include the type name. makes your response bigger. There's also kind of this monolith, right? Um You have one place to go get your data from, which is your Django project's data model. So the querying and the resolvers and things we've seen so far are dependent on that single Django project And GraphQL actually doesn't have a built-in or out-of-the-box way to federate in data from other other sources. If another team is building, even if it's another Django project, there's not a built-in way to to combine the two It is possible though, and

with some elbow grease, you can get that done, or you can use some commercial projects like Apollo Federation out there. And these allow you to kind of graft your graph into another graph. Um and um sort of build out this larger graph of all your types across your teams, which can be really powerful. But it also you you also have some other dangers about efficient querying too. So there's still a danger of kind of the n plus one query problem. GraphQL doesn't inherently solve that. And there's still a danger of running into slow queries if you haven't indexed a column and and you're trying to get a lot of data or filter down data.

by a column value. So querying is still kind of on you. And in general for API design, um You know, naming is is kind of a classically difficult problem and with GraphQL's type schema all the names have to be unique within this global namespace of of all your types. So it makes it that much harder. And domain-driven design is kind of a challenge too because when you have this graph, all the types are introspectable, viewable. accessible and it sort of blurs the lines and the boundaries between contexts.

And then the other thing about API design is that Sort of a a common pitch about GraphQL and not necessarily by GraphQL or its creators, but definitely by people who use GraphQL, is that it somehow prevents overfetching of of data. So there you know with GraphQL you can grab only the fields that you really need. And this is often true. compared to REST, because in REST you're sort of beholden to whatever the API designer decided to return to you for a resource. And that's predefined, so you just get whatever they send. In GraphQL, you can specify just the fields you need. But you're still as

the team grows and as your system grows, you're still beholden to kind of the cow paths that the the team decides to follow, usually because of code reuse So there's this concept of GraphQL query fragments, and those intend to extract sort of common sets of fields that you might need in your response. and it's much easier to use an existing fragment than to write your own. So if you reuse a fragment and it asks for more fields than you actually need for what you're doing, you're still overfetching. So then just generally for performance, um You know, GraphQL is as performant as you make it. So it isn't uh

it also isn't as amenable to caching as REST because all those resources don't have a unique URI. So there are there are certainly ways to do it and there are standards within GraphQL for how to do some of these things, but it is a bit more work. uh and your experience is gonna vary from others depending on your needs and your cdn uh and so on So this is all to say if you have type error problems, GraphQL could really be for you, but like don't let it be a hammer looking for a nail So, uh, over the last few minutes I've kind of shared some of the positives as well as some of the negatives of GraphQL.

And hopefully I've shown you that the path to trying it out on a new Django project or even an existing one. is uh not so much effort using graphing Django. And I sort of encourage you to to try that out. Bring your data onto the graph. uh have a bit of a fun time exploring it and and sort of see where it takes you. Um There's with such little overhead, I guess, of of integrating it into a project, uh, feels like it's worth exploring to see how it suits your needs. So it was a pleasure speaking to you about sort of just one of the ways that Django fits into kind of the modern web development space. And again, I'm Dane Hillard.

I wrote practices of the Python Pro, writing publishing Python packages, and I hope you enjoy the rest of your time at DjangoCon US.