GraphQL in Python and Django

Speaker 1: Hello, can you hear me? Okay, it works. Hi everyone, my name is Patrick. I am chairperson of Python Italia and I work as a full stack developer at SyncSud, which is a creative agency in London, based in London. And you can find me at Sparta 91 online, uh more or less uh everywhere. Um being a full stack developer in an agency it's it's quite nice because you tend up to to do short projects so you easily can try new technology quite often. And today I'm gonna talk about a new technology that I've been using over the last year or so. And first let me explain why it's been created. So as you all know, the the old web was more or less like a collection of documents where the user could interact with the pages just by clicking links or using forms.

Speaker 1: And you will usually have just one single server that's gonna return all the pages. Uh the pages could be generated dynamically or not, but that's not the point. But as the web evolved in what we can call the modern web or the web two point oh, um the requirements have changed. We now have applications that run on mobile devices like smartwatches, phones, uh desktop devices like laptop or even fridges for example. Um and also we have more complex data structures so we have different data sources for example could be a Postgres DB, um Elasticsearch server and so on. And so we used to build um like gateways to to have to gather all this information into one single API that could can be used by all those team

Speaker 1: different devices. And we mainly use uh REST APIs uh which is uh it's not a standard so it's it's more like a design concept that basically says um Our API is a collection of resources and you can use HCP verbs to to do operations them. So you can get resources, delete them, edit and so on. Uh but REST is not perfect. Uh you might have seen the talk uh two days ago about REST. There are some some pitfalls that you have to consider when you're creating an uh an API with it. Um I'm gonna go through a couple of um U a c a couple of them. So first one is might be too many API calls. For example we have an API the

Speaker 1: um returns a user. So you do a get request to userslash ID, in this case slash one, and then you can get a response like list where you can say oh there is a name, a list of friends, an avatar. And this is quite nice because it's using the building blocks of the web, so it's usually reusing uh links so on. But It's nice but what if you need to get the the list of the friends and then their names for example? You need to do um an API call for each of them And one of the workarounds you can see is to create another endpoint where you can say I want the user with the list of friends expanded, for example. And so you get something like this. But what if you what if you need to um to get the list of friends and images. So you

Speaker 1: you might end up to create another endpoint. And same if you just need the user with their in their their avatar And what if you have to create a mobile application where you just just need to serve smaller images because serving like a big image is now a good user experience? And this goes on and on if you have to create other other pages and so on. And this is a huge problem if you have like a complex application or if you have many devices or if you have different pages For example, Coursera, which is now migrating to GraphQL, at some point they had more than 1,000 different points, which is a hell of a lot. You you can imagine trying to maintain this code base it's really hard, especially if you for example, if you hire a new developer

Speaker 1: you have to keep them up up to speed and takes a lot of time and also you have to maintain documentation because if you have to use this API on a mobile application you need to know all the different endpoints and the way you can fetch data So you might say, Oh yeah, I have a simple API so I just can return all the data. So you end up doing um requests are like returning too many info too much information and you can this is just simple example but for example if we have we merged the previous API to have something that returns everything by default, you basically having a bad user experience and also a developer experience because you're returning data that's not needed. um by your clients usually. Um and this is

Speaker 1: this is our waste of bandwidth you as well Um so can we do better? Would say yes, we can extend REST for example. Uh there were some ex there were some examples in the talk. A couple of days ago, for example, you can add uh um get parameters to say, Oh, I want to expand these fields, I want to show a different uh shape of the response based on the device. I mean But since it's not a standard it's everyone is gonna do it differently and you still have to document it, which is uh one of the pain points I think, at least for me. Uh especially if I have to work with other front end developers where I have to hand over the API. Um so GraphQL, GraphQL was created to resolve some of those issues and others. It's been created by Facebook around 2012

Speaker 1: and It's been released as open source in 2015 and it's been adopted by major companies like GitHub, Twitter, of course Facebook as well and Coursera as you can see before. So what is it? So first of all, even if there is graph in the name, it's not really about graph database, it's just a way to create the data. So the One of the main definitions is a query language for API, which basically means that you have um a language that you can uh request data. Um and I'm gonna show you some examples. But first um Uh let me tell you the uh GraphQL is a specification though so there is no default implementation. Uh the base implementation is using HTTP. But you can use any other protocol if you want.

Speaker 1: The HCP uh implementation usually is one single endpoint, which is usually slash GraphQL, and you do a post request where you can send your your queries. But what do you send to to to GraphQL? You basically send documents in this form which is more or less similar to a JSON. It's like a JSON without the the actual values too. Basically in this case you are saying oh return me the results for the user with their name, their email and their friend's name. And the API is gonna respond like this. Basically it's gonna return only the sh the data that you actually requested Um and something that's quite nice as well is the ability to have types, so everything in GraphQL is gonna have types, which is really important

Speaker 1: because For example, you can have a build step where you can say, oh, check this uh query. If the query is not valid, you're not gonna build the the API the application. It's just quite quite handy. And also it gives you allows you to create documentation without really bothering about it. You can have different types. You have integer floats, strings, Boolean and any other user defined type. which uh like the basic type is called scolar. And you can c for example in the Python uh library uh there is date time as uh uh type which is quite handy so you know that for example the type the date is always in that format And also you have object types, which is basically a collection of fields and types.

Speaker 1: For example, in our previous API we can have the type user, the object type user. That's got a string that's required, an email that's required, and then a list of friends that's required as well. And then there is another object type which is friend, which has got only a name. Um and yeah the power of having types uh plus the introspection which basically every GraphQL API by default has introspection enabled allows you to explore the API without having to dig into the code. For example, the there is an ID that's called graphical, which allows you to to test your queries in the browser. without having to open the documentation and as you can see there is auto-complete documentation and everything in just one tool, which is quite handy

Speaker 1: Um so so far we only have seen uh queries, so we can only can only fetch data. Uh if you want to do other uh operations, we we need to use uh one of those three up main operations. So Q that we've already seen, which is basically a way to request data to the server. Daniel mutation, which is a way to modify credit data. But it's not You can also, for example, use a mutation to create to to run a task or something else, which is kind of new. And then you have subscription, which is way similar to to queries, but it's in real time. It's usually using WebSocket. Um so this is uh like a shortcut for for a query. Since it's a common operation they they provided a

Speaker 1: a way to to to to do a query without having to type too much. The actual uh uh uh syntax is this one, see the extended syntax. So you have the operation name on top and then there is a query name which is only used for debugging. It's mainly used for debugging. For example if you need to have logs on the server you can just add a query name and then On the server you can find them by using this name. Then you can also have parameters. And then you have the list of fields that you wanna uh uh the U wanna query. It's quite interesting that you can have uh arguments for each field. So for example in this case we can limit the number of fans that we can fetch. But we can also for example if we have like different languages we can query by language or we can change the unit or measurement which is really helpful.

Speaker 1: Um then mutation it's pretty pretty much similar so you still have to query all the fields and then you have the uh name of the operation and the rest of the syntax is the same. And you can still like uh ask for the fields that you need to And then we have subscription, it's just the same. So we have Python conference, obviously we want to use this with Python. And we can. There is a library called GraphIn. And you can install it like this. Just install pip install graphene and then you can use it. You import it and then you create a club a class for the query. As I said everything is an ob uh it's typed in GraphQL, so even the root query is a type. So you have object type. In this case we have a query the

Speaker 1: only have has one field which is hello. And then for each field you have to create a resolver function or method. Um that basically it's is It's um it's the code that's returning the data. It's fetching and returning the data. So for example in this case we have a field that's called hello that's a string, and then the function is gonna return hello, uh hi DjangoCon. And then we get the schema passing in the query to the um schema constructor. And then we this case we are just executing the schema in here but we obviously can use uh a post request. Uh but yeah we have a Django con so can we use this with Django and can we use Uh we use our own models and forms with Django. Yes, we can the same people that built Graphene that made uh Graphene Django library

Speaker 1: which has got uh quite a few interesting tools. So you can install like that and then you have to install to add it to the installer apps mainly for the static files for the IDU. And then you need to to point out the the schema Then you need to add the path to the URL URLs and you can add the graphical view and you can also enable the ID direction we have seen before. And then you can create your object type and there is Django object type that's gonna create an object type based on a model, for example, and then you return the query set in your resolver. I'm gonna show you this in more detail in a second. There's also uh REST framework integration which is Uh quite handy if you already have uh REST framework serializer. It's similar to the to the jungle object type

Speaker 1: but only works for mutation for now. Uh this is something I would like to work on maybe in this print or in the next uh one um but yeah use it like this and then you can reuse your sterilizer if you have some custom logic there so instead of uh building the imitation by the f by hand So let's make a simple API. So we have a hotel model just with us with a name and then we have a post with title body and a foreign key to author. What we have to do first is to create the object types Which is quite simple with graph in jungle because you just need to create a couple of class and specify the the met the model in the meta. And that's it. So you already have um the object types. Then we have to create the query. So we create a simple class that extends object type and then we define the f

Speaker 1: post field which is a list of post object type. And then we create a resolver that's gonna return the uh post object uh all the objects in database. So let's quickly test this. Um oops So this is the idea shown you before and it's quite um it's quite easy to test this so we can do posts then we can fetch for for example for the title. I can do the request and I can see all the posts I have in the database. But for example if I need to get the author and the author name I can just ask for it and it's gonna return it without me having to do any any other thing just quietly. Oh no. Okay.

Speaker 1: So let's create the single post query as well. It's see pretty pretty much similar. The only difference that in this case we have an argument because every every field can have an argument and the argument is propagated to the resolver function I'm gonna show you maybe this later in a bit in the how it works in the IDE. But yeah, this is what you have to do for example to do a a simple query. Of course, you also want to allow the user to create POS, for example. And you do it using a mutation, you create a mutation extending graph in the mutation. You still have to define the the fields that this mutation can return. So you can allow the user, for example, to return the post title or other information. Then you have to define the arguments. Which are basically the argument of the mutation. You can consider

Speaker 1: mutation like it's a simple function. And then you have a mutated function that in this case it's gonna create another if it doesn't exist and then it's gonna return the post uh create post mutation with the post object that we had just created. And then you have to create a mutation object with all the mutation fields. In this case there is only create post and then you pass it to the the schema Done. Let's test this quite quickly. Um Um so as I said mutation needs you need to pass the uh operation name Then the field you can pass the title. Everything as I said is auto-completed so it's quite and because I I did this like

Speaker 1: in five minutes I don't really remember what I did. Um Then you can uh get the post, then you can get the title, then you can get the auto for example and the ID. So if I do this it's gonna create a new post and Oops. You can see in the list of the posts It's probably somewhere. Yeah, it's over there. She's quite handy. And as I said, we can also create posts by ID. So for example, I can create the post with ID one. And something I wanted to show you that I didn't really like at first is that uh graphene by default or even any GraphQL implementation in other languages, they catch the arrows for you.

Speaker 1: So if you have a server error, this the error is gonna come up in the uh in the front end which I'm not sure if it's nice but sometimes it's really helpful. For example if if I'm trying to do um Uh trying to fetch a post that doesn't exist, I'm gonna get an error. So you have a list of errors there and then you still have the data, but the post is none. So Uh something I like with this that if for example you can do multiple quiz or for example if you do first uh So if you do multiple queries in a single uh single request, you still can get the data that

Speaker 1: that uh came through. So for example you can see that there is a first is null because there was an error that doesn't exist and then you still have the second title which is quite handy if you have loads of data. For example in a project that we are doing a work we have We have different components in the page, like I don't know, 10 components and we batch all the queries so we don't we only do one query like every 100 milliseconds And if the anything fails it's not gonna break the app only the the part that failed it's not gonna show up, which is quite handy. Um yeah. Okay, this is cool at least um from my point of view. Uh but you still have to consider that this is a new technology, especially in the Python world.

Speaker 1: So the library is stable but uh there might be some some gotchas. So one of the Main things is the security. One of the um frequent asked question on the at least on GitHub is the authentication. How can I do an authentication with GraphQL? Well, we are using HTTP, so you can reuse it you reuse the HTTP blocks. So you can use for example jungle session, which is quite easy. So just say Well you don't really have to do anything with for using session. You can create a mutation to log in the user, you can use a form to log in them. And then you have to say to the front and not send the cookie as well uh when you do the request and that's really easy. Or you can use others so you can use JWT tokens or you can use

Speaker 1: basic hosts. And then you can also use parameters. For example if we if we in the In the create post mutation we could could add another field, for example, instead of having just create post with title and body, we can also pass a token, for example. We could authenticate the user using that parameter. This is handy if you have, for example, um just a few mutations that require user authentication. Um you don't really have you don't really want the user to to deal with adders and stuff. And then there is also permissions. There is no built-in feature in Graphene yet for permissions, so you cannot reuse Django permissions for now at least.

Speaker 1: Uh but one one of the interesting things of GraphQL is that you can have permission on single fields, for example, if I'm uh authenticated user, uh if I'm an an admin, I can for example return uh demail and not show the email to like normal users which is quite handy. But you have to do this manually because for now it's not bit there is no way to to do it like with decorators or any others specific sun syntax and also why you can have you can have public and private fields um um So for example GitHub is using this they have a single GraphQL API that's public and private at the same time, but the fields that are only available for internal development. Uh it's it's private. So it's not

Speaker 1: It's not showing up in any documentation, which is quite handy, which is also interesting for me. And then you can have malicious you have problems with malicious squeeze and caching. So my issue squeeze we are giving the the clients so much power because you can say, oh I want all these fields I can also nest all of them. For example you can nest it like this which is Might be a bit worrying if you for example do posts, then you get the author and then you have the post set, then you then you do something like this and you can go on and on. And it's not gonna complain, it's gonna do the query anyway. But yeah, you might not want this, especially if you have complex queries or you have someone that really wants to break your website so it they can nest it like one

Speaker 1: under leap levels deep. Um and caching as well. So one of uh the way that you can fix this um these issues. One could be to use timeouts. So for example if you have a request that's taking more than one second we can drop it. Uh this is something that Facebook is doing. They do if the request is taking more than one one second we just drop it because uh it's not a good user experience and it's probably someone doing something wrong or anyways. Um and you can also have a limit of the nesting so for example you can check the query so if the There is a field that's more than three levels deep, don't do the query, so just return an error, which is interesting as well. And then there is another one that GitHub is doing is called query costs. So you can calculate the cost of a query So you can give uh

Speaker 1: to each field a a coefficient. So you can say oh this field costs one, this field costs ten and so on. So you can calculate how much this query is costing to you and you can say oh I can only do queries that um more less than five hundred for example and also you can have static queries and this is gonna also help you with the caching. Sadic queries is a way to m to a queries that cannot be changed by by the user. So for example Example in imagine if you have a website that's only used by you, you can have a build step where you fetch all the queries that are done by the client and you can save them and then you can uh instead of g doing a post request to the server you can do a get request, say, oh I want to do um a query with this ID

Speaker 1: and then the back end is gonna get the query for you and is gonna return the data and the it's easily it's easy to cache because the query is not gonna change. Uh it's only gonna change if you do another deploy or so which is quite handy. And also if you only have static quiz um you won't have any problem with like nesting or a malicious query because the queries are limited to what you have. And I think this is something that Instagram is doing because I was checking their code and they basically they send uh like a request to graph to the Graph Kalum point passing um uh an ID, which is a long ID with the Q , QI ID and the variables as well. Some consideration um I've been using GraphQL I think more or less for a year in a couple of projects

Speaker 1: and we are using on other other projects that uh are still being built. And it's it's I think it's quite handy, especially if you have to work with uh many developers because um I I really love the fact that you have documentation built in. For example, I was working on an internal project that we have many different mutations. Every time I was uh finishing some mutation I was telling to the like front end person that was uh going to build the form or the like front end part, say, oh I've done this mutation and it was okay. And then it was checking the documentation by by itself without having asking me to or not to change something or how something worked, which is quite handy. Yeah, the I think uh it's quite nice.

Speaker 1: It's probably uh still hurling in the Python world because uh this is a technology that's mainly used uh by big companies where they use uh uh JavaScript based uh backend technology and so we still have to we probably we can improve the library quite a lot But yeah, I really would like to to see people using this and and improving the library. So if you have any question, uh you can feel feel free to ask me. Thank you

Speaker 2: Thank you, Patrick. You wanna take questions?

Speaker 1: Oh yeah, sure.

Speaker 2: Okay. Hi

Speaker 3: Patrick, thanks for your talk and I didn't knew GraphQL so is a nice introduction for me. And w um why do you prefer uh or I mean what's the Things that you prefer are GraphQL instead of REST API.

Speaker 1: Because for example I had some issues with uh one of the front end developers that they were complaining about the way I named some stuff in the endpoint. So for example I had uh quits slash ID slash answer slash ID and was with problematic well for example with GraphQL you can just create a mutation that's covers uh answered quiz for example it's quite handy. It's much much easier like the s the syntax and to it 's exchange information between like developers.

Speaker 2: Thank you. The question in the bank? Um hi, thank you. Um I have a question.

Speaker 4: Do you can you recommend any client libraries for JavaScript that uh help you using GraphQL?

Speaker 1: Oh definitely use A Apollo. Apollo is probably the best library of so far. There are two libraries, main mainly two libraries. There is Riga that's done by Facebook and then there is Apollo. Apollo is the community one I would say and it's really amazing. It's probably the best one.

Speaker 2: There's time for a few more questions.

Speaker 4: Going once, going twice. Thank you, Patrick.