Auto-generating an API using PostgreSQL... by Mjumbe Poe

Yeah, I think that's a good thing.

Speaker 1: Hi. Um So thank you for being here, first of all. Every time I stand up in front of a a large group of people, it is nerve-wracking. But After it's over, I enjoyed it and I feel like it it usually goes pretty well. So I'm hoping that we can all have a little bit of fun while I'm up here. Today I'm going to be talking about auto-generating APIs from an existing Postgres database using tools like Django REST Framework and Django filters. So in this talk, I wanna I wanna try to dive into some of the specifics of uh how

Speaker 1: we go about doing this, uh talk about some of the situations in which it makes sense. and some situations in which it doesn't make sense and uh try to give my sense of whether or not this is something that uh I recommend. um for other people to do. Spoiler. It is often not a thing that I would recommend for other people to do. But it's still a really cool, a really neat idea. And something that I would recommend people try, even if it's not something that you would need for your production application. Uh so um yeah.

Speaker 1: So I realized earlier today That I didn't actually talk about what I mean by generating an API. And so I I added these slides uh a little earlier this afternoon. Um and so at a high level What I mean by auto-generating an API. You can think of an API as consisting of very high-level three components. You have your database, your data store, wherever the stuff that you are serving out is stored, or the stuff that you are taking in gets stored. You have your views or your external interface to your API. So in Django, this is views. And then you have your models,

Speaker 1: which serve as your interface, your internal interface between your external interface and your data. So there are a few ways that people generally go about uh constructing applications in general and uh specifically APIs. A very Django way of doing things is to write your models and your views explicitly. So actually code up a models. py file. code up a views. py file or whatever you want to call these things and then use the rules that are built into Django to inspect the structure of your models. py file to generate a structure for your database That's one way of going about things. Another way that some people go about things for

Speaker 1: APIs is they already have a database full of a bunch of data and then they build models. py files to act as the interface on top of that data between the database and their views. What I'm talking about when I say generating, uh auto-generating an API is more something like this, where you start with a database full of data. You use a set of rules to inspect that database and build models which you then use To build views, um, well, build an external interface for your API.

Speaker 1: So you don't explicitly code your models, you don't explicitly code your views. You just build rules to take the data in your database and construct models and views on top of that data. Very high level. All right. So that's the that's the what about um this this endeavor. Um so now I just want to talk a little bit about why uh why why why we decided to to build an IPI. And actually let me back up and tell you a little bit about who we is and who I am for that matter. So I I'm a I'm the CTO of a company called Stepwise Analytics. We uh build tools for uh helping people making

Speaker 1: uh helping people to make smarter and more impactful real estate investment decisions. Um so a lot of the data that we use For these tools comes from open data sources, data that's published by governments from city level up to the federal level. So we work with a lot of open data. And when I say a lot of data, I'm not like talking about, you know, for example, we're not We're not uh working with human genome level uh big data kind of data. But uh we are working with data on the on the level of uh for each of the cities that we support. Um uh a few dozen gigabytes of data for each of those cities.

Speaker 1: Um so it's a significant amount of data uh and it's split up into a significant amount of data sources. So a little bit about my background with these kinds of data sources. So before I was at Stepwise, I used to work with the city of Philadelphia in the uh open data and digital transformation uh office, the Office of Open Data and Digital Transformation. Before that I was at Open Plans , a company up in New York where We made tools for helping to helping citizens to get involved in the urban planning process. And before that, I was a Code for America fellow. So I've been working around government open data sources and in the space of government open data for

Speaker 1: several years. And in that time, I've learned a few things about these kinds of data sources. So in my experience, government data sources Can be scattered. So they are there are a number of different places that you might look and find data sources that are related but not directly connected to each other. So they require a lot of cross-referencing and cleaning up in order to do that cross-referencing. Uh and you know there are a number of a number of them across different departments. So um, you know, the government of a city as a whole is generally not the one that's producing

Speaker 1: uh data sets for that city. It's usually um, you know, licenses and inspection in the city and the Department of Records and you know, all of these other independent entities that don't necessarily communicate about uh what the single single way Uh single format around that data, single uh schema around different things that should be included, and so on. Um so these data sets can be sat scattered. Uh moreover, uh you can Oftentimes you'll find that uh data sets uh pop up and disappear without warning. And more often, within any particular dataset, you'll find that new fields appear or disappear without warning.

Speaker 1: And these are all things that we just have to deal with in dealing with open data. So we stepwise had to come up with some place to uh ensure that the data that we were using and serving up to our application through our API. met certain expectations. And so we we built an entire data pipeline around ingesting all of these different data sources and built in certain expectations to that data pipeline. And what we initially did was in addition to building in expectations in the pipeline, we also created models by hand for each one of these data sources.

Speaker 1: That we were pulling in from all these different cities. What we found is that because we were repeating ourselves, we were building in expectations about what the data was going to be structured like at the data pipeline level, and we were building in expectations about what the data was going to be structured like. structured like at the application level, those things often got out of sync. And so in order to stop repeating ourselves, we said, well , why don't we try instead auto-generating our API based on the data uh that we're getting in. Since we're already checking that the structure of this data meets certain expectations, though we we wanted to be flexible enough around uh where it didn't need to meet hard expectations. Um But

Speaker 1: as long as it met certain expectations, we wanted our API to run with it. So uh we decided to start auto-generating our API. So this is all of those reasons that I just laid out. So there can be a lot of different tables coming from a number of different stores sources. Those the schema of those tables may change. on a regular basis. And our table structure was already defined elsewhere specifically in our data pipeline, so we didn't want to repeat ourselves by defining it as well in the application. So that's the um the what and the why around our approach to generating an API.

Speaker 1: Now we're going to talk about the how. So at a very high level, very high level, these are the steps that we take. So first we inspect the database. Uh depend uh based on that structure we create models. Based on those models we create serializers and filter sets. Serializers are a Django Rest framework construct and filter sets are uh in Django filters which works with Django REST framework. And then based on those or using those as well as a few other things. we create and expose views. So this seems pretty simple and like all things simple, it's actually not. But at a high-level view This is this is the this is the general process. And I'm gonna dive uh into each one of these

Speaker 1: um each one of these points. On their own. So we start with inspecting the database. And this is this is really just figuring out what is in your database. And so the way that we do this is with something called introspection. Introspection is actually a term that comes from psychology, I believe, and it's it's the process of asking oneself about oneself. How am I feeling? uh what am I thinking? Um so much like introspection in psychology is asking oneself about oneself, uh introspection in uh in databases or in programming languages is about asking that database

Speaker 1: about itself. And so there's a there's a tool that we use uh to do this that's called the information schema. So the information schema is uh an ANSI standard Um and as such it's supported among um most of the major relational database management systems And there are slight differences between what's available in each one of these relational databases. But by and large, the information schema is a thing that you can use to ask the database about itself. So it gives information on columns, tables, views. uh and procedures as well as uh a number of other objects that exist in the database, indexes, and so on.

Speaker 1: And there are some difficulties that you may run into when working with the information schema. For example, since we deal in real estate data, one of the things that we have to use a lot of is geographic data. And so we use an extension on top of Postgres called PostGIS , where the GIST is GIS. It stands for Geographic Information Systems And that is a Postgres specific uh it well it builds a set of specific objects within the database that you can use for managing uh geographic information system. Now, PostGIS object types don't always fit very well into the information schema.

Speaker 1: And so that's just one example of uh Where a database specific feature doesn't always make its way into the information schema in a clean and clear way. But for the most part, you can query just about anything about all the objects in the database that you want again. with some caveats, uh, but just about anything you want uh through the information schema. And you query it just like uh you use uh normal SQL to query the information schema. It looks something like this. So this is an example that's similar to some of the code that we use in our code base to query the information schema.

Speaker 1: In fact, I will show you um what this actually looks like. And can people read that text in the back? Yeah. All right. So uh this is actually what it looks like when we query the information schema. So you know instead of just asking for table name, column, data type, and whether it's nullable or not. We're actually asking for all of these different fields because all of these different fields may come into play when we're determining What type of thing we're pulling out of the database. But this is essentially asking for all of the different columns that exist in the schema that we are concerned about within our database

Speaker 1: Um there are some things here that uh are ill advised in database programming. And I was talking with Tim a moment ago. It turns out that both Uh us at Stepwise and them at Wharton are doing these kind of auto-generating of APIs. And one thing that we uh were talking about. was that there are in both of our code bases a lot of things that you might kind of scratch your scratch your chin at. And say, really do you want to do it that way?

Speaker 1: But uh it works and it's it's still kind of neat. Um So uh so yeah, so this is this is again just pulling out all of the columns in the in our schema, ordering them by table so that we can start to get a sense of what models are in our database. So with that tool, with the information schema, we're able to know what's in the database, but we still need to tell Django what's in there. And also, if we want to use all the awesome tools with Django or that come with different Django packages like Django Rest framework and Django filters, then we need to create some models.

Speaker 1: So Django loves models, and when I say Django loves models, I don't just mean like the core of the application platform that we all know and love. But I mean the community Django and the uh ecosystem of tools and add-ons uh that is Django Loves models because they are this consistent extendable interface on top of so many types of different data systems. So there's good reason for Django to love models, but uh one of the implications is that it becomes very difficult to uh do anything with all of those different add-ons in Django

Speaker 1: without having models. models. So we have to create some models. The way that we create models, we had stepwise, because it turns out there's a thousand different ways to do anything. And you know, again, uh in my conversation a moment ago with the folks or with Tim from Wharton. Um We're doing this in slightly different ways. But the way that we're doing it at stepwise is with metaprogramming. So what we're doing is we're building up so for those that aren't familiar with metaprogramming, the idea behind it is that you're building up programming components using other programming components

Speaker 1: So in this case, what we're doing is we're creating classes using functions as opposed to explicitly writing out classes. So this is this is kind of an important concept in the way that We're auto-generating this API. So I want to make sure that it's clear to everybody what it means. So quick introduction to metaprogramming. Let's say you have the following Now, you know, most people in the room will understand what this means. This is creating a model called my model. That's the name of that class. It is extending the class models. model. That's the set of bases.

Speaker 1: It's a set of one base class. And it is adding an attribute to that model called my field. So, fairly simple model here. But let's say you needed another one that looked, you know, similar but had a few differences. So, you know, maybe you copy and paste this one and create a a Model 1 class that also has a MyField. Maybe it has MyField 1 or it has some slightly differently named uh attribute. Now let's say you need another one. So you could copy and paste in your uh models. py file again. Alright, so now you have three model classes that all look about the same but are slightly different.

Speaker 1: Um let's say you I don't need uh fifty of these. Then after a while, you know, you start adding and adding to your models. py file and You know, this is this is a bit uh you know hyperbolic and is is uh is uh very much a toy example because there's not much utility to 22 models that all have a single field called my field. But at the same time, I have found myself in a situation where I have a models. py file that's uh you know several thousand lines long. And it could be split out into multiple models. py files or multiple sub into a package that has multiple modules within it. But there would still be thousands of lines of code to maintain in there.

Speaker 1: And instead of doing that, one thing that you could do is a little bit of metaprogramming. So for example. This code here will create fifty different models. Uh yeah. This code here will create fifty different models. named my model zero to my model 49, uh it will give it a set of base classes, a set of one base class called models. model and it will add some attributes to that model , a set of one attribute called my field, and then it will create a class from that name, set of bases, and set of attributes.

Speaker 1: Uh that last line set adder on this module name model class is Because of the way that uh we are getting Django to recognize our models, uh they have to be registered within the models. py module at load time. So we have to actually bolt them onto the models. py module So it's it's a it's a it's a little bit of a hack, but it works. Um now the astute among you may say, well, do you really need some metaprogramming to do this? No, you might be able to just do this, right? You might be able to say, okay, well I have a for loop and uh you know

Speaker 1: I will there is a typo in this uh in this code. But uh I have a for loop and I'll just uh put a normal class declaration inside of that for loop and add each one of those classes to However, the challenge starts when you need more than a single attribute called my field on each one of those classes. If you don't know all of the different attributes that you need to put onto those classes beforehand, then you're back to Needing some more powerful solution. And the beauty about metaprogramming is that you can treat the attributes on a class just like data. You can create a dictionary of attributes.

Speaker 1: For the class that you're creating. So let me let me show you what this looks like in practice as well It's a little bit less clean than that simple for loop that I showed before. But this is this is essentially, let's see. make API models. Yeah, this is essentially the function that we use uh to create the models in our API. So it goes through, uh it creates attributes, creates base classes, a set of one base class, and creates a name for each one of the models Uh and then it actually uses what's called a count caching geometry aggregating query set

Speaker 1: uh as the manager on those models. And I'm not gonna go into the geometry aggregating piece, but I it's a really long name, and that's okay because I only use it once. But uh I will go into the count caching part of that later. Um not the geometry aggregating part though. That's gonna be out of the scope. And I have plenty I have plenty of opinions uh on uh integrating geometries into all of all of this thing. Um, but I'm not gonna talk about all of those opinions because it's don't have quite enough time. So if you want to know more about integrating uh geometries and GIS type stuff into this kind of thing, talk to me. All right. So um

Speaker 1: Let's move on though, and note about primary keys, right? Okay, so the Django RM requires a single primary key field on every model. Now, because we don't control entirely the um the data that we're ingesting from all these city sources. I also learned a neat trick when I was talking to Tim about this. But uh because we don't control all of the um the data that comes in from all these sources What we do is we actually add a throwaway field to each and every table when we create that table in our data store. And we use a tool, an awesome tool. This is an awesome tool, by the way.

Speaker 1: It's called DBT. Uh it's it's I think it stands for database tool, honestly. Um but it is it is really good. Um it's from shout out to Fishtown Analytics. There's some folks out of Philadelphia. Um And it allows you to manipulate your database with SQL queries, but templated SQL queries. It actually uses Jinja templates so that you can do things like instead of creating gnarly huge SQL queries with all sorts of nested SQL queries inside of them and so on, which is sometimes necessary, you can embed include SQL queries from other SQL templates. So it's it's actually that's one of the things that it does, but it's um

Speaker 1: it's an awesome tool. You should check it out. Alright, so after this we have our models. So the next thing that we have to do is create the serializers. And this is actually a lot simpler than creating the model, so it should be quicker. Um well let's see. Yeah, it involves less metaprogramming, not not none, but less. It subclasses the model serializer class, so and we exclude the primary key that we added because Django needed us to. So this is what our um our serializer creation function looks like. Uh so here is where we're actually getting rid of that um

Speaker 1: Primary key because it is useless for us to expose that Django ID. It doesn't mean anything. It's just an auto-incrementing number that we uh add onto each table because we had to. So we get rid of it when we serialize the data for um rendering uh through the API. And then we Create a model serializer class that points to the each model. And uh yeah. That's that one. So serializers again are relatively simple. Um and the filter sets are slightly less simple, but still not too bad.

Speaker 1: So the way that the filter set generation works, uh oh, by the way, so for those of you who don't know, uh have never used, have never been exposed to Django filters. Django filters the package, not just filters in Django's, for example, on query sets. But Django filters is a package that allows us to query make queries in our API that are Django-like. For example, this one here, this is uh a sample query string where we're saying, okay, well, uh give me everything where the region is in a single region. It could just be region you can equals PHL and improved area is less than or equal to 3,000 feet and vacant is true. So it allows us to construct query strings like that that will just pass that information.

Speaker 1: It will actually validate that information and then pass it along To our query sets. So we determine what filters are available based on the type of each field. And then we also add in our own custom filters to Django filter sets. So That code is also not particularly interesting. It simply loops over each one of the tables in the database, uh gets the model class for that table. Generates the names for each one of the fields. Uh it comes up here and says, all right, if it's a character field, then you can do these things with it.

Speaker 1: If it's a integer field you can also do these things with it. All fields you can do these things with. And it just creates those sets of filter uh that those filter sets for each one of your models. So that you can do things like query from your API. So the last thing that we need to do is create and expose the views or more um More uh or rather the view sets. So Django Rest framework has a concept of view sets which are essentially um collections of views that are bottled up into well they're essentially uh class-based views. They're very similar to class-based views uh in Django.

Speaker 1: Um But they're based around each one of your resources. So you can say, okay, for this resource I want to have a list, a uh detail, a re um a create and an update function. And all of those are exposed at different endpoints or with different HTTP methods. Uh so the view set creation code for us is mostly straightforward. It does require some more metaprogramming. And it does and one of the things that we do is start to auto-generate some of the documentation. I don't know if how many people in here have ever seen Django Rest framework auto-generated document

Speaker 1: documentation How many people have used Django Rest framework? Let me ask that. And how many people have used the browsable API feature on Django Rest? framework. Okay, so though that that documentation feature that comes with a browsable API where you can set the Essentially For each one of our endpoints, we build up a uh a set of documentation that if you go to that endpoint You can see the uh the documentation for using that endpoint directly

Speaker 1: on it. There are also fancier ways to do this. Again, I have to keep referring to the work that they've done over at Wharton because I got a tour of it from Tim and I'm just like, oh my god, this is this is great. But we have been trying to solve some of the same problems independently within the same city somehow. So, as I said, generating these view sets requires a little bit more metaprogramming where we are grabbing the table name. generating the name of the view set, using a whole bunch of base classes that do things that are somewhat outside of the scope of this talk, some of them are.

Speaker 1: And then creating the actual view set uh using the type meta class. And one of the things that I didn't talk about in metaprogramming is meta classes. And a meta class is essentially For this purpose, it is a thing, it's a function that will return a class object. Not an instance of a class object, but a class itself. So this, when we're calling type here, we're using type as a kind of meta class. Type is the metaclass of most classes by default. But we're using type as a metaclass as a function to create a class object. just

Speaker 1: like we did with model base, which is actually the base, the meta class for models. model in Django. That it uses. So we use that model-based class to create our model classes. But they're all just functions that create classes. So additionally with view sets you'll also want to set up throttling, authentication, permissions, pagination, and any special renderers that you might. might want to use. Most of those, again, are out of scope for this talk, but there are two things that I want to talk about, which are the permissions and the pagination.

Speaker 1: So as far as permissions go, we wanted our API to be consumable by our clients, but not all of the endpoints. So the thing that we decided to do in our API was use um Django Rest Frameworks permission framework. So first what we did was we created uh a an access profile that is attached to each user And essentially tells what uh which routes, which um which resources uh a particular user can access. We then use that uh auth

Speaker 1: profile uh or that access profile uh in a Django rest framework permissions class that is derived from is authenticated and essentially just checks whether the user is authenticated and if they are, well if they're not, then they can't access this. resource, but if they are, then we've added a uh a function to all of our views that was this comes from one of those base classes uh on the view sets So it adds a function to each of the views that allows us to get the endpoints that the user is requesting. Uh the endpoint that the user is requesting

Speaker 1: and allows us to check whether the user has access to that endpoint. So the other thing that I wanted to talk about was pagination. Not specifically pagination, but really just counting. counting in the API. So by default, Django REST framework pagination classes include the total count of records that match any particular filter that you throw at them in the response. Uh now initially I didn't think this was going to be a problem. I thought databases were pretty fast at counting How many things match your query? And it turns out they are. When I say, you know, fast, I mean like milliseconds, not seconds.

Speaker 1: It turns out it's closer to seconds when you get up to millions of rows of data. So that is too slow when you're talking about an API that a uh an application is is being built on top of. So in order to get around this, what we had to do was implement our own pagination class. Uh and by the way, it is awesome that Django Rest framework like even allows you to do this. Like that it it had that uh you know Tom Christie and everybody who else who works on Django Rest Framework framework, all of those awesome people had the foresight to say, you know, maybe people are going to want to customize the way that they paginate their

Speaker 1: their data in the API. So specifically, we use something called a conditional count pagination class, which subclasses limit offset pagination And if the user includes a certain query parameter in their query, then we give them the count of the objects along with uh the results of their uh the results of their uh their query. If they don't then we don't count the objects. All we do is we give them the the response. This speeds things up dramatically. You would be, if you've never experienced this, you, I kid you not, you would be surprised how long it takes the database just to count the things that

Speaker 1: that it is able to return to you. I mean, but otherwise, Postgres is wonderful. It just takes a long time to count. So uh so we have this custom paginator And uh we override a few of the limit offset pagination functions, specifically paginate query set, where we only include the count if uh We are asked to, otherwise we just skip it. Get next link because in Django Rust Framework you have a link to the next page of results And the way that Django

Speaker 1: Rest framework works is if you don't have any more results on the next page, then it won't give you a next link. It'll just say no. If you have not asked with our paginator, if you have not asked for the count, then we always give you a next link. And you just have to check it and see if there are any results there. But it turns out that this for our use case is faster than including the count with every page of results. So at this point, we have uh an auto-generated API. We've We've created our models, we've created serializers and filter sets and views as well, and now

Speaker 1: people can access data all the way from the database out to their browser. Um so in conclusion, should you auto-generate an API? So my recommendation would be if you just have a few tables, uh no. Just create models for all of those tables. And what's a few, I mean it's really up to you. It depends on how many fields are in those tables and how complicated your logic gets when uh dealing with each one of those tables and how uh customized your logic is in in dealing with each one of those tables. For us, uh

Speaker 1: the the number of tables that are generally served by our API is somewhere on the order of several hundred. Uh for some folks it's several thousand. Um but if you have fewer than like 40 or so tables? No, don't don't do it. Just maintain your models. py file. If the majority of your data is user-generated, I wouldn't recommend you do this either because you're not starting from a um an existing data source. Your data is coming from your users. And so there's really no need for you to go through the extra headache of uh Spinning up a um a database to do a bunch of extra logic on.

Speaker 1: You should just let Django handle your migrations uh and manage your database uh if most of your data is coming from. is generated by your users. If you can trust your schema to be predictable And this one's a little a little uh I'm not sure. Uh because if you can trust your schema to be predictable, but it's still thousands of tables. thousands of predictable tables, then maybe you should auto-generate your API. But uh you know you could you could go either way. You could just um generate your for example generate your models. py files. There are tools to do this. You could use InspecDB, but that has some drawbacks.

Speaker 1: The folks at Wharton uh are using I I know I'm just fanboying on Wharton at this point. Um but the folks at Wharton have uh developed uh more sophisticated methods uh of generating models. py files and we should totally all open source this stuff but takes it takes some time. But yeah, the if if your schema is predictable, a better choice might just be to um generate models. py and leave it at that. So would I do this again if I had the chance to go back in time?

Speaker 1: Uh I don't know. It's still kinda neat. Um and you know, for that matter, we've been running on this for uh upwards of two years. at this point, which is a pretty good time for for any particular system. I mean, um, you know, it's still it's still going. Sometimes it's a little bit of a headache to uh to maintain and to uh have to uh go in and fix something for a particular endpoint and not have it break all the others but um But I don't know, it's it's all trade-offs. So if you're creating a data warehouse and none of those points above apply, then maybe

Speaker 1: maybe you should use this method Otherwise, I'm not really sure. But it's still neat and I would recommend it just for your own edification. That's it.

Speaker 2: Thank you. Uh do you I was just curious, do you have a lot of relationships you have to work with? Um in my job we've got a lot of relationships and that didn't seem mentioned.

Speaker 1: Right. No. And that is useful. It is useful to not have a lot of relationships to work with in in in this case. I have thought about that a lot because there are a lot of relationships that we could build into the database. It just happened to be that we weren't when we put this together. And if we did It would require rethinking the way that we do a few things. Specifically in uh reading the In introspecting the database, right now we're not picking out any kind of foreign key relationships, any kind of um Any any of that stuff. So we may move in that direction in the future.

Speaker 1: I don't know whether we will use this exact same method if we do.

Speaker 3: Custom paginators are going to be a solution to a problem we're facing in production right now. Thank you for that tip. Um have you given any uh investigation to using explain plans from the database to get the counts faster

Speaker 1: Uh yes. Uh and um uh Ultimately, it was a it was a better use of my time to just do the custom paginator. So I spent a lot of time trying to explain What in the database was taking so long? Um and ultimately I I I tried a number of things. One of them was just using uh an estimate of how many things were in the database or in in a particular query as opposed to the exact number.

Speaker 1: But it turns out for a number of reasons, sometimes we need the exact number. And so it it for our use case This was just the best solution. There are other approaches that people might be able to take in terms of estimating the number of the number of results, but for our use case this was the best solution I guess the best. I don't know. It was the solution that I came up with at the time.

Speaker 4: Thank you. That was really great. Um I was wondering if you do any tricks or anything or if it's fast enough when you do application restarts, but maybe you only have one new data table, if you like cache that or anything because you're pushing everything onto the stack on start.

Speaker 1: Right. So caching caching tables or rather caching the information about tables is something that we've considered. Uh it just hasn't been something that we tackled yet. We're four people right now and growing, by the way. But um But it it just hasn't been something we tackled yet. So like you know, something I was thinking about is is at the end of our data pipeline refresh, maybe we uh you know dump out a like a SQL representation of the schema and then we can just read that in as opposed to going all the way to the database and running those queries to because that does take I mean not an inordinate amount of time with hundreds of tables. It's maybe Okay. Maybe like thirty seconds or so, but yeah.

Speaker 5: Hi. Uh it looks like when you get a data set that's different from your previous datasets uh that's gonna change all the filters on your API. So the interface for clients is gonna change. Have you done anything to control for that?

Speaker 1: So there are um there are things in our in our API that are okay to change Some things are not, and those things we check for at the data pipeline level. For the things that are okay to change, right now we just let them change. And we do we do get notified when things do change and so we are able to let clients know and so on and you know we were able to respond to it internally, but um there are just things that are okay to change. And for those things that are not okay to change, we we respond to it earlier than at the application level by building those checks into the data.

Speaker 6: Thank you again, Jumbe. You ready for the next speaker?