Extending Wagtail CMS as a Spatial Data Infrastructure

Speaker 1: Enough.

Speaker 2: That's good. Okay.

Speaker 1: All right.

Speaker 2: Pardon the technical context. Okay, I'd like to welcome our next speaker, Cesar Benjamin. He is a Digital Government Director all the way from Monterey, Mexico. He will be talking about extending Wagtail CMS as spatial data. Give a big warm welcome to Caesar.

Speaker 1: Thanks. First of all, I want to say English is not my primary language.

Speaker 3: I I have difficulty sometimes to express uh

Speaker 1: excuse me so

Speaker 3: uh uh in any way uh I will try to do my best. Uh uh a disclaimer uh in my country there is uh some um thing with the work infrastructure and it's difficult to say and I I I will say it's the SDI instead of special data infrastructure uh from this month Is the definition in boxes uh something about a set of uh technology uh

Speaker 3: rules, people, data, so on for uh the management of territory , mainly in in in government uh aspect uh topics but uh can be used in uh research uh academics and so on On the other hand, what style uh everyone knows is uh CMS, not uh special data uh infrastructure uh is uh it's mainly for informative contact Content management for blogs, news, forms for surveys

Speaker 3: and document and image management. Gurry Well, there is not an SDI, right? I personally I I worked with uh Wachtel for uh deploy many sites uh in in Monterey in government. Uh We have a single instance of WACT to manage all the content of all the sites in uh multi-site uh Perspective. Uh I have to

Speaker 3: uh Something about uh the old databases uh in the in in in in Hughes Stars uh was not uh thinking uh about the uh special types uh in the beginning uh was the text types uh they type they types um Yes. Fortunately Postgres have uh an extension

Speaker 3: which allow us uh to manage special data in in the manner that is uh is not only the representation of the geometry uh Instead of that uh have uh many operations, uh uh special indexes, um Can we get uh do queries like in the tabular data, but uh in the geometrical way? That was possible along the evolution of the databases and today with post -GIS we can do that.

Speaker 3: When we think about representation a place, the most natural way of thinking is cut files like in this slide. These files give a special representation, but it's only a trap. It has no information relative to the features of setting drawing. We are not able to extract any useful information without the adequate context. uh the information is in fact uh part of the draw of the drawing it's not data data information uh in fact

Speaker 3: There is a format called Shapefile building by the company called Sri Who is uh a company that uh provides uh technology for special information. was the the first of uh of company that did that but shape files uh today is uh an of that outdated format is based based on dbf files uh who comes from the d base tree a database from the seventies uh His special component was an independent file apart from the

Speaker 3: DBA and is a collection of many files. uh and that uh becomes uh complex to handle with the time. On the other hand, post Postgres uh Postgres QR have natively all that uh uh I only want to show the the a great amount of of that data types that it supports natively. Django is able to support the majority of the operation with that

Speaker 3: data types. uh this form of the R O R M and Yangular framework do it through his serializers This means that nowadays all the necessary pays to have a special data content management system exist. And S D A is a reality representation about where something happens. Mainly we use abstraction of reality using points, lines or polygons to represent something in space.

Speaker 3: And that is is now possible in databases level thanks to the extension for keys. With all that pieces we can uh build an Sti In fact, Django is able to handle queries uh to the to the ORM uh and his special operations natively without no needs of external libraries or other things. But uh I if all the parts for this system exist, I

Speaker 3: is not uh one yet. Well yes it it exists and in the market are some uh alternatives today but are some complex uh like that uh diagram uh need a lot of components uh integrated and connected uh to get uh the the complete uh system. Nowadays there are two solutions that were widely known and used for online for one hand arc arch g online which is expensive and not is and not so

Speaker 3: standard in many aspects and GeoNode built with Django and GeoServer since all of us here value In and care about free software, I am going to leave out Argis from this conversation GeoNode is a strength project that gathers many existing libraries and frameworks and uses them to create a platform for special data management as a sort of CMS They even promote themselves as a special data infrastructure content management system. But it 's a huge uh amount of

Speaker 3: technologies, libraries and stuff is uh it can be very complex. For uh any person who approached for his first time. So uh I I I will uh uh comment uh a bit about my previous experience with spatial data I was working in the Universidad Nacional Autónoma de Mexico in a project for the government of Jalisco at the year 28

Speaker 3: This was an inventory of urban trees in the metropolitan art area. This needs the handle of data and database databases about the existing trees in the city, his health stake status , So on. Uh when I did this work, uh I had to do it uh in an artisan way uh of the The parts there I did uh uh a platform uh for registry of hazard and risk map with gender

Speaker 3: disparity for the sapopan halist ho Sub opense in Halisco in the 2020 year. This platform is active in the yeah And uh index of social environmental conflicts in all the country. This is the This last was made completely in wacht tail when the pi the persons uh who request me the platform

Speaker 3: they uh come with a set of files and I ask uh how they think they will uh warranty the uh extensibility of the of the data. Then I propose to use rack tail instead of the files But this way uh was uh a bit hard for me because each time each new platform uh I had to build a new platform from zero uh model each part uh and so on

Speaker 3: and was uh very hard work. Then I I discover uh a platform called John as I as I say Before uh in fact we used uh GeoNode in the government of Monterey as our main uh special that special data infrastructure uh is is is really a very good uh product Well I I guess it's uh can be not enough for certain things of of stuff

Speaker 3: Uh uh for uh for example, uh I discovered that the last uh Version was stuck in Yango three dot two version um So all then I uh open an issue in in GitHub, I take the tasks of operate uh uh many of the dependencies and proposed that it must be updated. I did uh a lot of work to upgrade uh GeoNode to the recent version of Django uh it exists uh uh uh issue

Speaker 3: close it today but I I guess uh we can do it better the platform is so huge is the the m it demands a lot of higher computer resources. It requests two database, one for Tjangon metadata and one one more two data for your server and the two services exist uh separately and needs a bridge to handle information from one uh place to another and the sum of all the it all its parts makes into a weird and painful Frankenstein

Speaker 3: system And to make things harder, installation, fine tuning, and maintenance require a high knowledge and skills of many technologies. So my proposal is following the keys principle with only one thing and a library that publishes special data standards called by OGC API Py OGC API is uh a labor for the standard OG C API, uh which is a standard uh in development yet is a model is a modern standard

Speaker 3: then with only that uh these two pieces we can have a simple and modern SDI I I was working in a proof of concept using these two parts. I am about to show uh that uh proof of concept. Uh now it runs and allows me to store and publish native RESTful APIs from Django REST framework and It consists in a single service, single service, Django, a single database.

Speaker 3: Can be a can have can be have a fine-grained ACI basic in in Django only And well over this was the pump into this picture. I like a lot the workflows, that revisions, uh that kind of stuff that uh provided. us in pages but I I did uh the y I used that uh the mixing stuff implements that for handle the the flows in in in my props

Speaker 3: Okay. I I want to show How can I change I can laugh. Yes, yeah. Great box. Thank you. That's a good promotion.

Speaker 3: Okay, this is my demo of uh using uh Water for uh special data management. Uh In in GeoNode, for example, we have flat tables with data. Each uh file we apply to the platform has a set of registry. in a flat mannet and we have no relations and uh a way to update uh or add new registries the set uh keeps like one

Speaker 3: supply of updates and we can add new registers then uh well i have uh many types of resources tables that for forms polygons lines I have uh this was made as as a snippet there, not not as not as page because the the page had a lot of Feels uh that

Speaker 3: is is not necessary. Uh we we have the ability to add uh files but uh in this is another manner different to the documents handle because I want to upload the file once per uh layer uh and not the file uh don't need to stay for other uses When the file is is is aided I can uh progressive progressively

Speaker 3: add more files and extract the spy the special data uh and uh i i will try a wrong uh file This file this will make a show and error because uh it's not the The same kind of uh file and it's a cube.

Speaker 3: If we start with no files, I I can add a special register. And the there you are. can have the any shape of of his structure. I want to avoid the modeling of data each time the new data set is needed then I can use your uh JSON data

Speaker 3: And does it not matter because it is magnet with the capacity to only at at the the data. Then if If I have a special format file, I can add Apply with geometry and data

Speaker 3: No , we have Each registry is in the is the in individual space with his uh not special data in JSON format we can can change the the fields on this it's not constrained to uh fix it uh structure. All this I I am uh telling uh to you is uh about uh limitations I count in GeoNode

Speaker 3: when I upload an a file in GeoNode, the structure of the file constraints me to always use that structure But with this way I can add a new field and

Speaker 3: I don't know what happens. But uh with this uh We can have uh management of special data inside Wachtel in the Wachtel way with The use of revisions and the uh workflows and we can see the history of all the updates in in in the in the data But connected and published as uh

Speaker 3: a specially standard using the IGL API library. Por ejemplo, eh, este We can take information uh that is in in wacht tail at this moment uh using as this api standard

Speaker 3: This page uh seems uh uh visualiz uh seems to be a visualization application but it it is not is uh An H Tm H D M L Way because React in in fact is an API With two modes of view, the Api view and the H D I L view, with the same data. Then The thing I I want to achieve is to have uh API

Speaker 3: to handle uh uh special data but handle special special data is hard today because we have to do it manually uh taking all files uh configuring uh uh editing configuration files and so on. There is not a way to handle this uh information in a comfortable way like of tail or yang one then i want to take the the the two parts and make uh uh uh only tool

Speaker 3: to manage publish and using the special diagram um this as i thought uh told uh uh is a proof of of concept there is yet uh work to do I have to completely integrate this by open Geo API because today I have to configure manually in a manual way each layer added in Wachtel to see in this space. But like this made with Python,

Speaker 3: it is possible. then in a couple of weeks I guess I can achieve the on the feeble integration. And Well I I lost the presentation but the last uh the last uh so Okay.

Speaker 2: Thank you, Caesar. That was really cool what you built with uh geospatial wagtail. So we're gonna take a five-minute break and then we will uh wrap up with our panels, panels, uh sorry, packages, packages, packages discussion, uh, which will be a panel. So uh five minutes and we'll see you all