Wagtail charts and graphs - Sævar Öfjörð Magnússon & Arnar Tumi Þorsteinsson

Speaker 1: I'm very excited to introduce our next speakers. They're going to be speaking about the uncharted territory, creating highly flexible charts. and graphs in the Wagtail admin. They'll be presenting their talk from remote, but they are here live. So if you have any questions, please put them into the Zoom chat or into the Slack and we'll make sure we get them. And we'll also be taking questions from the room. We have, I can announce that it is Sivart and Timmy , we're gonna talk to you more about Wagtail.

Speaker 2: Yes, thank you. So I'm going to uh share my screen here. So uh hello everyone, Back at Telespace in Cleveland and all of you joining uh via Zoom. My name is

Speaker 3: Saiva, and my name is Tumi. And we are two of the founders here of Overcast in Iceland. And we're gonna talk a little bit about charts in Black Del today. But first we'd like to tell you a little bit about Overcast

Speaker 2: We can switch the slides. Yeah. So uh the company was founded in 2013 by four developers. Two of them present here now. We were going to create products for the advertising industry here in Iceland. Today we run most of the infra infrastructure involved in uh planning and serving ads to the Icelandic market But we also create websites for various uh medium-sized and large companies here in Iceland. Uh we've always been uh big fans of Django. So we naturally wanted to uh use Django, but we

Speaker 2: always had problems finding useful content management systems for So when we discovered Wachtel, we immediately saw its uh potential and we have been using it for all of our web development work. Since then uh that was twenty fourteen version zero point six. Um Yeah, we've been uh involved in the wagtail community a little bit also. We hosted the uh uh sprint here in Reykjavik in 2017 uh we uh we hope to do more of the same in the very near future but uh back to the topic of this talk uh

Speaker 2: The charts. Soon after we started creating websites, we found that many of our clients needed help with moving their annual reports from print to the web.

Speaker 3: Uh these reports contain a lot of statistical data, which means a lot of charts. And These are just some type examples of types of charts that we have created for crack clients using Wagtail. In the beginning, you know, back till 0. 6 and early Streamfield days, this was uh yeah, this was a very manual process We were basically creating JSON configurations for chart. js and embedding them in text blocks in Mugtail. We soon figured out that this was way too hard. and created custom blocks using the char fields for comma separated data and basic

Speaker 3: blocks for configuration, which made it relatively easy to create a simple bar and line graph Our clients needed all kinds of charts. So here's a common example where you have stacked column chart with negative axis combined with a line chart. And sometimes we think, you know, our clients basically invent new types of charts just to mess with us. And the client won't take no for an answer. They just want, you know, they're pre-chart.

Speaker 2: Um so we uh run through all the charts that we have s shown you here and then uh Well yeah using chart js which we have found to be a very uh flexible and developer friendly library for creating charts And uh as we continued creating different types of charts, we added more options and chart types and configuration like multiple series of different types and colors, multiple y-axis, custom labels, different types of axes such as time-based logarithmic etc etc so this is all quite versatile

Speaker 2: but all these options make it quite uh okay so This is an actual screenshot from a wagtail page of one of our clients. It's about 12 charts, but I don't know how many

Speaker 3: kilometers

Speaker 2: yeah tens of thousands of pixel it takes it's

Speaker 3: it takes a way too much space

Speaker 2: yeah it's way too much space so uh we need something better

Speaker 3: And there are other problems with this. I mean this is obviously taking up way too much space, but um data is entered in in uh plain text. uh into uh chart char fields uh with comma separated values um and labels are in other fields they're disconnected from the from the values uh so series and labels are not together Which means this makes the whole process of creating and updating terms error-prone and easy to make mistakes. And and they take up yeah of course way too much space.

Speaker 2: So after uh few years of using this approach we were starting to think about uh better solutions. And uh at a similar point in time we saw that Bactail has this new telepath Functionality for creating blocks in Ruck10. So creating you know more complex blocks uh is more feasible now. Uh But uh what do we really want from this solution? What we wanted was that uh data entry would be more like working in a spreadsheet where all the

Speaker 2: series data and labels would uh live together uh so it would be more easy to you know edit the data and spot errors and such If we could find something that allowed easy copying and pasting from spreadsheet software like Excel or Google Seeds, that would of course be a bonus. We want to have a uh like a more we we want to have multiple configurable uh chart types uh preferably all the types available in chart js of course That's our end goal. To make things simple, it would also be nice to have it all living in just a single block.

Speaker 3: And we want the UI to be compact. We burnt us with the other UI. We only want the essential things to be seen by default and we want to have some basic configuration options. And so that developers using the block could just plug in, it plug it into a project, customize it a little bit, and start creating charts. Template rendering should be convenient. You don't want the developer to do too many things, little things here and there to get the app working. It should just be like two lines and be done with it. What we've come up with is a package that we call MockTell Charts. It's still a work in progress, but we hope to be able to release it very soon. Chart rendering is still based on chart.

Speaker 3: js as before, but we have now implement implemented a spreadsheet editor that we wanted it using a library called JSPreadsheet. This allows us to have a very compact layout for a simple chart. When you create a new chart, you just see three fields, title, chart type user, and a spreadsheet This is where you enter the data and some configurations for each series. And then there's a little button in the bottom that holds all the chart settings that are hidden by default Right now general settings for the chart look like this. This is just some basic configuration with minimum and maximum maximum values for axes, legend options, staffing options and such.

Speaker 3: And we plan to add uh some more options in the future.

Speaker 2: Yeah. Now

Speaker 3: Sarah is going to give you a little demo of how this works

Speaker 2: Yes, if I can find it. So uh here we have uh Here we have a simple wagtail page with a stream field. Just uh before I dive into creating some charts, I just want to show you how this was created. We have a basic home page Uh we are using a stream block here, which uh is defined over here in the blocks. We are importing our chart block from Locktail

Speaker 2: Charts. As you can see, the colors have been overwritten. So we're not using like a default color scheme.

Speaker 3: Very creative colors.

Speaker 2: Yes, I like the names. So all you have to do is like install the packets, put it in your installed apps, do the block thing and connected to a page and and then you have to have this little template tag where you render the chart JavaScript and such. So fairly easy to install. I want to show you how we create a graph using this auto. So as you saw from the screenshot before, it's just three fields, Thailand. I'm going to show you some scary data. It's the temperature in Iceland

Speaker 2: it's called um you have the type of chart and then you have the data entry which is spreadsheet like uh I have prepared the data here. So I'll just uh quickly copy it over here. So here we have the data copied from Cuckoo Sheets. I'm going to copy the left hand here also. We're going to set up type for each series. And perhaps a color for maximum temperature. Temperature should be red, of course, and

Speaker 2: blue for the minimum temperature we have yeah something

Speaker 3: average is also cool

Speaker 2: so uh it should all be blue so we have input out data we're going to preview this So yeah, that's nice. I would like to see the legend. So now we check the uh settings button for the legend I'm going to display it on the left and I'm going to put a label on the Wax So yeah.

Speaker 2: Uh for all of you folks over there in the For all of you folks over there in the States, uh we have also prepared some data. In Fahrenheit, so you can fully comprehend how cold it is. Uh So all I have to do now is just copy the block, of course, and I'm just going to Paste the data itself. So we'll be very

Speaker 2: So that was pretty quick. Um So now we have like these column charts with uh line line series also mixed in We can play a little bit around with uh some configuration options, etc. But maximum

Speaker 3: mean on the x-axis and y-axis.

Speaker 2: Yeah, yeah, yeah. So we're planning some more options for this, but This is what we have right now. Uh I have some more interesting data. Uh one of our clients is the state-run monopoly on alcohol beverages. Uh here in Iceland, you can't buy alcohol in the in a regular grocery store. So uh We have these we have these uh special alcohol liquor liquor stores. You can't buy alcohol anywhere else. So we do the annual report for this client, and uh I was thinking of showing you the uh total sales of red wine by

Speaker 2: country of origin for 2021. So you should never make a pie chart, but I'm making a Special uh mission here, sorry. Uh So here we have the total sales. It's a little bare , so I guess we could add the Legend of course. Put it on the left. Oops. Review. So this is this is nice, but I think we could

Speaker 2: Make it nicer. I'm I have a feeling that it would look better as a donut chart. So I'm just gonna copy this real quick. I'm gonna change the type to donuts And preview again. So I think this looks nicer, but that's just my opinion. And when you've selected the layout that you like you can just delete the block that you don't want anymore uh

Speaker 3: Italy is by far the biggest

Speaker 2: yeah

Speaker 3: And the US and the French, I think they're pretty much the same.

Speaker 2: Yeah, so Italian red wine story, yeah, by far the most popular. Then you have Spain and Chile and United States. So uh I have one more data set. I think we have time. Again, uh this one's about alcohol, of course. I mean you can't you have to

Speaker 3: go wrong without

Speaker 2: yeah yeah yeah and of course there it's a bar chart so after alcohol uh this one is uh sales of alcohol this is pure alcohol so taken all the other stuff calculated it away. So it's the total sales of alcohol in liters from 2011 to 2020. We have uh the data over here. I'm just gonna copy that over quick. I'm gonna copy the settings over as well here for the series. So I think I'll add the legend at the bottom

Speaker 2: And preview. So this is the sale of alcohol. Uh here is most popular. I wonder if anyone knows what happened in 2020. which increased the consumption of alcohol by quite a lot. Uh yeah. Of course it's uh it's not a good fact but That's what happened. Um I think if we want to compare the data in a better way, it's common to put it uh showing in a stacked graph so we just select the stacked option preview that uh so we see that the total volume of uh alcohol sold is actually yeah it's

Speaker 2: It's uh a lot more, but uh I'm not really sure if uh we are selling more of beer versus uh wines and spirits or strong liquors. So So the final option I I'm going to show you is could go to a stack 100%, which normalizes the data and uh yeah normalizes to to 100% uh so that you can see the relative uh sizes of each series. I'll go ahead and put an uh label on the X axis also. Gears and uh left

Speaker 2: this um and leaders sorry people in the us this is leaders not going to convert it so yeah we can see now from the data that it's uh The proportions are mostly the same.

Speaker 3: A little bit more strong.

Speaker 2: Yeah. But the total volume has increased. So this was our quick demo of our charting module that we are working on right now. This is most of the graphs that we have available. There will be more. And yeah, work continues. So I'm going to switch back to the presentation. If I can press a button. Yeah.

Speaker 3: Yeah, so um This is where we are right now. We have a few steps left before releasing. Our next steps are that we think we need a A little bit better way to implement the user interface. This was done in Vanilla. js and becomes quite complex quite fast. So we're uh moving the rendering of some of this to the React to React. js. Which also makes it easier for us to implement new functionality and having custom rendering per chart type. Well, easy rendering both in the admin and in the content And we also want to add

Speaker 3: some nice overrides , some nice defaults that the developer can override for chart rendering settings for supplying a configuration direct dictionary um similar to what uh chart. js has and uh we need to release this and we expect to do that now in april

Speaker 2: So yeah.

Speaker 3: So that's it from us. Um do you have any questions?

Speaker 2: So I'm looking at the chat here. One person got my bar chart joke, which is super

Speaker 1: Awesome. Thank you so much. I don't know if you got the end of the applause in here. There's been a lot of chatter in the in the in Zoom and on Slack about these amazing bar charts. So um let's take some questions. We have a few minutes. We have about five minutes, and so I'm thinking that's like three or so questions. I see one from Andrew in the chat. What are your thoughts on the trade-offs between pulling the data from behind the data behind the charts into the into the CMS versus using the external charting system and pulling the results only.

Speaker 2: Yeah so For our purposes, I would say that using an external tool is not very feasible because what we are doing is we are creating annual reports which they have to live for You know, some foreseeable future, possibly some years, five years, ten years, I don't know. But we're creating a snapshot of something, and uh I think it would not be acceptable to lose the data in the graphs uh or charts uh if a If an external tool would uh I don't know, stop service or something like that. So we have to have the data in the CMS.

Speaker 1: Awesome. Thank you. I'm going to alternate between the chat and in person if we have any, and then I can keep going to some more questions in the chat. Is there anyone here in person with a question? Awesome. Great.

Speaker 4: How could the configuration to charge. js specifically if someone wanted to run say different charting library on the front end of that page.

Speaker 1: Okay, in case someone wanted to use a different charting library, how tightly coupled is the configuration to chart. js?

Speaker 2: No, not really that coupled. Uh obviously right now it's And of a minimum viable product, so more coupled, of course.

Speaker 3: The data is stored basically in a JSON-like format. The text field.

Speaker 2: Yeah. So I mean it would probably be fairly easy to create a renderer of shorts.

Speaker 1: Great. Do you plan on allowing CSV imports?

Speaker 2: Okay, I haven't really thought of that

Speaker 3: because uh it's easier just to import The CSV to actually then paste it in.

Speaker 2: Yeah.

Speaker 3: But I don't see any

Speaker 2: if someone sees the need for it. I mean it's probably a possibility. Right now we we usually you know do something to the data in a spreadsheet you know where you can import csv already

Speaker 1: Awesome. We have a question about any idea to connect to external data sources? Are there plans or ideas to connect to external data sources?

Speaker 2: No plans today, but uh that's an excellent idea.

Speaker 3: Yeah. We're open for

Speaker 1: awesome.

Speaker 2: Are you a volunteer is that a volunteer for

Speaker 3: awesome?

Speaker 1: And he's in person so I can see his face. Can you share details about the complexity of the admin UI switch to React?

Speaker 3: Yeah, well not our

Speaker 2: suh I I think So it would be a bit complex probably to uh do the initial conversion. Right now we're just uh doing it uh vanilla. js. Uh but uh we are we use React. js a lot, so we uh we know it quite well. Uh I think what we would gain by converting to or going to React. js is that we could develop much more uh complex UIs and and settings. using React instead of you know having a bunch of if and else is

Speaker 1: So we have one more minute, I believe. Depending on how detailed the question is, we can probably do one or two more. Anyone in the audience have a question? Okay, we do have another question in the chat. What would be the complexity in adding data analysis via results from another module like pandas?

Speaker 2: All right. Well, I've never used Panthas, so uh I can't really answer that. So sorry.

Speaker 3: I mean no, I don't think

Speaker 1: Okay. Sounds like a potential con cont contribution. Yeah. Well thank you so much. I appreciate you uh your talk. It's been great

Speaker 3: Thank you