Bringing a live blogging tool to the Django ecosystem by Tidjani Dia

Speaker 1: Hello everyone, my name is Tijanja and I'll be doing the live blogging with Wakel Live Talk. So The summary of the talk I'm going to give is I will first introduce myself, then I will explain a little bit about the GSOC experience I had last year. with Woktail Live and I will then present Walktail Live and make a live demo that Kelvin was mentioning uh earlier and then we will I will explore some of the ideas we have for the future of Wakel Live. So my name is Amodu Tianija. I come from Senegal. And I have a bachelor in mathematics from the University of Lille in France. That's where I did my university studies

Speaker 1: And currently I'm working remotely as a junior developer at Toshbox, which is a digital agency based in the UK, but they also have branches in US and Asia. So Yes, that's me. So yes, Y G S O C. So I uh basically I have uh yes, I have a bachelor in mathematics, so I have a learned web development in itself. So Yes, the way I started to learn it is yes on a Sunday on the spur of the moment I just say, Oh, let me just try and see how how is it so f because I always liked the idea of building websites since I was younger. And I had an algorithm and data structure scores in in university that I loved a lot.

Speaker 1: So uh yeah that was all of that made me consider oh why don't I try to learn it. So I've started by learning Django building small projects following CS5 And the more I was learning, the more I was saying, oh, this is yeah, it looks a little bit to mathematics, like yeah, there are many things, many, yeah, many, you have many challenges to solve and that was very exciting. So and after some months of learning uh the basics, I wanted to go beyond that and have maybe real world experience, see how what I w was learning would uh would be applied in the real world and then I wanted to participate to the Google Summer of Code program So uh what is uh oh

Speaker 1: what is Google Summer of Code? The Google Summer of Code is a global program focused on bringing new contributors into open source software development. So yeah, open source organizations have submit applications where they present some projects they would like students to work on and then at uh When the application is approved, students can now uh explore the IDs proposed by the organization and choose one from that to that they will work on. during generally during the summer and yes so G and uh the is historically GSOC was only for students but this year the that the format has changed and now they are welcoming any new open source contributor, whether you you are a student or not.

Speaker 1: So yes, that's what is GSOC. So now the thing is yes, when you do that you have to choose a an organization that you want to work with. So as I was learning Django, so the you know naturally I wanted to use Django and Before that, I also wanted to build a block where I would document my learning journey. So I tried to do that with Django. But yes, when I was doing that, I had some obstacles, if I can say so. And mostly you the one that was more uh obvious was uh managing code blocks, like when you want to include code snippets inside the uh block article. it was not very, very, you know, easy to do so with Django.

Speaker 1: So yes, Django is a great software, yeah. The admin interface is very useful for managing maybe models administration to website however it is not a CMS maybe it is not designed it is not designed to manage content. And yes, it makes sense to that. So that's why I wanted to choose something else. So and then why did I so Woktail, which is a CMS. So before when I was doing research to build a live block, we uh a a block a live block a block with wagtail with django i was i always i would always say that uh wagtail had was very well Ranked always on the top three, if not in the top one, in Python's same rankings. So yes, I'll say I decided to give it a try.

Speaker 1: So I started building a little block with it. So yeah, I discovered the stream field and yes, it wo it gave me, you know, a very, you know, great flexibility on merging content. You can mix many types of Things. I hope that I will have the chance to show you how Streamfield works. And yes, it I also found external packages that would greatly simplify handling code blocks. And I yes That's why I loved using Wactail first as a user and then yes considering how it was to uh to use worktail being as a developer or being an editor and having a good ecosystem around it. And I said in myself, yes, why don't I why don't I try to work on this?

Speaker 1: I started making small contributions, so with the help of the Wactail community itself, which is a very nice one, a very living people are very welcoming there. So yes, and it was very nice. I also liked a lot the open source world, like the yes, the idea of yes bringing uh you know great software quality free for to people to use so yes I really like that and so yes and the way GSOC works is that organization will propose a list of IDs and they will give some details about how what they want to do, how they want to achieve it, and maybe how what what level of complexity will it involve. And then we choose which one you want to work on.

Speaker 1: So that's how I choose Wactail Live. So that's the that's what I chose Wactail Live. So before talking about Wactail Live, I will just take uh Little moment to talk about my personal experience about GSOC. So, yes, in one word, I would say it was amazing. So it was a very rewarding experience. You know, I have uh it helped me learn a lot of new stuff like Django and Wacto vector internals, asynchronous programming, the WebSocket protocol, testing more using PyTest and all packaging and releasing software, continuous integration with GitHub actions and also it was really um you know I'd learned a lot during you during the this

Speaker 1: uh the G S O C the last year. So and when you are when you have when you are chosen to work on a project You are you have some mentors from the open source organization that will help you, yes, that will mentor you during during during the duration of the project. So I had, you know, wonderful programmers as mentored, but also they are very nice human beings. So all of that were yes. It was really, really a very cool experience. Doing the G S O C and so yes, so the project I choose to work on during during the GSOC program was named WalkTell Live. So the initial ID

Speaker 1: was from Kern and Tom who were in the who were actually my mentors for the program. And they made an initial put a proof of concept so that you know that would show how what they mean by Wactail Live. So yes, a live uh Wactail Live is an open source project Which builds upon Wagtail to help people craft live blocks easily. So what is a live block? You may certainly have already seen an example of live blocks. A live block is a blog that provides real-time updates on an ongoing event. So for example here we can see a live block from CNN that uh shows uh the Russia in invasion in in Ukraine, which gives live updates. So here we can see that fifty yes, we have five new updates that we can

Speaker 1: See here, so yes, 15 minutes ago we had this. So yes, most live block have this format where you know we want to deliver uh information. You know, the posts are not very long, so they you know they there are timestamped, they have a title. Sometimes the yes they can include images or yes, images or maybe link to YouTube videos. So this is the format of most live vlogs. And yes. So yes, considering this format where it uh you know it allows you to be to you know to give real-time update on something and an ongoing event. The format of live blogging is very is very light is very popular among journalists. So yes, the m must use cases of

Speaker 1: live blogging involves some kind of reporting maybe a new uh yes for example there were there are live blogs for when uh Apple presents a new product that they can we can have live blog about that We can have live blocks about the soccer match, uh any sport match about elections. So there are many type uh many uh use cases of live blocks, but most of them involve some kind of reporting. So yes, so now what uh when I will talk about what I love I will often talk about uh These concepts which are input source, live page, publisher and receivers. So what do they relate to? What do they mean So first the importers. So yes, as we said earlier, most use case of a live block involves some kind of reporting

Speaker 1: So to make yes, Woktail Lab user friendly, we we all we the idea of my mentors was to allow people to post Today I live looked from the messaging app. So this will uh reduce the setup and the learning curve to to to try to start using it. but also use uh cutting edge features that are already proposed by the by most messaging apps APIs like Slack, Telegram or Facebook. So in that logic, the main way to publish a post To a live blog using WorkTel Live is to use a messaging app. So you may use Slack or Telegram or WhatsApp or any other messaging app that yeah. that you like you can use it to post messages and that's what we'll be

Speaker 1: we will go over the steps but yes first uh there is this idea of having an impact service So yes. And the idea la why why we also choose using a messaging app is that, you know, we imagine that a reporter can be on the field. and just use their mobile phone to give updates and that will not involve much hustle. But I would also uh point that We can also use the worktail editor interface as an input service and I will also show you how that can happen. So After the import source, we have receivers. So what are receivers? Receivers there are a bridge between a messaging app like the import source and our live blog. So any message that we post in an input source will be processed by its current corresponding receiver.

Speaker 1: Like yes, a receiver, you know, every input source has its corresponding receiver and the receiver knows. how to try how how to extract r relevant informations from uh from an update received. So Yes, and under the hood, uh most receivers use the webhook technique. So we provide an endpoint to them to a messaging app and whenever an event we specify and By another one we specify, I mean for example, when we post a message or when we edit a message or when we delete a message, the messaging app sends a post request to that endpoint. So yes. So for example, when you pass a message in a Slack channel We will receive a payload that contains all information relevant to the message posted and we will have a Slack receiver that will process that message and will extract

Speaker 1: Who posted it? When was it posted? What is the contents? Are there any links? Are there any images? How should we treat all of that? So that is the goal of the receivers. And after the receivers, once the receivers have done their work, so live posts uh will live live in a live page. So this is maybe our data model. If you are maybe familiar with Django, Django has a concept of model and Worktail has a concept of page in a tree structure. So yes, the life force lives in such a page. And a live page is uh so is kind of mapping it maps to a channel in an input source. So when a message input posted in a messaging app uh receiver uh processes the message and the message is saved on the on the live page

Speaker 1: and and the fourth uh part is publishers So while receivers connect a channel in a messaging app to a live page, publishers make the connection between a live page and web clients who are watching our blog in live The goal is to publish new posts to the client as soon as it's saved. So we received it, the receiver has processed it. has processed the message. The message has been saved in the live page. And then the now the last step is to show these those posts to the users who are seeing the blog in live. So to achieve the data publishers take leverage of different techniques.

Speaker 1: So mostly they use the WebSocket technique And but they could you can you we we also have polling publishers which use long polling and interval polling and yes That's most how mostly how it works. So to sum up, how how how does WorkTail Live work? So now now that we are more familiar with the concept, we can yes describe the machinery as follows. So a user or a reporter. We will post a message in a channel, like in the input source Jason. Here, input source, we mean maybe Slack or Telegram or WhatsApp or maybe even the Woktail admin interface. And when the message is posted, an update is sent to the corresponding receiver, generally via webhook, and which processes the message and extracts

Speaker 1: relevant information from it. so that the message is saved in the corresponding live pages. Earlier I mentioned that a live page maps to a channel. So excuse me. Each channel maps to a live page. So when a message is posted in the channel, it's also processed and saved in the corresponding live page. And then lastly, an update is sent to the web client by a publisher as soon as possible. Excuse me.

Speaker 2: Uh to the end you're muted.

Speaker 1: Excuse me.

Speaker 2: Okay, you're back now.

Speaker 1: Oh. Okay. So now uh the next step that I hope will be The most exciting one is we will make a demo of yes of fact life, how that works. So I will continue to share. Yes, so this is the home page of of the Worktail Life project in GitHub. So yes, we have uh here our documentation. So we have tried to, you know provide a very good documentation, you know, yes, giving a turf overview of what we are proposing, you know, document

Speaker 1: every method in the API and everything. So To expect that if you want to build a live blog with WactaLive, you will just head to this tutorial. And yes, now what I will do is yes, I will just I will not maybe take follow all the steps But I have a local project where I have all this set up and I will look here and and we will check that we have everything that is needed to run our live block. So yes. The the first thing is to install and run Wactel Live. So yes, we need to create a virtual environment, install Wachtel, then install Wachtel Live. And then this will generate our set. This is similar to Django Create project. So this is just a command line to start a new worker

Speaker 1: project. And yes, so inside our website we will have many applications. So we will create a new application named LiveBlock, which will which will yes contain our live block. And then yes, this is these are uh Django specific things. So yes, so we if we want, yes, we need to add WorkTail Live and the app we just created in our install app in our settings file. So here this is this is yes. So here I can see work tell live PVC the settings here. This is our base settings file. So we have added the uh the live blog application and the worktail live application. And then the next step is we can create the the live page model that I mentioned earlier, which is yes, where the live posts live in themselves.

Speaker 1: So Yes, so we im the Woktail Live provides a live page mixing which has all the functionality needed to, you know, to bring le real-time capability to work tail page if I can say so. So here we can ensure in our models. So here is our live block application. So here in the models we can see that we can have Uh here is our live block page. So yes, I have essentially added a hero image. So here's some hero text. This is just for styling purposes. But yes, otherwise, here we have the live page mix-in panels. So the panels are uh I will have I will show you in the worktail admin how this uh maps what this uh relates to.

Speaker 1: Once we have our live block page model, we have to specify also in our settings the path to the model like this. So in our we'll also go in our settings. Oh yes , I'm having trouble with in our settings here. So here we can see I have added the path to the model we just created here, which is the value of the work to live page model setting. And then we will

Speaker 2: think we're seeing your editor. If you're showing an editor.

Speaker 1: Ah, you aren't seeing it.

Speaker 2: We're just seeing the browser still.

Speaker 1: Oh.

Speaker 2: Are you seeing it?

Speaker 1: Okay.

Speaker 2: Just see the browser.

Speaker 1: Okay Okay, I are you seeing it now?

Speaker 2: All right. Much much better now.

Speaker 1: Sorry. So yes, so okay. So I was just mentioning here our models. So this is the live block page. that in here inherits from live page mixing and page and yes I have added hero image hero text for styling and yes this is essentially uh what uh what is described here so are you s seeing the browser too

Speaker 2: Yes, good.

Speaker 1: Cool. All right. So yes, we need to also add a template, which you know, for each page we will use the corresponding template. And in that template, we need to include uh this uh bit of yeah this bit of template from Wactail Live. So here if I go in my templates folder inside live block, live block. html So we can see uh exactly here we are including uh the line specified. So the next thing is to configure we have we want to configure a publisher and then we will have to configure in butters so wactel as I said earlier Wactel Live has different types of publishers we have polling publishers But we also have publishers that use the WebSocket technique. So for example, we have long

Speaker 1: polling, interval polling, and yes, these are all uh WebSocket publishers, but they use different maybe libraries or different setups. For example, Django channels is the is maybe the classic way to add uh WebSocket functionality to a Django project. So this is provided. And for we also have PySocket, which is uh you know, an external service which will handle everything. So When we want to have an update, we send it to the to the maybe to PySocket's API and PySocket will handle delivering it to the client. But we can also use uh WebSocket libraries like WebSocket or Starlet or Tornado , every every library, every framework that you know

Speaker 1: that uh allows using WebSocket can be used to configure publisher Um for this tutorial for this demo I will just use long polling since it's easier to set up. So yes. And yes, the steps to follow to set up a long polling publisher. So for the first thing we have to specify the path that the publisher will use. So we will say WactaLive publishers, polling, long polling publisher. So here in our settings, settings here in base, we can see here that line, Octel Life Publisher is polling, long polling publisher And we can optionally set a polling timeout. So with the default value is 60 seconds, but for this we will just leave it as is. And then we also we need to hook the

Speaker 1: the polling, maybe the JavaScript that the polling will use by including this this bit of code. So We can in my template here. I will open it here. So I'll we can see here that we are including the you know the template here. So we also need to add the URLs from Wachtel Live. So if you know the Wctel Live uh URLs are handled there, so yeah, just to check, yes We are importing the URLs and we are adding them here like this. So yes, now that we have set up our publisher We are all set. That's all all that's all we need to do. We now need to configure an input source and its corresponding receiver.

Speaker 1: So yes. Yeah, so we can choose from Sluck. Currently, Woktail Lab only proposed Sluck and Telegram receivers. So We can we it's possible to add WhatsApp receivers or Facebook receivers or any other messaging app that you know that exposes an API. and can send this update when something happens, we can use that messaging app as an input source. But uh for this for the scope of the project we are current currently we are only providing these two uh input sources. So for this one I will set up the Slack. I have done yes, so the Sluck telegram receiver So yes, the first thing we will need is to create a telegram board.

Speaker 1: So in we'll go in uh if I click here So you can see, yes. So you know the boats have very you know useful things that we can use in in Telegram. They allow to have many types of interactions. So yes, we need we'll create a boat. And once we create the boat, we will have a notarization token that will be generated. So if I go in, I will open it here. Yes. So here, yes, I have the the the t have the tokens. This is just for testing. I will remove the boards, uh no problem if you see the token. So then the next thing we will want to do is to add that

Speaker 1: uh the token in our settings like to specify it like this, Telegram bot token equals the bot token that we just had. So here in my settings we can see that I am saving the but token as a environment variable. So here we have telegram but token equals this. this thing. So yes, now that we yes, we want to receive evidence from telegram when a message is posted or when a message is posted. So in order to do so we need to hook a web hook URL that we will give to telegram and they will use it to send us post requests uh whenever an event we specified occurs. So yes, yeah in in development phase because most uh messaging app require uh a URL which

Speaker 1: started which httpps to yes so in in uh Uh when you in the development phase we can use ngrock. So to have uh yes, uh you know, a URL with A URL with HTTPS. So I will here start in Grock and I will grab here this year URL we start with HTTPS. Copy this and I will specify it as the value of the telegram webhook URL. So I will I will paste it here I'll save here. So yes, we need to allow the f the following in our URLs, which is already done. And now we also need to specify which receiver are we using.

Speaker 1: for the Wcttail Live Receiver settings. So here we see that it's the Wcttail Live that receivers, the Telegram and Telegram webhook receiver. So in our settings here we have the Wcttail Live Reiver, which is exactly the Telegram webhook receiver. So, yes, that's all we need to receive events from Telegram. So, if we run the following, so if we start our development server Just here. So I'm in the WorkTail Live P WC repository. So I can run the server now. So here what happens like So here we can see in the documentation it says if everything went well, the server should should start without errors. So here what has happened is that we have made a request to the telegram API saying them, hey This is our webhook URL we want you to send us uh any new

Speaker 1: notification in this URL. So if that happens, so If we have no errors here, that means that uh telegram has received uh have received our URL and they will send us an update whenever we make some changes. So now, yes, as I mentioned earlier, the general idea is that you know you will post messages in a channel, in a messaging app, and the channel maps to a live page So users viewing the left edge will have will receive updates uh instantly. So if I go in uh in telegram here , yes, I will just Start a chat with my board. So this is a command that is provided by Woktail Live to get the ID of the chat. So we need to have

Speaker 1: the the the the chat ID so we have now the ID which is this and we will use this to uh this will be the ID this will map to a live page so we will need this ID to just to configure our live page. So that was all we need to set up the telegram appropriate receiver. We are almost there. So the next step will be to create our database, create an admin user. Yes, I have already done these steps. And now I think we are good to start live blogging. So now yes, our server yes here also we can see that yes uh we re we sent uh request to uh to uh telegram and they have answered with yes everything is okay so now we can run our local server which is already running

Speaker 1: Yes, and then what we'll do, we'll create a new page of live block page. So I will go here. So yes, this is the Wagtail admin. So this is where you manage pages and all. So here uh what we should do is here we want to add a child page and then we'll add a live block page. This is the model that we implemented earlier in our like the value of the worktail live uh page model. So yes, we'll fill the title, the hero image, like you've seen, and the uh the hero text. Here we will uh put the channel ID, which is the value of the get chat ID that we did earlier in in Telegram and but also in Slack and all we each channel will have an ID that's what we want to put here. And here will

Speaker 1: here will is the stream of live post. So this is what I mentioned earlier, which is a stream field. So maybe, yes, so this stream field is a you know a way to have yeah like you can add multiple things i can add here a live post for example i can add whatever i want and This is how we structure our life post, but we have many flexibility in how we want it. So this is also a reason why maybe Wagtail was a good fit to do this. So for example here to yes uh a live post has an id, the ID of the message, uh that when it was created, when it was modified, and we also have this option to show or to hide uh a live post and it has a content so currently the content of a live post it is a text or image

Speaker 1: or embed video so here you can add text if you want if you want to add image we can add image if you want to add uh a numbered video we can add ember we can do it in any order we want we can reorder duplicate and delay so This was just a demo of how you would set up a live block page. But here, I have already set up here a live block, a live block page. So Yes, the title, I have some hero image to make it nicer, some hero text. So we may want to ensure that we have the same channel ID here in here. So yes. We have the same channel ID, so this is our live blog page. So now we will view this page in live.

Speaker 1: Up yes, so here is our our page. So I have done I'm using this. This is a little styling. So now what I'm going to do is up, I'm assuming This the client is on the right, you know, and is watching our live blog, and the reporter is on the left on the on the channel, and they are they want to report uh some event. So What happens is that we can now say title some content for the demo. And yes, here we see the message comes here. So we can maybe add an image. So in in in Telegram, yes, it just the idea is like you can add a

Speaker 1: You can add uh yes, you can add uh a caption, that is how you can call it. Uh It is it is cool. So I can also send this and uh put my wifi Is not yep. So now that uh it's sent, we see it's here. The message coming here PVC is cool. I can edit maybe this uh The message PVC is cool. So I can add maybe here check out Lip Tel Live. Yes, it's it it appears here too. So I can also you know share a link a YouTube link, for example

Speaker 1: Here is a talk from my mentor talking about what what live and then I will pass it here. Yes and here it comes. Here is the talk. So Yes, that's the idea. Yeah, I I hope you are getting it. You post your messages in uh messaging app and they will appear as soon as possible in the user's uh device. So yes, as I show has shown earlier, we can edit messages. Um for Sluck if you delete a message it will also be deleted here. But here if I for example if I delete this message here in in here

Speaker 1: Like nothing will happen here because Telegram, that's how that's how Telegram works. Telegram doesn't send an update when when you delete a message. So What we can do if we want to delay this, we can go to in the admin interface here. So we will do it where we are managing our page. So that's what I was mentioning earlier, that we can use input uh we can use input sources, but we can also use the the uh the admin uh the admin interface to make changes so here if i do a change in the left force here it will appear instantanely here so if i for example If I say that now I want to you know to hide this dispos, not just we just want to hide it and I publish this page

Speaker 1: Uh I uh okay. Uh what have I hidden Oh, okay. So I've hidden the first post so the it's already gone. So to confirm everything is okay, I will Yes, this is the last post. Yes, which is here. Here's a talk from my mentor. This is the one we want to hide. So if I can click the show, I click publish. Yes, it's gone. So Yes, as I said, yes, we can use the admin interface. So if you if you want, you can use this. If you want, if you prefer using messaging apps, you can use messaging apps. So here I can you know del delete everything and maybe just reshow this one.

Speaker 1: And yes, or all or the post are gone, we are we are uh we only see this. So this was the demo. This yes, we were using uh telegram as an input service and we are using a long polling publisher. So you can mix any type of setting you want. You can use Slack receiver, you can use WebSocket publishers. It just depends on your needs and yes, how you want to set up and everything. So that was for the demo. I hope that you you liked uh you liked it. You liked our work tail live demo and yes So now what maybe what what is the future of what

Speaker 1: tele life? So now we We are always and we'll be always looking at ways to improve it. So a future we are currently working on, and we hope to ship it soon, is to give users the ability to upload videos So because currently if you notice we we can only upload images and embed videos so we want to provide users the ability to also pro uh post videos and And the desire of making Woktail Live async capable has recently emerged in in discussions, but yes, this is a complex task that we will have to think about carefully before doing anything but just really something and yes the way Wactail Live works and how the receivers which make requests to the API the publishers

Speaker 1: this is you know a very good fit to use asynchronous programming. But yes, mixing async and sync and yeah. But that's something we are we are thinking about. And yes, we also hope to find more users, more contributors to let us know what they think of Hotel Life and help us improve it And yeah, so if you have any idea you'd like to share, any improvement or whatever or question to ask. You can surely get in touch in the one-to-one lately, but we you can get in touch with us in the WorkTech Life channel in Slack And if you want to try Octel Live in one minute, uh my mentor Kern has gracefully set up a Gitpad environment which allows you to live block

Speaker 1: easily. I will share the link. Soon. So the yes, the ide is that you just click uh yes, you just click uh open in Gitpod And then you will log in with some pre-created credentials and then you will be able to do some live logging. This will give you a feel of how uh what your life works. So yes, I think that's mostly it. I hope that you like the presentation. And yes, thank you very much.