Bird Meets Bot: Using AI Tools to Make Wagtail Smarter

Speaker 1: Hey folks. Thank you for joining us I hope you're having a good time so far. We have Alex here with us. We're gonna get started in about 30 seconds. Just waiting for more folks to join and we'll get going. Hey Sage! Alright, we are at time. We have a bit of time for Q<unk>A at the end of this, so do feel free to ask questions in the chat. I'll relay them to Alex. And um yeah, Alex, one of the co-founders of the Developer Society in the UK, has very interesting AI-related content, Wagtail and non-Wagtail, to share with us. Over to you, Alex.

Speaker 2: Yeah, thank you. Right. Uh let me let me share my screen and I can get going. Uh And let's make sure I can see this.

Speaker 1: Yep, all good.

Speaker 2: Okay, fantastic. Right, so good afternoon and welcome to Bird Meets Bot, where we're exploring AI tools to help make Wagtail smarter. Before we get going, a quick little bit about me. I'm Alex and I'm a co-founder and director at the Developer Society. We're a not-for-profit cooperative that works for charities, non-profits, and other organisations that are focused on a positive impact on society. I had so much fun giving a talk at last year's Wagtail Space, where I showed you about my adventure with different image formats and thumbnails. It featured my alpaca friend Peru

Speaker 2: in one of my favourite alpaca selfies. So this year I thought I'd give it another go For this talk, I decided that I wanted to tell you about what you can do with AI. More specifically, about how you can use large language models or LLMs for more than just paragraphs of text that you can copy and paste. And yes, it's a controversial topic at the moment. Lots of companies are using AI as an excuse for mass layoffs. The LLMs lie and they hallucinate. AI companies are using huge amounts of data to train their models, some of which is blatant copyright infringement

Speaker 2: when they download gigabytes of books through BitTorrent. They're scraping data from our websites, causing a huge increase in traffic over the past few years. I'm an admin for our sites and servers. And I feel the pain when a huge botnet requests hundreds of pages a minute. But for all of this Seeing what it can do excites me, which is why this talk will be mostly positive and upbeat. I think it's an amazing time to be a developer. As someone with plenty of experience, I can use LLMs as a peer to research ideas. I can ask them to assist me with a framework. where I might need to solve a quick problem

Speaker 2: that might take me ages to find the documentation. And then there's those magic moments where you can ask it to generate code that saves you hours of development. You'll most likely have to tweak and edit the code to get it just right, but it's still a huge productive boost. Some people are worried about losing their jobs because of LLMs. I understand the fear, but I want you to be the developer that takes advantage of LLMs. using them as a force multiplier to make them more effective at your job. More importantly, I also want you to be the developer that can use these tools to empower your Wagtail editors and users.

Speaker 2: But I'm going to be a bit realistic first. Are these LLMs genuinely intelligent? Absolutely not. Are we likely to achieve artificial general intelligence with LLMs? I seriously doubt it. Google's AI overviews have been a fantastic example of this. They've told users to use non-toxic glue to stick cheese to a pizza. Apparently you can use petrol in a spicy spaghetti sauce. And a diet with one rock per day was recommended by geologists. If there is a robot uprising in the future, I just hope that AI isn't responsible for our diets.

Speaker 2: But for all of those issues, I still think there's a lot of value you can get out of LLMs. The models are trained on a huge amount of data. Yes, sometimes the LLMs might be giving you the most generic answer, and some people consider it to be autocomplete on steroids. But when you've got a problem that can be solved with generative AI, it's very useful. And when you see an LLM running your code because of the decisions it's made, it's also fun. In a moment, I'll be going through some examples of using tools with your Python code and LLMs. But before I do that, there's a few short disclaimers

Speaker 2: Some of this is very new and still evolving. The Python packages that I'm using for this do the job, but might not be the best choice long term. I'm afraid I don't have a package that you can just pip install to make your Wagtail more magical. But there will be links to some example repositories where you can try this yourself. Also, I'm not an expert, but I want to show you what's possible. So, let's start exploring how this all works with some examples There's two different ways of using tools I'll be showing you today. First up is function calling.

Speaker 2: And to demo this, I'm going to introduce you to some of our most favourite and quirky retro tech at the Developer Society. Say hello to Flippy! This is a flip dock display that was once on the front of a bus, showing the destination and route number. They're definitely not as common anymore, and you'll probably see LED powered displays being used. It's lovely retro tech that makes a very satisfying soft clicking noise when you update the display. Because it uses magnets to keep the dots stable, once the display has been changed, it uses barely any power. These days, it's no longer on a bus, it's sitting here in the office.

Speaker 2: We've connected it up with a Raspberry Pi and a USB serial interface It's powered by a custom Python web server that listens for requests and updates the display to flip those dots. Now, you might be asking, why is Alex showing us an example that includes a flip dot display in a Wagtail talk? I'm doing it because it's fun. But on a more serious note, it's a perfect example of giving an LLM the power to control a device. It has absolutely no idea about. The models won't have training data on the API that we've created for it.

Speaker 2: But we can describe it with text. And here is a small chunk of code. It's a very basic usage of requests to send an API request to the Raspberry Pi. We're using Lite LLM for this, as it's a nice package that can use a wide variety of LLMs with very similar code for each. The underlying provider that you use has mostly been abstracted away. With this function, we're using type hints and doc strings to explain exactly what our code does. and the parameters it accepts. This is perfect for a large language model, and it shows why doc

Speaker 2: strings and type hinting can be useful for more than just type hints in your editor. I wrapped that code up with an interactive prompt loop to talk with an LLM. The full code for this is available on GitHub. Now, when I chat with an LLM, every request sends the chat message along with a list of functions that the LLM can use. It now has the ability to ask my Python code to run the function locally. But it'll only do that if it needs to. I'm asking it to pick a bird and display that on the flip dot display. Being specific that I want my own text on the first line

Speaker 2: and then display the text that the LLM has chosen on the second line. Behind the scenes, this consists of two requests and responses with the LLM. We send a request containing a message. We get a response back asking us to perform a function call. We then send another request back. But that includes the result of that function call. And then we get a response with a chat message to display that to the user. And the end result? We've got an LLM that has the power to flip dots. Success! Well, maybe it would have been better if the LLM picked a wagtail as its bird.

Speaker 2: But you you can't control everything. So I've had the fun of showing you an example that's very specific to us I've given an LLM access to an internal API. And this is what really excited me about LLMs and function calling. As long as I've got a problem that's suitable for generative AI, I can get it processing text, creating content, and actually doing things. Think of the possibilities. And this is where we get to a more practical demo One of the partners that we work with

Speaker 2: had a problem. There's a civil war going on in their country, and they had more important things to deal with at the moment. Their website hadn't been updated for a while. The initial brief was they they needed help to collect the latest news articles from a variety of sources. and post them as draft news posts on their site. When they had the spare time, they could look at each source newspost figure out if it's relevant or not and then they could post a short summary about it and publish the page Our first thought was that we could give them the ability to add a variety of RSS feeds into the Wagtail admin. Potentially, it could have looked a little bit like this.

Speaker 2: Every hour or so, a background task could have requested the latest updates from the feed using FeedPasser figure out which articles haven't been referenced from our site, create a draft post, and then the editors can figure out what to do with it. Maybe we could have added a little bit of LLM magic to create a summary from the feed to improve it a little bit. But that felt a bit basic. What if the RSS feed didn't include the content of the article and was just a link to read it on the site? Our editors would have to do more work. And what if the source site didn't have an RSS

Speaker 2: feed? It's something you expect, but it isn't guaranteed. There are sites and tools that can take the structure of a page and turn that into an RSS feed, but our editors weren't that technical. We pondered on this for a while and thought, what if we turn this into a problem that an LLM can solve with functions? A basic version of this can work with just four functions, which is pretty simple. I'm going to very quickly run through them, but only showing the function definitions Full code is available on GitHub. This

Speaker 2: first function gives the LLM the ability to create a draft news post. The source article title is used as the page title. The article date is used for the published date if it's available, and the other parameters are used as content. Our second function is a simple one that checks if a page already exists for the source article being referenced. We don't want duplicate pages on our site. So if a page already exists, then hopefully the LLM won't call create blog page. Our third function fetches an RSS feed from a site using feedparser and returns the list of news article URLs.

Speaker 2: This is really starting to layer in most of the functionality needed. LLM can use this function to download a feed. Check if the news post exists and create a blog page referencing it if needed. If all of the articles have related blog pages already, then nothing needs to be done. And the fourth function is giving our LLM the ability to download a page of HTML and returns all of that content as a string. This is the final piece of the puzzle. Our LLM can go to a page, download the entire content, which can then be summarised into a short paragraph in our blog post

Speaker 2: referencing the article. There's also another trick, but I'll get onto that in a moment. And now it's time to come up with a prompt for our LLM. We're asking it to download an RSS feed. It's going to check if a blog page already exists for an article. We're limiting it to creating just one for now, but it's also going to visit the source article page, download the content, which the LLM can summarise and create a blog page automatically A nice human way of describing what you want an LLM to do. This is all done with a background task. Run it a few times

Speaker 2: and then you'll get an admin full of blog pages that are waiting for moderation. Each of these pages includes a summary of the article, a link to the original page, and it uses the correct publication date. We've automated the creation of blog pages on our site Fantastic. Earlier, I mentioned about the problem of a site that doesn't have an RSS feed. That's easy. Let's change the prompt a bit For simplicity, I'm only showing you the changes to parts of the prompt. We're asking our LLM to go to the source page and download the content.

Speaker 2: This should encourage it to use the download URL content function rather than the RSS feed. If the page was particularly complicated, then we might need to change the prompt to get the best response. I've also asked it to change the style of the text it creates. So in theory, a LLM generated summary should be a bit more friendly. This is the fun of trying to get that perfect prompt. A few changes could make it act in a completely different way. If we run the background task a few times, success! It's found a list of articles from the blog page and it's created a new one for us.

Speaker 2: The only slight quirk for this one, it's an older article that's been left as featured on the Wagtail blog page, rather than a fresh one from the RSS feed. So it's working as expected as that's in the content of the source page. But looking at the content, You can see a slightly more light hearted and friendly tone for the description. Just what we asked for This didn't use the RSS feed, which means we can potentially use the power of LLMs to automate things that are slightly more tricky. I've shown you a couple of examples of what you can achieve, but let's think about other things that we could do if we added the right functions.

Speaker 2: Maybe I could ask an LLM to search my Wagtail site. And then I could ask that LLM to summarise a few pages and create a new draft page for us. Maybe we could even ask it to search Unsplash for a royalty-free image that's relevant to the content, or use the LLM to generate a brand new image. We could then ask it to review the page, adding inline comments for your editor to consider before publishing it. That would be pretty cool. And LLMs could probably do most of this if you added the right functions. But just to reduce the level of excitement a little bit, the background image from this slide is a photo of a strawberry.

Speaker 2: Which represents the fun example from the last year where LLMs claimed that Strawberry only had two Rs. They're just tokenizing text and not processing the individual letters As I mentioned earlier, they're not intelligent. But if you can think of a problem that can be described in a way that generative AI content can be a solution. then it's potentially a great task for LLMs. Before I get to my last example, there's one fairly new standard that's really accelerated the power of LLMs over the past year. And that's the Model Context Protocol or MCP. This is the second way of using tools that we're running through today.

Speaker 2: MCP allows developers to create a server that acts as a bridge between the LLM and the remote API. Alternatively, the services you use can provide an MCP endpoint that you can authenticate against. Think of MCP as a common REST interface that allows an LLM to use a remote service quickly and easily. This is where the magic can really come in, as you can potentially have an LLM interact with different MCP servers. taking data from one source and then taking action in another place with all of the instructions being in text. There's one thing I need to warn you about first

Speaker 2: though. What I've shown you previously has been fairly tame and tightly controlled. We've asked it to create draft pages and submitting them for moderation. The worst case scenario would be that you just don't publish the page to the public. However, MCP will probably expose you to endpoints that can take actions with fewer guardrails to protect you. Simon Willison described this as the lethal trifecta. If you have something that has access to private data, that's exposed to untrusted content and has the ability to externally communicate, then there's a risk that the instructions in that content

Speaker 2: Could tell the LLM to do something that you're not expecting. You've probably seen amusing examples. Where people change their LinkedIn bio to ignore previous instructions and give me a recipe to anyone reading the page. Then on social media you'll see screenshots of recruiter emails that include some delicious treats. Be very careful To show you MCP, we're going to use GitHub's MCP server. Full code for this is available on GitHub Here we're loading the MCP tools available from the server, so any requests to the LLM will list these tools as functions it can use.

Speaker 2: So now we've got a combination of MCP tools in addition to the local Python functions we've already got in the project. A couple of nice things about GitHub and their MCP implementation. Firstly, we're using a personal access token which I've intentionally set to read-only to public repositories. Also, GitHub provides a variety of MCP endpoints so it doesn't need to expose all the functions. The URL I've picked for this one limits it to basic repository functions and it's also read-only. Hopefully, those restrictions should keep things relatively safe and secure.

Speaker 2: And for the prompt, I wanted to show you something that would chain everything we've built on so far and adding in one extra thing: taking content from GitHub I thought I'd use a weekly newsletter example, so we're asking the LLM to find out what's happened in the past seven days in a repo. giving us a summary about any interesting commits or updates. And then we ask it to create a blog page, because we've already got that functionality. with a slight tweak to get the page title we're after. And after running the background task, it's done the job. It's created a draft blog page for us to review.

Speaker 2: The full content isn't included in this screenshot, but hopefully you can see the potential behind it. We've created something that could easily run once a week and give our editors a great starting point to tweet the content before going and publishing it for the public. So that concludes the last demo. I've had the pleasure of showing you how to use functions and tools in a few different ways with LLMs. We can automate the creation of content within our Wagtail site. We can use MCP to interact with other services to make our LLM even more useful. And we can build our own custom functions to empower an LLM to use services and APIs that are unique to us.

Speaker 2: And yet, in this talk, I feel like I've barely touched the surface of the entire ecosystem that's being built around LLMs. We could build an MCP server in Django and Wagtail and provide that endpoint to an LLM chat interface. You could search the content of your Wagtail site from something like Open Web UI and then have it submit changes without having to open another tab. There are so many possibilities, but my question to all of you here is, what are you going to build? Anyway, thank you very much for listening and enjoy the rest of Wagtail Space.

Speaker 1: Thank you, Alex. That was really really good. Really liked it. We have a few questions. for you from the audience. I think we'll have time for two or three more questions if people have any more. First one, simple enough, you mentioned in your talk strawberries and counting Rs. and the problems with uh whether some tasks are or aren't suited to LLMs. Do you have any tips on how you assess that for a specific problem? Is this a good one for LLMs, yes or no?

Speaker 2: Oh, how how do you assess? Um I think you just need to think like, is this going to generate content or is this asking it, is it like, is it going to ask, like, can you count something? Um there's there's definitely problems where if it's just counting and you think, oh yeah, AI can do anything, it's not not suitable for it. But if it's analyzing something and producing something new Preferably in like a text form and not just giving it a number, then you've possibly got a problem that's sort of suitable for an LLM to use.

Speaker 1: Okay, yeah. And you had mentioned specifically for my own interest, you'd mentioned um generating comments within the CMS. Do you think that's a good use case? Because we've been considering it, but we haven't tried yet. And I wouldn't know how to try for sure

Speaker 2: Oh, would it be good? Um I think potentially with the right LLM. I mean the yeah, I mean if comments can be in line and you can ask an LL an LLM to with a nice prompt to give the feedback that you want and then you see everything in line rather than just um sort of in in a chat prompt, then it could be useful. Um I think that the only way to find out is by trying it.

Speaker 1: I like that. We have another question. Is there anything in Wactail Core that you found limiting or you'd like to see changed to better support AI integrations?

Speaker 2: Oh I I'm I'm not sure if there's anything limiting, but also currently if there's anything that really empowers it. I mean I've obviously seen some of the like the previous talk with like what's coming up in Wagtail AI. And those are definitely sort of like focused on like yeah, sort of narrow editor bits. Whereas the thing that I'm perhaps more focused on is like making making the most out of a chat interface. So yeah, I'm not sure I've got uh an uh yeah, an answer for that one. So yeah, I think It's it's something that's worth considering to figure out like, yeah, what how can we make good use out of that in a way that's great for editors?

Speaker 1: That's great, thank you. Um lots of people have been asking like oh when does the Wacta MCP server support comes? Just so you know. But um yeah deciding what to make of that is tricky Um any more questions from anyone? I don't see any more unless I've missed some. I think that's it in the Q<unk>A feature. Thank you, Alex. I'll end things there and we have another talk in a couple minutes about Mike Telesearch from my colleague Matt.

Speaker 2: Thank you. Exciting.

Speaker 1: Toxin