Dynamic static sites with Django and Sphinx | Carlton Gibson

This video features Carlton Gibson at DjangoCon Europe 2021 in Online.

Dynamic static sites with Django and Sphinx | Carlton Gibson
0:36:43
Published June 27, 2021
1,748 views

On a content site these days, especially on a personal project, or where the
team are technical, you often don't want the limitations and complexity of full
CMS. No, you want to use familiar tools: you want to write in Markdown and
manage your content in git.

That doesn't necessarily fit the traditional Django workflow. You end up
looking at a static site. This is fine but you loose the ability of Django to
serve dynamic pages. You have to decide which of the numerous generators
you're going to use, or whether indeed you'll roll your own. It's a bit of a
mess.

You can have the best of all worlds though. Sphinx, yes the
documentation builder, is a super capable static site builder. You can
use it to build your content. Yes it can handle Markdown. Then you can
use Django to serve that content, given auth, forms, interactivity,
custom content, and all the rest of it.

I'm going to show you how.

The talk has two parts: Sphinx and then Django.

  • First we'll introduce using Sphinx to manage and build your content.
  • We'll look at project setup so you can use Markdown for files.
  • We'll organise those on the file system, and use git to manage them, rather
    than a full-blown CMS.
  • We'll push to GitHub and use a GitHub Action to build on each commit.
  • Then we'll setup Django to serve our Sphinx built content.
  • We'll use the DTL to get control over our templates.
  • And we'll add auth, to provide dynamic per-user content.

From there, the sky's the limit. Content management never looked so rosy.

Summary

Carlton Gibson shows how Sphinx can manage content for Django sites while preserving a file-based, Git-friendly writing workflow. He explains how MyST lets authors use Markdown with Sphinx’s cross-references, directives, and other features, then demonstrates Django Sphinx View, which consumes Sphinx’s JSON output and renders it through Django templates. The result combines Sphinx’s documentation tooling with Django’s authentication, permissions, templates, and deployment workflows, while remaining lightweight; he notes that a traditional CMS is still more suitable for non-technical editors.

Key takeaways

  • Sphinx can build ordinary static sites as well as documentation, and its JSON builder exposes rendered content for use by another application.
  • MyST Parser allows Markdown files in Sphinx projects while retaining features such as cross-linking, admonitions, and directives.
  • Django Sphinx View maps URL paths to Sphinx-generated JSON and injects the resulting content into a Django base template.
  • Because the view is a Django TemplateView subclass, normal authentication, permission checks, and custom context data can be added.
  • GitHub Actions can build the documentation and upload it to a server automatically whenever changes reach the main branch.
  • The approach suits technical and extended writing, but browser-based CMS editing remains better for non-technical users.

Summarised automatically from the transcript.

Chapters

  1. 0:00 Introduction and Project Context Carlton Gibson introduces himself, the Django Fellowship, and the motivation behind Django Sphinx View.
  2. 1:42 Sphinx for Static Sites The talk explains how Sphinx builds static HTML sites and why its documentation features are useful beyond docs.
  3. 3:14 Markdown and MyST MyST is introduced as a Markdown parser for Sphinx that preserves Sphinx features such as cross-references and directives.
  4. 7:11 Dynamic Content Workflows The discussion moves from static sites to authentication, interactivity, CMS choices, Git, and file-based editorial workflows.
  5. 9:31 Sphinx’s JSON Builder Sphinx builders are reviewed, with particular focus on the JSON builder used to feed rendered content into Django.
  6. 11:07 Django Sphinx View The speaker demonstrates installing Django Sphinx View and routing Sphinx-generated JSON through a Django URL configuration.
  7. 13:25 Customization and Access Control The talk covers custom templates, authentication, permissions, and adding application-specific context to documentation views.
  8. 14:59 Continuous Deployment A GitHub Actions workflow is shown for building Sphinx content as JSON and deploying it with rsync.
  9. 17:31 Roadmap and Project Scope The speaker outlines the possible template bridge, explains the project’s focus on Django integration, and rules out bundling themes.
  10. 18:59 Questions and Discussion The Q&A covers personal-site use, CMS integration, nontechnical authors, images, interactive layouts, and testing.

Transcript

6,824 words · auto-generated Show

Automatically transcribed, so expect mistakes in names and technical terms.

0:08

Speaker 1: Hello. Good morning. Welcome. Thank you for joining me. It's nice and early, so I guess I'm going to go easy on you. My name is Carlton Gibson. I'm one of your friendly Django fellows and I help maintain various packages in the Django ecosystem on top of that. I'm Carlton Gibson on GitHub and Twitter, and you can you can find me there. I'll just say a word about the Django Fellowship Program. There are currently two fellows, is Maris and I, and we're contracted by the Django Software Foundation, the DSF, to do the day-to-day running of the framework. We handle trick uh ticket triage, we do patch review, we handle security of issues, we do the releases, that kind of thing. We're having awesome contributors, but it's the kind of stuff that on a project side of Django's, it just it just wouldn't get done. So If you depend on Django, or if your company depends on Django, you should definitely make sure you have an individual or better yet, a corporate sponsorship of the DSF.

0:55

Speaker 1: You make a small contribution to help ensure the sustainability of the framework that you built your business on Having said that, when I'm not on GitHub, I do a podcast with my co-host Will Vinson called Django Chat. Essentially, we have various guests from the community and we chat about Django. We're coming up for a hundred episodes there. So if you haven't already, grab a listen of that. And finally here, then over the last little while I've been not so secretly pottering on a little side project called Button. It's uh at btn. dev. They tell me that no vowels are cool. So BTN. The tagline for button is a simple, simpler deployment story. It's how you get your Django app online if you're not and perhaps you have absolutely no desire to be an infrastructure engineer. That's coming this summer, so go and drop your email in the box there if that's of interest to you. Right, talk today is about dynamic static sites with Django and Sphinx.

1:42

Speaker 1: It's about how you can use Sphinx to manage certainly part of part of the content of your Django-based website. We're at DjangoCon, right? So I can probably let the Django bit go for a moment. But Sphinx, the documentary station builder, the answer is yes. Like we know Sphinx is a great tool for building docs, right? The Django docs, the Python docs, they're built with Sphinx. And all that awesome cross-linking you get in the Django docs that take you from right where you are to right where you be need to be, that's all managed by Sphinx, right? You simply can't maintain hyperlics like that at any scale. But Sphinx, it lets you define targets, it keeps track of them for you, and it lets you know if any of them are broken, and then it compiles them all up with the right URL when you build your docs. So Sphinx is basically what makes documentations like Django's or like Pythons possible. Right.

2:27

Speaker 1: And just a quick aside, like crosslinking like that, one of the best features of Sphinx, is only one of its super powerful features. So we know that Sphinx is great. But Sphinx is also great for building static sites. When you install Sphinx, you get a command, Sphinx quick start, and you run that and it answers your question, what's your project called, et cetera. And then it gives you a make file. And you run that make file, you run makehtml, and what it does is it builds a folder full of HTML files. It builds a static site, right? You point your web server at that and your job done. So given that it's just building a stock static site of docs, you can use that to build your website as well. Now I don't want to spend too long talking about using Sphinx, but um for s for building static sites because this is Django Tom. We want to talk about we want to talk about Django.

3:14

Speaker 1: But Paul Everett of JetBrains has done an awesome job of promoting Sphinx for building static sites. He's got a talk that he's given a few times and you can find it online. He's given it a PyCon APAC 2020 and just recently at PyCon US 2021. And there's also a tutorial on the PyCharm web child website about using Sphinx to build your static site. So rather meet than me go into all that detail and do it not as well as Paul, you can go and check out those resources. One thing I do want to talk about though before we go on is markdown. Right. Because if I were to say to you that, you know, you should use Sphinx to build your your website, then I'm going to use lose 70% of you instantly because you're going to say something like, but Sphinx uses restructured text and I can't use that. I have to use Markdown. Now, maybe you wouldn't, you personally wouldn't say that, but history shows that probably a plurality of of

4:04

Speaker 1: Python users would. Markdown and RST are almost identical for me. You write your paragraphs, you write your headers. They're a bit teeny bit different, but okay, they're basically the same. But it's the links, right? With markdown, nobody can remember the sync tents. Is it square brackets followed by round brackets? Is it round brackets followed by square brackets? Nobody knows. And then Restructured text has got its own version of links. They've both got inline links where you put the the URL next to the the the anchor text And they've both got these reference links where you define a name and then you put that down. That's a bit more effort, but it's probably easier to maintain. But then restructured text has got these crazy anonymous versions, which just blow everyone's mind. It's too much, right? People are averse to restructured tests. Now I think it's worth learning both, but just as a matter of historical record, the need to use restructured text, I think, has clearly put people off using Sphinx.

4:55

Speaker 1: So we'd rather spend time learning another static site builder or switching between static site builders looking for the features or even writing our own static site builder, which is a great, you know. always a bit of fun, right? Rather than learning this extra restructured text syntax. Now that seems kind of silly, but it's the reality of how we feel about it as developers. You know, I've been there, you've been there. If only, if only we could somehow use Markdown whilst continuing to get the power of swings. Well, we can The wonderful people at the a project called the Executable Book Project, and I'm just going to look that up later on. They've developed a pass called MIST, which is a Markdown Parser for Sphinx.

5:40

Speaker 1: It's at mistparser. readthedocs. com. I'll put a link up in a minute Mist does two things, right? First of all, it passes markdown, not restructure test, but markdown into the internal representation used by Sphinx. And then it adds extensions to Markdown, which are all basically variants of the fenced code block syntax that you'll be familiar with from GitHub. for all the extra roles and directives that give links it it its power so that you can do the cross-linking and the admonitions and the call-outs and so on. The promise with Mist is that you can use Markdown and still have all the power of Sphinx. And it works. And it's awesome. So mist is really easy to use. First of all, you just install it from PyPI,

6:26

Speaker 1: pip install mist parser. And then in your Sphinx configuration, you just add the MistPasser extension. Look at that invisible water. And once you've enabled that extension, you can then use restructure test and markdown files. Or, you know, either or. You can mix and match if you want to, but you can use Markdown with your Sphinx, um, with your Sphinx project. So with Mist in play, you can use Sphinx to build your static site. There's no reason not to, right? It's it Sphinx is a truly powerful static site building you should go and check out those videos of Paul's to um to look into the full details of that

7:11

Speaker 1: but Um, this is DjangoCon. We're interested in Django. What about dynamic science, right? What happens if I need, say, authentication or other cool things like per user customized customization or forms or you know interactivity in general. What am I going to do? Put it in an iframe. No, I'm not. Right. So at that point you start going away from static sites and you start building your Django project and you start throwing stuff in the database. And then that leads you inevitably, well, you think, well, what about a CMS? Because I don't want to build all this content management myself. I need a system. And Django has plenty of options there. It's got the big hitters like Wagtail. And Django CMS and then more late lightweight options like find CMS, which perhaps stick closer to you know raw Django. They

7:56

Speaker 1: those are all great. They're all brilliant. CMS is a wonderful, right? But I might be sitting there using the fancy editor in the browser and I might be thinking, you know, but uh what about my editor? I like my editor. I I I I'm used to my editor. I've got the color scheme just, you know, just so I'm not sure I really like editing in this browser window, however fancy that might be. I might be thinking, well, what about Git? I like Git. Sometimes it's overly complex, right? We'd we'll grant that. But the history, the diffs, the power of version control, pull requests, like all of these things, that's not something I want to give up. I might be thinking, but what about Sphinx? What about all its goodness? What about that cross-linking, the admonition, the tables of contents, all the rest of it? I really wanted that power.

8:44

Speaker 1: for a certain kind of audience, for a certain kind of work. And what's the kind of work? It's like anything that involves extended writing, right? The For that kind of workflow, the file-based version control, lightweight markup language-based workflow, it's pretty much unbeatable. Now don't get me wrong, there are many people, many tasks, many workplaces for which a CMS is clearly the right way to go. But where we do want that kind of static site workflow. Wouldn't it be nice if we could leverage the power of Sphinx for creating and compiling our content? but somehow render that into a machine readable format that we could lend leverage in our Django application. Well, of course we can Sphinx has this concept of builders, right?

9:31

Speaker 1: So a builder takes the in-memory content of your site and it renders it into a specific comp uh format. So when you type make HTML using the make file, it's the HTML builder that's responsible for the final output of your files. But Sphinx has many builders. There are builders for EPUP, there are builders for PDF, there are buildings for builders for Unix man pages, for Apple helpics, for for no sequ for for Everything, right? This is another awesome feature of Sphinx that, you know, not only can you build your static site or your your documentation, you can output an e EPUB 2, right? But crucially for us, there's a builder for JSON, the JSON builder that that renders each page of each um of the page sections as HTML, but it wraps those into a JSON structure that you can then access the parts of

10:19

Speaker 1: So this is exactly what the Django docs do, right? We write the Django docs in restructured text using Sphinx. Then they're rendered into HTML in JSON using the JSON builder. And when you request a page, Django then loads that the correct JSON and renders that using the Django templating language. And for me, that's the exact workflow I'm looking for. There's no way that I'm going to go back to not using Sphinx. But at the same time, I've got to have Django. So when I come to think, oh, okay, I'll start putting some more content onto my own personal side. Or when I want to work on the guide component of the button, it's this pattern that I want to leave leverage. Okay, now this results in a package sort of called Django Sphinx View. It lets you serve your Sphinx-built dogs.

11:07

Speaker 1: with Django. It's still an early version, but it's really small in scope and I'm actively using it. So I'm going to say, yeah, come along, join in, use it. Let me walk you through it. The first thing you do with Django Sphinx 3 is you install it from PIP. So pip installed Django Sphinx 3, easy peasy. Then you add it to your installed apps for template discovery. That's the setup, pretty standard stuff And the basic usage is just to import the documentation view class from SphinxView into your URL conf and then route the view. So this is would be in your URL conf. Let's zoom into that middle section because it's a bit um It's a bit small on the on the um on the slide. Look at the path here. The docs, the docs bit there, the docs bit of the string there, that's the prefix you want to root the docs under.

11:54

Speaker 1: So that's like the folder you want to put them in. And then you use the this path part, this is a mouthful, you're defining a path keyword argument for the view called path using the path key uh path converter. That's there's too many paths there, right? So we We have a keyword arg for the path. We use the path path converter to capture everything after the docs that comes in the URL. And then the documentation view uses that path to know which is the correct JSON document to serve. The JSON builder argument here is that is a path to where you've put your output from the Sphinx builder. So I have one of the, I put this in a setting and I have one for local and one for production. Local is just the build deer inside my

12:40

Speaker 1: my development project and the loc production is just wherever I put it on the server The base template name is how you tie the Sphinx docs into your site's design. You need to provide a base template with title, doc, table of content blocks, and so on. And Django Sphinx View then has its own template that extends your base template and it injects the Sphinx rendered content. Also here, you could pass a template name if you wanted to customate that last bit, if you just wanted to use your own template entirely. So have a look. That then serves. That's it. Hey, press the way. This is Django Sphinx View serving its own docs. That's the basics

13:25

Speaker 1: So customizing. If you want to customize this, what would you do? Um Django's Django Sphinx View's documentation view is a Django template view subclass. So you can customize this as you would any Django view. And there's two things that come to mind that you might want to do. One is auth. For buttons guide component, I want to restrict access to logged in use. So I subclass and I add the usual access mix-in-based stuff. Just authentic it require login, right? I want to have sections. Maybe I want to have sections. I'm not sure. But I might want to have sections which aren't available to free accounts. They're only, you know, if only if you've actually paid, can you access those? So in that case, I'd want to use user mark, user passes tests. And you might want to put up, say, docs which are only for staff, or you might want to have members of a particular group that can access these docs.

14:11

Speaker 1: And so all of that can be done with Django's normal access controls. And then the the other clear thing you might want to do for customization is if your base template has got other things going on. So you might have, I don't know, profile data or a news feed or I don't know anything. What is it that your site adds in to almost every view? Well, you can put that in. And for that, because documentation view is a template view subclass, you can override get context data, call super, and that will give you the doc with the JSON populate, the the the doc with the JSON data populated in the context, and then you can add whatever X extras you need. And then though I think those two extension points really cover the the the main case is kind of authentication and then custom extras in your template.

14:59

Speaker 1: So let's talk about continuous deployment, right? Because we might as well add that. And this is this is where I think This kind of a probe really shines, right? There are a million ways you could do this, but the basic outline is you build the docs and then you put them online. So We're using Git, so let's just use GitHub Actions. I'll show you some snippets from the workflow file. You can check out the whole thing on the repo that's there, and you can just you know adapt that to your own need. So first of all, we build the docs. Let's have a look. So these are just two of the steps from the workflow. As I say, it's um in the repo, the full file is is in the workflow file there. You can just look at it. But we install the dependencies, pip, install, upgrade pip. I only I do that because I don't like it when Pip shouts at me if it's if it's not updated. So we'll update Pip. And then we install Sphinx and the all-important missed parser.

15:45

Speaker 1: Okay. And then we just from the docs diet, we just use the make file to make the JSON. And that builds the output. Okay. You don't have to use the make file, but it's simpler than you know the the actual Sphinx command. And then we've got to upload it. So here we are, we're just using rsync. The gray dots here, they're just a few options that didn't fit onto the slide. Again, the full example is just in the repo. You can go and get that. But the all that's in those gray, those little gray dots are a couple of um rsync options to begin with and then the ssh details and the target location. And for that last those last ones we use repo secrets. So GitHub lets you enter these secrets and then you can just access them from the workflow run. And that's it, job done.

16:31

Speaker 1: We set the workflow up to only run on an update to main. And each time we commit a change there or we merge a pull request, the docs are automatically built and then they're updated live. Now you don't have to use rsync. You can do anything you want with the files, right? I I'm nostalgic, so I'm basically never going to turn down an opportunity to use rsync, but you you do what you like. And that's that's kind of it actually. That's that's Django Sphinx view. What's the roadmap for it? The main view is done, right? There'll be edge cases like it doesn't work if I do this. Or so, okay, we'll add tests and we'll fix those. I imagine small adjustments like that, but but nothing beyond that So even though it's a young project, if you want to jump on and use it, then that's great. One goal I do have is that Sphinx has this concept of a template bridge.

17:18

Speaker 1: Now what that would allow you to do is plug the Django template engine into the HTML builder so that if you wanted to output your um docs as or your your site as HTML, you could use the the make HTML command, the HTML builder. to do that. That would kind of complete the circle. I don't know. It's it's it's a nice extra. That's the sort of only extra goal. And then beyond that, I have one non-goal, which I'm not looking to add a whole load of themes and things like that. There's a there is a whole world of Sphinx theme, that's a mouthful, Sphinx themes out there. You can um, you know, there's all sorts. You can go and browse those and search for them. There's galleries of them. I don't want to do that. The goal for

18:03

Speaker 1: Django Sphinx View is about integrating Sphinx built docs into your own Django site. So you bring the styling with your your site, your own site's Django theme. So themes aren't something they want to add to the repo. So that's Django Sphinx View. That's how you can build dynamic static sites with Django and Sphinx. Again. I'm Carlton Gibson. I'm a friendly Django fellow. I'm Carlton Gibson at GitHub and Twitter. You can find me there. If you haven't listened already, do check the podcast at DjangoChat. com. If simpler deployments sound like your thing, then take a look at button, the btn. dev. That's rolling out this summer again. That's thanks for joining me. That's it for Django SphinxView, right? I hope you enjoyed the talk. I hope the next time you're building a content-based site, you'll consider adding Django Sphinx View to your arsenal, right? I'll be around for the rest of the conference and if you've got any questions do let me know.

18:50

Speaker 1: I'd love to chat. Also, if you want to hop onto the repo, there's discussions and issues, even if you need that available there. Thanks.

18:59

Speaker 2: Thank you, Carlton. Great great talk. Let's uh people can join you in a face-to-face meeting. You can just follow the link below. It's a call.

19:12

Speaker 1: Pull it out

19:12

Speaker 2: video. Thank you.

19:13

Speaker 3: I can give some of your video. Okay.

19:19

Speaker 2: Okay. Thank you, Carlton, again.

19:21

Speaker 1: Thank you, thank you. Just trying to get there. I'm gonna call leave this one now, okay?

19:36

Speaker 4: Good morning.

19:38

Speaker 1: Good morning.

19:40

Speaker 4: Nice talk.

19:41

Speaker 1: Was it good? I was a bit nervous. I was a bit like, ah, is it too short?

19:46

Speaker 4: Yes, it's good. I had no f clue we'd do this for Django using Jason.

19:53

Speaker 1: Yeah no that's it I was like how all You know, because I okay, so this is the backstory. I was like, I need to I want to use it. I want to use Sphinx, I want to serve it with Django. I'm like, how on earth is Django doing it? So you know, go digging around in the Django project. com source code. Oh, that's quite interesting. So but nobody wants to do that, so I thought I'd wrap it up in a nice file and then making it kind of put So it's all it's one thing to put it in your own repo where it's nice and private and then another thing to sanitize it to actually put it on GitHub. That's taken me much longer than I thought to get not as far as I wanted because obviously it has to be clean. Go on, speak whoever wishes to speak. Do I have to drive it? No.

20:36

Speaker 5: I was muted. I I was thinking morning. Uh I had to drive it when I was doing talk yesterday because there were there isn't a moderator for this.

20:47

Speaker 1: Right. Okay.

20:48

Speaker 5: Yeah. The question I had was um um Wow, there's big spectacular echo. I don't know where that's coming from.

20:54

Speaker 1: That's been me. Uh no. Um hang on, if I Look what the I'm gonna do strategic muting because it's me because I haven't got headphones on because I was doing a presentation and you

21:08

Speaker 5: All right, okay. So the question I was going to ask was uh if this looks a lot like um something I I might be able to use for I don't know, like a blog or something as well as as other things. Um what does I suppose the question I actually want to have is I wanted to do something like this. Um and build something like a small site for example is this force powering Numenaz your existing personal site because I'm curious about where that might fit in or if it does fit in at all.

21:39

Speaker 1: Yes, it does. Okay, so it's um it's powering bits of it. Like so it's y I've had that site going it from like, you know, for about It's the same code base from a long, long time ago. And so it's you know, it's got all the nice craft that you would imagine from that. But like the notes section is is using um uh Django Sphinx View now. It was b it was the using the prototype of it, the sort of private prototype. It's now using that. And I've put the Django Sphinx View docs on it. So I'm using that. The homepage itself, for what the homepage is worth, is just a template view. And there's a few other pages which are just template views and then some other bits that I use for, you know, managing my own life, which are um there. But th I intend to use this for basically as much as I can

22:25

Speaker 1: because having been fellowing for three years now or something, I I've just got addicted to Sphinx. I can't I I can't go back to not using Sphinx. So this is kind of like a um a compulsion now, is it? And it's really simple. It's just one view. It's just so simple. There's nothing to it. And so it it's not You know, it's not a big thing. The only thing I want to add is this template bridge to sort of complete the circle. But beyond that, yeah, you could use that. And then really um Paul's stuff. Paul's those Paul's talks that I linked to um that I mentioned. Do look those up because he's got all sorts of details about using Sphinx to generate your static site. And there are people who are there's a blogging thing that's you can use you know that's actually set up to do blogs in Sphinx. You can use that and then you could serve it with Django, no problem.

23:12

Speaker 5: That's fantastic. Thank you so much.

23:15

Speaker 1: Thank you.

23:18

Speaker 6: I have a question, Carlton, Carlton.

23:19

Speaker 1: Yeah. Go on and shoot.

23:21

Speaker 6: Good to see you again.

23:23

Speaker 1: Good to see you too.

23:24

Speaker 6: It's such a lovely project. I really like it. Uh I was wondering if it's possible to um have a workflow where where content could be edited either either through uh through uh a Django CMS like Bagtail uh or uh using using um uh markdown let's say on on on GitHub. So let's say You could accept pull requests on GitHub. And you could also have like internal users who are not very tech savvy to use a Django a Django-based CMS. So I wonder if that workflow uh is uh uh could could be supported because this seems like it's rendering through Django and I'm wondering if it could be like you know also editing through either of these options.

24:06

Speaker 1: Good question. I would imagine that you'd want to use Git as the source of truth there. And so I could imagine somebody building a um a Django editing thing that sits on top of it You know, when when you hit the submit button and it validates the form, the form valid function, what does that do? It it writes a git commit. You know, it's the sort of thing you've seen Simon Willison doing, you know, left, right and centre would you know, get everything. That would so be doable. It's not something I'm gonna be doing, but Anyone?

24:43

Speaker 7: Hi Carlton. Um thanks. Uh that looks really interesting and I think it solves a a whole number of problems that um people typically find themselves picking up much heavier tools to solve. This looks really lightweight. Um I wonder if you s If you've encountered any the limits of this or the the places where you think, okay, that's as far as I'm going to go with this and now sadly I need to take the extra step and invest in the heavier machinery. Um

25:15

Speaker 1: I think for me on my personal projects, no, because I'm looking at writing technical documentation. I mean, you know, the the main focus of it is the the guide for the button app that I'm developing. Um and so that's a t it that's right in Sphinx 's sweet spot. Um there's two things that I've have looked into, Sphinx extension. So once you get it once you start wanting to add like better um, I don't know, interactivity or better You want to have a little bit of JavaScript in your Sphinx page. Well you can write a Sphinx extension that adds that and that's kind of cool and that's easy to do. And there's a rich ecosystem of that. So that's that's great. I think where this really fails is where you try and apply it to non -technical um audiences. So, you know, like real human beings, like non non-programmers, non-technical people, because the literally

26:06

Speaker 1: The that requirement to format a a link synta a syntax, you know, be it markdown, be it RST, that that kind of even though it's a minimal markdown for us, that that's just not feasible for an awful lot of people because they just they just don't think in the right way. They don't have the right mental model or whatever or the right experiences. And at that point, I think the form-based CMS controlled environment is is is the way forward. And that's where I guess Dan's question comes back is like, could could you could you have the best of both worlds? I don't know. I mean That those would be my initial thoughts, Danny.

26:41

Speaker 4: So do we just put them on the top?

26:44

Speaker 1: Yeah, yeah, yeah, you're on, you're on, you're on pretty good.

26:46

Speaker 8: Okay. So yeah, I was going to reply to you to I uh uh uh because I what I end up doing is uh when I have a Django project typically This needs a few flat pages as we call them in Django, but flat pages are uh uh uh they're just not powerful enough. And I I end up For for every Django project for many Django projects I end up uh installing a WordPress site which I accompanies my main site. So That's the problem however because they are there are these uh non-technical users typically. So uh

27:31

Speaker 8: if you have non-technical users you you have to do it some other way I'm afraid.

27:46

Speaker 1: It's nearly clicked the wrong button and kick nearly kicked you out for being for asking a really good question. Sorry, I that's why I panicked. Um Yeah, I mean I I think that the no I think, you know, what would I do in that case? I'd add in, you know, either Wagtail or Find CMS, Django CMS, you know, one of the one of the the the the options. And the The sort of the issue you have then is that there's such a big step up. You know, I what do I do on my own personal site? I just create a template view and I just budget together and I'm happy budgeting it together. But that's not That's not a real solution, right? That's certainly not something you could present to a client. So then you've got to bring in the CMS and there's, you know, define your pages and the higher that You know, that there's there's labor to that.

28:31

Speaker 1: So I mean I certainly know in um the Wagtail world there's um the code red CMS which is set up to you know get get you all out of the blocks very quickly. That's like an add-on to Wagtail and there's another one for um blogging which you know gets you out very quickly with a wagtail blog. I guess the solution is for us in the Django community to have a compelling story that's we go to instead of having to go to WordPress, right? Why is it that 15, 20 years later, we're still, oh yeah, I installed WordPress alongside rather than I use this Django solution. I think that's that's something that That's the question for the Django community. As Django developers, we shouldn't be bolting on WordPress. We should be bolting on a Django solution. I don't have the answer there, but that's my kind of thought, but I agree entirely.

29:22

Speaker 8: So, is it easy to deal with images?

29:28

Speaker 1: Yes. Um so you can add images to Sphinx sites and it will compile them all and put them in a static folder and then you just configure the static folder to um serve as normal. So you you'd you'd know where those static files were going to end up and in in static dears you'd add that extra configuration. That's a good point. I should add that to the um repo because that's going to come up. That's gonna come up again, again, isn't it? But yes, Sphinx Sphinx takes takes the static files that you add to the Sphinx projects and it puts them all in one folder for you

30:03

Speaker 7: Carlton, as far as really complex um layouts and presentation are concerned, I I guess then it doesn't matter because as long as you can present it through Sphinx in a s reasonably structured way, then Django can do all of the clever things with it, including the kind of front-end layers for presenting it interactively. So It takes the layer that you where you'd be doing things in say a Django CMS interface for some kind of interactive thing, it would just be in the a in the markdown or whatever structure within Sphinx.

30:50

Speaker 7: And that becomes very lightweight then, doesn't it? Very fast and powerful. Yeah. Sorry, uh it's not.

30:57

Speaker 1: No, no.

30:58

Speaker 7: I'm just wondering how far how far you've taken this to to demonstrate that kind of thing.

31:02

Speaker 1: So I've taken it as far as one thing I'm very interested in is this kind of tabs example where you have um Um you have the same example for Windows, for Mac, for Linux, and you know, person clicks on which operating system there are and they get the example for them rather than um having and there there are Sphinx extensions which um do that already and built include including the Using the existing Sphinx extensions, it's very good because you just get the HTML and then you insert the the um in the Django templating language, you that then say, right, I need this static file, and you can you can adjust it as you want, but Uh when you dig into the source code of these extensions, they're not particularly complicated. Right? Then that that you know

31:47

Speaker 1: that actually this is quite an easy way thing to do to build a Sphinx extension. And if you knew you were targeting the D DTL at the end of it, you wouldn't necessarily have to jump through quite all the hoops to make sure you had the static files in the right place and all the rest. You'd just you'd just generate the HTML that you need to attach onto with your JavaScript or CSS later. So these kind of interactive widgets, they're really doable. The extent to which they're not, then I don't know, you could you could define a template which um would then include another one which you could then use a Django view to back and I don't know I you know start making it up at that point but What the JSON builder gives you is this nice structure where you've got here's the page content, here's the table of content, here's the other additional bits that Sphinx

32:37

Speaker 1: give you, and you can put those anywhere in the template that you like. And that template can be anything you want. So that to the That's that's why I'm quite excited about it. I think it's it it has lots of potential for doing exciting things. Hope that's an answer.

32:54

Speaker 7: Absolutely, thanks.

32:59

Speaker 3: Hi Carlton.

33:01

Speaker 1: Hello.

33:02

Speaker 3: I I would like to make a question regarding testing. And uh actually is there a a a pa uh in the the other way there is a package uh there are packages like Factory Boy where you can produce dummy data and do end-to-end testing It's not so important in static sites, but I would like to ask you if there is an option or how do you do do you do you manage testing?

33:27

Speaker 1: Okay, the the the way I tend to do it is I have so I'm busy writing my my my docs with Sphinx and then I um have the the watcher the Sphinx watcher which will rebuild every time you save a file And that's there. And then I I just have an output of the of the static site built. And then, you know, what am I testing when I use when that when I run the test? Well, I'm looking to test, well, did did the view render Did it get the right template? Did it I'm not like literally testing the content of this of the Sphinx view, if that makes sense. So of the of the Sphinx doc. That's not the sort of thing I'd unit test because I just end up sort of copy and pasting the content of the docs into the into the unit test. So what could go wrong? It could go wrong that it didn't find the right template.

34:13

Speaker 1: So when I put the path in, Did it did it was it able to correctly pass the the data that it got from the URL resolver into the name of the template file that or the JSON file that it needs to load? That kind of thing Um does that answer the question? I'm not sure if it does or not.

34:31

Speaker 3: Yes, of course. So so for demi data, for example, in case where you have a hundred of items and you may use may you want to check if pagination or something like that is working. So how do you manage?

34:43

Speaker 1: Well then you'd you'd have for in uh for y this particular I mean it's it's not loading a query set from the database or whatever. So I don't I don't know. I would have if I needed that sort of thing, I'd have a known output folder. So I'd I'd output it with a known state, and that wouldn't be live, that would live in my test directory as a um you know, as a as a static, fixed, known state of the of the of the um JSON and I would then route the documentation view to that folder rather than you know my development one that I'm using all the time.

35:21

Speaker 3: Okay, thank you.

35:23

Speaker 1: But I don't have anything like factory boy in mind. I just I just I just wouldn't do that. I'd just have a a fixed rendering and that would have to do Now if you don't ask any more questions, I'm gonna go, yeah, that's it. I'm off. I'm gonna get a coffee.

35:40

Speaker 5: Thanks a lot for that. Really, really, really enjoyed the talk, Carlton.

35:43

Speaker 1: Thank you for saying that.

35:44

Speaker 7: Enjoy your coffee, Carl.

35:46

Speaker 1: Thank you very much, Paulo. Okay. Thank you everybody. Oh no, I haven't seen the question. Oh yeah, this is a

36:01

Speaker 4: chat box.

36:03

Speaker 1: Is the talk talk deck online anywhere? No, the talk deck talk deck is not online anywhere yet, but I can work on that over the course of the rest of the day and see if I can arrange for that to happen. I'm sure it's fairly easy. I think I go export PDF done. I think. We'll see. Okay, let's go and get the other two. See see you in the conference, folks. Thank you.

36:34

Speaker 6: See you.

36:34

Speaker 1: Bye bye.

Questions this talk answers

Can Sphinx be used to build a static website, not just documentation?

Yes. Sphinx can build a directory of HTML files that you can serve directly as a static site, and it provides features such as cross-linking, tables of contents, and validation of links.

Discussed at 2:27

How can I use Markdown instead of reStructuredText in Sphinx?

Install the MyST parser and enable its Sphinx extension. MyST parses Markdown into Sphinx’s internal representation while preserving Sphinx features such as cross-references, directives, and admonitions.

Discussed at 5:26

How do I serve Sphinx documentation from a Django application?

Use Sphinx’s JSON builder to turn each document into JSON containing the rendered content and related page data, then have Django Sphinx View load that JSON and render it with Django templates.

Discussed at 9:31

How do I set up Django Sphinx View in a Django project?

Install the package, add it to `INSTALLED_APPS`, and route its documentation view in the URL configuration. Pass the URL path, the location of the Sphinx JSON output, and a base template for integrating the docs with the site’s design.

Discussed at 11:07

How can I protect Sphinx pages with Django authentication or permissions?

Subclass the documentation view and use Django’s normal access-control mixins, such as login-required or user-test mixins. This allows whole sections to be restricted to logged-in users, paying users, staff, or particular groups.

Discussed at 13:25

How do I continuously build and deploy Sphinx content for a Django site?

A GitHub Actions workflow can install Sphinx and MyST, run the Sphinx Makefile to build JSON output, and upload the generated files with `rsync` or another deployment method. The workflow can run whenever changes reach the main branch.

Discussed at 14:59

Can Django Sphinx View be used for a blog or an existing Django website?

Yes. Carlton uses it for the notes section and documentation on his own Django site, while leaving other pages as ordinary Django template views. It can also serve blog content generated with a Sphinx-based blogging tool.

Discussed at 21:39

Is Sphinx suitable for nontechnical users who need to edit content?

Not generally: even Markdown’s link and formatting syntax can be too much for nontechnical editors. For those users, a form-based Django CMS such as Wagtail, Django CMS, or django-funky would be more appropriate.

Discussed at 25:15

How does Django Sphinx View handle images and other static files?

Sphinx compiles images included in the project into a static-files directory. Django can then be configured to serve that directory like any other static-file location.

Discussed at 29:28

How should I test a Django view that serves Sphinx content?

Test that the view resolves the URL, finds the expected JSON file, and renders the correct template rather than unit-testing the prose itself. For repeatable tests, use a fixed, known Sphinx output directory instead of the live development output.

Discussed at 33:27

Presenters

Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.

More videos by Carlton Gibson

More videos from DjangoCon Europe