Using Wagtail to fight for press freedom

Speaker 1: Hi, I'm Harris Laperoff. My pronouns are he, him, and I work for the Freedom of the Press Foundation. I'm gonna be talking to you a little today about our four Freedom of the press websites that are all Wagtail-based. I've worked with Freedom of the Press for about two years now, initially as uh part of the web development agency Little Weaver, and more recently as their in-house web developer. So before we get underway, I just want to say people try to abbreviate our name all sorts of different ways. The one that we prefer, the one that I'll be using is FPF. So, first, an overview of what FPF is. The Freedom of the Press Foundation is a nonprofit dedicated to defending and protecting adversarial journalism.

Speaker 1: Through crowdfunding, digital security, and internet advocacy. We have some famous folks on our board of directors you might recognize, famous uh whistleblowers Edward Snowden and Daniel Ellsberg. Maker Laura Poitrus and uh star of high fidelity John Cusack. John Cusack does some great political work, but Uh that's always how I'll think of him. Uh we work on a lot of different projects, particularly around the intersection of press freedom and technology. Um Securedrop is the project that uh takes most of our effort. It's a whistleblower document submission platform. It is also a very high-impact project for us. It's in use at over 60 media

Speaker 1: organizations Organizations worldwide, both major and independent. We also raise money for a variety of press freedom organizations. We provide digital security trainings and tools, and we are the fiscal sponsor. For Signal, the encrypted messenger app. We recently announced a new project called Sunder, which allows you to store a secret that requires a quorum of participants. To decrypt it. So if you need something split across multiple people, they all need to be together to decrypt it. It's a very cool project for those of you who are cryptography nerds. It's an implementation of something called Shamir's Secret Sharing. But I'm not going to be talking about any of these projects today. This is Wagtail Space. I'm going to tell you about our Wagtail projects. If you want to find out more about these, visit our website.

Speaker 1: And our four Wagtail projects are our main website, freedom. press , securedrop. org, the website for the securedrop project. Secure the News, which is a news website security leaderboard, and the U. S. Press Freedom Tracker, which records press freedom incidents in the United States So starting with freedom of the press, freedom. press , this is basically our organizational homepage, gives you all the information about who we are as an organization, lists all of our projects. Also holds our writing and our crowdfunding campaigns. And in a lot of ways, it's a pretty basic CMS site. It shows off a lot of the bread and butter of what Wagtail is great at. We've got a bunch of different page types, I think Most Wagtail developers here have probably built a Wagtail-based blog at some point.

Speaker 1: But I think the thing that we have on this site that really shows off the power of Wagtail are these crowdfunding campaigns in the crowdfunding corner here. And what this basically is is we actually built a fundraising system into our Wagtail site. And we call these bundle pages because the way that we do fundraising for other organizations is sort of inspired by the humble bundle which is a game media store if you're familiar with it and basically they have a part of their uh checkout process where you can donate some money to charities and decide how to split up that money So this is, we have these donation bundles on our website and we integrate Stripe checkout and a React-based form. And we have a few extra models to track donations that get made.

Speaker 1: The donation model, donation split model records how a user wanted to split up their donation. But mostly this is just a Wagtail page that has some very special behavior. So as you can see, this is what one of those donation pages looks like and it can be completely configured from the Wagtail admin. You know, they can set which organizations want get to receive the donations, they can set a goal, they can say whether or not to show the statistics on the live site. Basically everything is configurable there. And what this means is it's very easy for FPF staff without my intervention to set up a new campaign on their own, to you know, retire an old campaign when it's Finished. And one of the particularly cool side effects of this is that because the bundle pages are just wagtail pages

Speaker 1: They can live anywhere in the site tree that we want them to. So most of the bundle pages live under our crowdfunding path or freedom. press slash crowdfunding slash project name. But our donation page, our primary organization donation. page that you get when you click that little donate link up there in case any of you want to give us some money later. That's actually also a bundle page and there's no special code built in there to handle it. It's just a bundle page that has one recipe recipients, that's us, and it lives at freedom. press slash donate. We also made use of Wagtail's admin hook. So on the admin side, not only did we want uh FPF staff who are not engineers to be able to spin up new campaigns. campaigns, retire old campaigns, but we wanted them to be able to monitor the campaigns, see how much money was coming in.

Speaker 1: Someone is responsible for actually breaking up and sending out the money to organizations every month. And they needed access to that information. So we used Wagtail hooks, which, if you've never used them, it's a very sort of neat and simple system for adding extra function. functionality to the admin. And in this case, what it does is it checks if a page is an instance of a bundle page. And if it is, it adds this little drop down that lets the uh editor access some stats about the bundle. So that looks like this. We get this cute little view stats button and we have a special template in view for the stats on the site. And they can also be downloaded as a CSV. I didn't check with the finance folks if I could share any financial data, so I just put in a million for every donation, which I think seems like a pretty reasonable amount for a nonprofit in a month.

Speaker 1: All right, so our next site is securedrop. org, which is the informational site for the Securedrop project. And while some of the problems that we solved in this building, this site are not strictly wagtail related, in particular around viewer privacy, I thought they were interesting and wanted to share them with you. So securedrop. org targets several audiences, and the two primary ones are media organization admins who want to set up a securedrop instance at their organization. And whistleblowers who are looking for somewhere to send their documents. You know, they have sensitive documents that are newsworthy and they want to know. how to submit them. And this second group has particular security concerns as you might imagine. And in fact, just the act of uh well uh

Speaker 1: so right so as you can see the site has information about secure drop how to get it Your organization, but it also has a directory of secure drop instances active around the world. So if you are a whistleblower looking to submit a document, you can find the publication that you want to submit to. There. Oops, let's go back a bit. Right, so whistleblowers have particular security concerns. might imagine and just the act of visiting this website is already actually leaving a digital trail that could connect you know a particular leak to them whether that trail is in the form of their ISP's records or the DNS request traffic or their browser history. And so one of the primary goals of this site

Speaker 1: is actually to get a whistleblower off of this As quickly as possible and if possible into Tor browser, as you can see from the obnoxiously bright warning on this page Now, if you've never heard of Tor Browser, that is a web browser that protects your anonymity on the internet. It does this by sending web requests through a path of relays. That are volunteer relays around the world. And here's a little diagram from the Electronic Frontier Foundation of how it works. And basically every time you send a request, it goes through a random path of relays, makes it very difficult to trace. And in addition to that, their browser software also blocks a number of technologies.

Speaker 1: That are either used for tracking or have been shown to have vulnerabilities in them. So we really encourage our users to use this browser, particularly if they're whistleblowers or potential whistleblowers. So we use a little bit of JavaScript to detect whether a user is using Tor Browser, which is sort of challenging because part of the appeal of Tor browser is that it is like hard to analyze that traffic. And there are three levels to this detection. We have if the JavaScript is disabled, we assume they're using Tor browser with the security settings all the way up. This may not be true, maybe they're using some other browser with the JavaScript disabled, but at that point, you know, we don't have a lot of power to do much more granular detection. That.

Speaker 1: At the second level, we have a bit of JavaScript that can detect or predict whether they are likely to be using Tor browser right now. So we'll know that they are probably using Tor browser, but they don't have JavaScript decisions. So their settings, their security settings are probably low. And at that point we'll display a warning that says, hey, we see you're using Tor browser, you should turn your security settings up. And at the lowest level, we detect that they are using a browser that's probably not Tor Browser. We display the warning that you saw earlier that says, hey, install Tor Browser, you're not anonymous. So uh because we encourage users to use Tor Browser, of course we also want the site to look good in Tor

Speaker 1: browser, it would not be very good if we were like, hey, use Tor Browser and they visit the website and it's a mess and there's like overlapping divs all over the place. and they can't click a button because it uses JavaScript. So we have to optimize this website in particular and all of our websites to work in Tor browser. And you know a lot of I could give a whole talk on designing websites for Tor browser. But I'll just go into it quickly today because a lot of the stuff that you do when you're optimizing a site for Tor Browser is stuff that web developers really used to do a lot of. And you know, as we've gotten more used to used to ubiquitous uh modern browser availability. We do it a little less, but it's basically just building in graceful fallbacks for stuff like JavaScript, for web fonts, for SVG, all of those New technologies, some of which are used for tracking or some of which have vulnerabilities.

Speaker 1: So once again, this is what securedrop. org looks like in Firefox And you have that warning there. And this is what it looks like in Tor Browser. It's a little less fancy, but I think it still looks pretty good and uh it's totally accessible Um oh yeah, and so I just added this in. I didn't get time to prepare uh any talk about this. But someone asked a question earlier that reminded me of it. One of the other problems that we solved when we were building this website was we built an integrated search that pulls documents from multiple sources So in this case, it pulls documents from the secure drop documentation, it pulls posts from the secure drop support forum, and it pulls the pages from our Wagtail instance.

Speaker 1: And integrates them into a single search. It ranks between those different search types. So as you can see, you know, the top link here is to the documentation, but then there's the forum, and then there's another documentation. Presentation link. If you are curious how we did this, feel free to come and ask me. I would love to talk about it. So the next site is Secure the News. And what this is, is it is basically a leaderboard of news websites according to their HTTPS implementation. We built this to encourage news organizations to adopt good web security and it grades their security practices and lists them all in this one website. It's basically a pretty small site.

Speaker 1: We do the grading using a library called pushed. If you need to remember the name, it's an anagram. of HTTPS. Um it was developed by the General Service Administration's 18F team. It was Initially developed to uh um to scan federal uh government websites for their security practices and to make sure that they were in compliance with security regulations. And I think this site exemplifies, you know, what I think is one of the really great things about using Wagtails that it gives us access to all of these features and And the ecosystem that developed around Django even before Wagtail existed, but that we can make use of in our Wagtail site.

Speaker 1: So for instance, we use a management command. And to run the scanner periodically. It's hooked up to a cron job, it runs the push scanner, it writes all of the results to our database, and that updates the Wagtail pages We also use Django REST Framework. Many of you are familiar with Django REST Framework. If you're not, really great, really easy way to build a REST API. It provides these nice web interfaces for browsing your API. And then we then use that API to power our secure the news bot, which will automatically tweet when an organization Improves its security rating. And some good news about secure the news. When we first started scanning websites in 2016, only 37%

Speaker 1: were offering HTTPS, now 74% Four percent are. Um for defaulting to HTTPS the number has tripled from twenty-two to sixty-six percent. And uh And uh for those of you who are really security nerds, the number implementing strict transport security has also jumped from 9% to 29%. For those of you who don't know what that is, it basically prevents downgrade attacks. where uh someone can get you to go from an HTTPS connection back to an HTTP connection. Um the third site that I want to talk about today is the US third fourth. The fourth site that I want to talk about today is the US Press Freedom Tracker. Um all of the sites that we manage, this one does the most heavy lifting.

Speaker 1: content-wise. It was created at the beginning of 2017 in response to an increasing atmosphere of hostility towards the press and in collaboration with 20 press freedom organizations. We decided to launch this website that records press freedom incidents in the United States. And that includes everything from journalists arrest to journalists being stopped at borders, equipment seizures, subpoenas, physical attacks, more Since we started recording in 2017, we've recorded 168 incidents in our system. Now each incident here is a Wagtail page, but because not that not only does this site aim to provide reporting on this incident, but also to actually provide a database that you can filter

Speaker 1: , that you can analyze, that you can generate statistics from and identify patterns from. We have over 50 fields on these incidents. pages that you can categorize and add information to them with. So here for instance are just the fields specific to the detention and arrest category of our incidents. And I also want to bring attention to the specific active widget up there, just for a moment. So that is from a package called Wagtail Auto Complete that we developed specifically for this project. It's an open source package developed by Emily Horsman. And it provides this auto-completing Is that playing? There we are.

Speaker 1: This auto-completing foreign key or many-to-many selection widget. And I'm bringing attention to it because I would love to really improve this widget. it, get people using it. I think that my dream is that someday this could make a great addition to Wagtail Core. So I don't know if it fits into this sprint or not, but I would love to get some of you to look at this and help me out with it. It later. And in addition to being uh, so the package provides an edit handler for Wagtail, so you can include it in the Wagtail admin, but the React component was actually written so that it can be used on either in the Wagtail admin or on your front-facing site. And we actually do use it in our filter form for visitors to the site. They can use that same widget to filter by state or by

Speaker 1: any of these other categories. So yeah, it's a pretty neat little widget. Love to talk to you about it. One other little trick that we did um in this site, which I'm not exactly sure that I encourage, but I think I think it's a pretty cool trick, so I wanted to share it. Is that we wanted it to be able to have dynamic content in some of our text fields Because we wanted our editor to be able to put in something like there are currently four journalists facing criminal charges. And you know, we didn't want them to have to remember when they add a new

Speaker 1: uh entry about criminal charges against a journalist or when they update one to say the charges were dropped. We don't want them to have to remember Remember that they have to go to that place wherever they put that contents and update it. We want that number to automatically update. So we actually created a couple template tags for this. The first one is this very simple render as template. template tag which basically accepts a string and it runs it through Django's template engine. Now this is the reason I'm not totally sure that I would encourage this. Django temp Django template language is not really designed around safety. It's not necessarily safe to feed at arbitrary strings, but in this case we figure that this site has a very limited number of editors. We trust them all if they're trying to Sabotage our website, we have bigger problems.

Speaker 1: Um so we put it in here. And then uh we generated a bunch of statistics template tags. We have a statistics system in this site that like registers uh particular template tags as being specifically for statistical analysis. This is the sort of simplest one. It accepts a bunch of keywords to filter incidents by and it tells you how many instances Incidents match those keywords. And you can see here we have a utility class called incident filter, which is sort of where we abstracted all of the complex filtering behavior that this site supports, and we use it here in this tab It's also used, it can basically accept parameters either defined in Python or it can accept parameters directly from a query string. So that's the same utility. class that's used when uh

Speaker 1: someone browses the website on the front end. So we have this number of incidents template tag. And in the template it just looks like this, you know, render as template the body of the page. And what that enables us to do is that our editor can just enter that template tag right into the body of the, in this case, a quick facts block, and it renders onto the the page. So again, not sure I would encourage the use of that trick, but I thought it was pretty cool. So a few final notes that I want to make. Freedom of the press is a big proponent of open source software. Secure the news is currently our only Wagtail project that is open source, but we are working

Speaker 1: On getting the rest there. The next one to get open sourced will be the press freedom tracker because we want, and we've actually had requests for other organizations to be able to open press freedom trackers in different countries. countries and the other two projects we'll we'll also be working on. Just wanted to mention A few things that we are constantly thinking about in our Wagtail deployment and our Wagtail sites. One is deployment and development environments. We've been constantly fiddling with our Docker and Docker compose setup. trying to find the ideal way to do it. And that's stuff that we're still working on. So if you're excited about talking about that, I'm excited about talking about that. And the other thing is we have four different websites and they all have different code

Speaker 1: bases, but they have a lot of shared code. And one of the things that we haven't really worked out yet is how to manage that shared code. code. And initially we kept it all separate. We thought about creating a shared library where you know we put the code that we plan to use over and over again. But you know even if you're only using something like that internally you end up wrestling with versioning and like Like you know, Press Freedom Tracker is on like version point two of our like shared code, but like we made a change to code blocks and now freedom. press is on point three. And so we avoided that approach, and now we're sort of thinking, well, maybe it's time for us to go back to that and stop duplicating code across repos. So that's something else that we're working on. I want to give some shout-outs to people who are in the room who worked on these projects.

Speaker 1: My coworkers Mike and Connor are actually not in the room because we scheduled a meeting right when uh right when I was speaking so they are missing my talk but I work with them every day on these websites. Um I also want to give a shout out to Rachel Stevens and Naomi Mordec Talbman who are responsible for large portions of these sites when we were working together at Little Weaver. They are currently available to hire, just saying. And I want to give a shout out to all of you without the Wagtail community and the Wagtail Project, this work wouldn't be possible. So thanks. Thanks.

Speaker 1: Any questions? I think I'm below time, right? Yeah.

Speaker 2: Do you do you want to talk about how you did your uh your search for

Speaker 1: Yeah, sure. I'll give a brief little overview of that. Basically, we have an app for search that defines a search document model. And so we use Postgres as our database, and we were really excited about using Postgres search and not having to rely on the rely on Elasticsearch or something else. But we still needed the we still needed the abilities of a you know something that would index the search and like pull the content from different sources and normalize it into one data form. So we created a search document model that has like a title for each search document, a content body. Actually, I think I recently switched that because one of the cool things about Postgres search is they support storing search vectors directly in the database.

Speaker 1: So that's actually a search vector field now that stores the search vector. data in a like ranked for rankable form. And basically we have three management commands that periodically scan our three sources. And update the search. Some of those don't need to be run regularly. Like we do technically have a management command that will rebuild the Wagtail page search documents, but primarily what happens is we use signal. to rebuild a Wagtail Pages search document every time it's saved.

Speaker 2: So you're like syncing here, you're like copying the document.

Speaker 1: We're basically, yeah. So like the forum management command actually goes to the the discourse forum and it like uses the API and scrapes all the posts and builds a search document for each post. I think Discourse actually has a push API of some sort, so we might switch over to using that, but right now We just rebuild the index every time. Great. The whole talk is worth it. Any other questions? All right, well thank you. Oh, there's one over there. Oh.

Speaker 3: Some of the things that users have to do to get things to you in secure ways. actually require a certain amount of technical sophistication.

Speaker 1: Yeah.

Speaker 3: And how many whistleblowers actually attain that? Because often there'll be people in Who aren't the technical credit of our credit, they're the the most the often they're the port upon people who have the fewer fewer skills.

Speaker 1: Absolutely. So this is not my area of expertise. I work on the websites. I don't mostly work on the securedrop project, but But I will speak to what I know of it. Um the first thing is that freedom of the press ourselves does not actually collect documents from whistleblowers. Um I think we do have a secure drop instance ourselves, but is mostly sort of a dog fooding and sample thing. So uh but the other thing is yeah that is basically the reason why we find it so important to have things like that very clear warning, you know. That like you're visiting this website, your privacy is compromised because we expect that people who visit our website don't know about Tor browser, you know, they don't know that their ISP can track them

Speaker 1: And we need to let them know that as quickly as possible. And I know that it is also a major concern of the Secure Drop project to make it very easy for Wisconsin. whistleblowers to access that. Does that answer your question?

Speaker 3: Yeah, it sounds like a thorny problem anyway.

Speaker 1: Yeah, it's certain it's certainly a thorny problem.

Speaker 3: Can you answer

Speaker 1: Anyone else have a question while you're going to be able to do that?

Speaker 3: Oh yes, I uh it's people always get made people always get fooled often get fooled by sites that look like real sites and end up um giving deposits to people that they thought were going to be for a an apartment or something and then they'd never see it again. at risk from site.

Speaker 1: Sure, so we have um sort of a way to verify a site identity. And there's actually like a few different sites that are involved in a secure drop instance. Um we have our own directory that lists all the secure drop instances and then each organization has what's called a landing page, which is a site that's available just from a normal web browser that basically, you know, for most news organizations, this is the website that says, have a tip, here's how to send it to us, and it will include information about their secure drop instance. And the landing page and our directory both list for each secure drop instance an onion URL, which is a special thing. Again This is not my area of expertise, and my coworkers would explain it much better. It's a special way of accessing a website

Speaker 1: using Tor browser that provides a very strong layer of verification because the URL itself is like, I mean it looks like a string of random letters and numbers, but it in fact uniquely identifies That server. And to add an extra layer to that, on securedrop. org, we actually have a little bit of text that says, hey, before you submit a document here, look at the URL in our directory, look at the Onion URL in our directory, look at the Onion URL on the landing page of the New York Times or wherever you're supposed to submitting this to and make sure that they match. And if they don't match, either someone is trying to trick you into sending a document somewhere that you shouldn't be sending it, or it's a miscommunication between us and the publication. and let us know and we'll work it out.

Speaker 1: So that's one of the ways that we sort of work around that. Any other questions? All right, thank you.