Django Built a Nonprofit

Speaker 1: Welcome James Chirk and Jeremy Carbock. Um

Speaker 2: so yeah, as as Jacob said, uh For eight years, Jeremy and I have been working together at the Sunlight Foundation, which is a nonprofit that the goal is really to open government data and to make the government more transparent transparent and more functional. And that has spread from eight years ago when we were only working on federal projects to some of the projects that we'll talk about, you know, working on state level transparency, working on transparency in in towns and also on the international level. Um and you know, we'll get in a little bit more detail on what sunlight actually does in a second here, but Jeremy's gonna give a little bit more background first.

Speaker 3: Yeah. So um when we started in 2007, Sunlight looked like much of the nonprofit world at the time, which was basically a Drupal wasteland. Um most most organizations didn't put much of a priority on like actual technical So you would find a lot of nonprofits, especially in DC, especially in the government transparency area, that uh based all that they did on uh a CMS uh that was often Drupal and they often contracted other people to build their tools. Uh sunlight however uh knew how important it was given the type of work So because of that, uh when James and I started, we moved Sunlight from Drupal uh to building almost everything on Django.

Speaker 3: Um and it's been a core part of everything we've done since almost day one.

Speaker 2: Yeah and we were taking a bit of a gamble. It was. 96 at the time. So thanks for keeping working on it. And

Speaker 3: So one of the things that we we learned really on uh one of the core things about Sunlight is that everything that we've written has been open source. Um uh but you know open sourcing is complete tool to view earmarks isn't really that reusable by that many people. Um so we started releasing like individual components of things, breaking things out into various small pieces and publishing components uh that had nothing to do with transparency. And um some people use them, which was good. Uh so um We want to go through a couple of them here and show um either cool Django stuff about them or how they reflected the government transparency. movement at the time.

Speaker 2: Yeah. So back in two thousand nine, two thousand uh two thousand eight, two thousand nine, you know, crowdsourcing was all the rage and, you know, we were being funded to do a lot of crowdsourcing projects and some of them were valuable. Um we were doing things like collecting earmarks, um, having people annotate um you know, information and um having people request data sets that they wanted to see from the government. And so one of the first things um uh one of the first projects that came out of this work with something. Um it was uh pretty I mean it was a series of hacks really into the admin, monkey patching a lot of things so that you could do kind of pluggable object moderation so that instead of building a moderation queue for every object in our system you could just we could just use the admin and it had some some nice sidebars and things like that where you could see you know which objects were moderated already and things like that so that we weren't getting um all sorts of spam and stuff

Speaker 2: stuff into the system and you know really relying on a lot of what was already there in the admin and 1. 0 or so, but you know taking that and making it something that we could then use on the dozen or so crowdsourcing projects that we would wound up launching. And we use it for things like you know speaker submissions and stuff like that, uh, even more recently than that. Um another one was, and Jeremy will get into this a little bit more later, but you know, uh nonprofit has a lot of content to put out, a lot of policy content and a lot of uh um you know just you know writing that winds up on various sites. And you know we wanted to be writing it in you know in Markdown a lot of the time and uh I think actually I had a conversation with Jacob uh at some event, I think it was an OSCON or something like that, about what, you know, wouldn't it be nice to To store the pre-rendered markup in the database alongside it, kind of a

Speaker 2: hacked composite field. And so I wound up writing a Django markup field, which has also been used by a lot of projects obviously outside of the government transparency space. And you know, again, really relying on what Django is already offering, but expanding that and you know having a way so that people could you know be editing that markup and storing it in the fields alongside that.

Speaker 3: Uh before a lot of the static files improvements that came into Django, um we were at this one package, Django Media Sync. Uh it started off as just like a a simple management command that would push your static assets up to S3. But some people seem to like it. So this was the first project that I ever released that actually got community contribution. Which felt awesome. Uh but uh we were able to add lots of great things like pluggable backends, so it would work with Rackspace Cloud Files. You could write your own custom backend for storing your media. It would take care of doing processing like CSS and JS minification, uh SAS compilation, uh there's a pluggable system for that. Uh so yeah, a lot of a lot of work went into it, and then static files came out and ruined it all because it was not needed anymore.

Speaker 3: So thanks Django. And then also one of uh the more personally useful projects uh that we've released is Django WordPress. uh which is an implementation of uh models that map to a WordPress database and also uh default views and routes so that You could basically point it at a an existing WordPress database and uh all of the URLs and everything that WordPress provides will still work. uh pro show your blog. Um this has been great for sunlight because uh our website and our entire content publishing system is based on this where Our authors get to use WordPress's uh authoring interface, which is actually quite nice, uh, but we don't have to write PHP and we don't have to uh find ways to

Speaker 3: hack uh you know, WordPress into a larger Django project. Um so that's that's been incredibly helpful.

Speaker 2: Do the intro.

Speaker 3: Yeah. Um so we wanted to go over a couple projects here to show uh some of the stuff that we've been able to do at Sunlight. Um one of the reasons we wanted to do this talk was mostly to thank you all uh because without Django and the people building the community community around it and building the ecosystem around it, we wouldn't have been able to uh achieve much of what we've done at Sunlight. So um thank you and peer some stuff that you all helped us build.

Speaker 2: Um so I think one of the ones that you know this community uh really helped build in a number of ways, uh not only Django, but also kind of you know using Um, you know, kind of piggybacking off of an existing Django uh community, an existing Python community was uh a project that's near and near to my heart, which is the Open States project. This is a project that collects information from all fifty state governments, um standardizes it and puts it into one place where people can do research and find out what's happening in their state house without um you know visiting uh what's often a very terrible difficult to use, mid to late 90s designed ASP um government website. And you know, this project was actually built by people That were at PyCon, uh interested in working on projects.

Speaker 2: And a lot of them, you know, some people are in this room uh that have contributed in some ways to this project. And you know, we couldn't have done this uh type of thing without, you know. this community uh without you know taking guidance from how Django ran its community, um, you know, doc you know taking guidance from documentation, you know, hosting our docs on Read the Docs. And And you know, and then you know hosting the site and the API that gets you know over a billion hits a year now uh to the API, uh all through a Django site. And you know So thank you for that. You all helped build that.

Speaker 3: Yeah, just to make it clear, when he says like contributions, there are 52 individual scrapers that scrape uh state websites. sites every day, constantly.

Speaker 2: And twenty of those were written by people at Sunlight, but 30 plus of those were written by, you know, people uh at uh that were part of this community. So um another project that was built entirely uh in Django is our Influence Explorer project, um which kind of aggregates campaign finance information and lobbying information, all sorts of information about influence in federal politics and state politics so that people can see you know where money uh is coming from and going within the government and all of that done in a you know pretty nice, relatively easy-to-use way so that you're not downloading Massive um EBDIC files from the FEC, which is how the data is actually put out. Um and you know, you can look at it like this, and again, that's built by all of you thanks to Django. And

Speaker 2: this one. The third one is uh the uh capital or capital words project, which is kind of a more fun project, something maybe just to bring up on your your laptop or phone at some point, which is a project that lets you see what the most common words are used uh in the federal in Congress every day and also you know compare you know what party uses this word more so you can sort of for just you know anywhere you can search for puppies and see you know who's talking about puppies in Congress or you can search for more serious things too if you'd like.

Speaker 3: One of our first projects that we did was called public markup. And it was a way to add annotations to the text of legislation. So this project really got its start when uh during the uh the bailout, uh bank bailout. up after the uh financial issues we had. Um and so uh we were able to get some uh text of the drafts of some of the legislation before they were actually introduced. produced and so uh we got them loaded up on the site and we actually had policy analysts coming and um you know giving their comments on various portions of the bill. So I think it really had an effect on uh you know how some of the final drafting of those bills uh came through. Uh we worked with

Speaker 3: IDO design firm to do an app called uh Sytgeist. Uh we wanted to make government information of uh approachable and like usable by the average normal person. Um so we aggregated a whole bunch of different government data sources and some private data sources. as well. Django provided the uh API that powered all of this, uh, but um it was basically mobile apps that would give infographic style presentation. They were actually web views, so that was all Django presenting those. We had a web interface that people could use to do uh demos and stuff. That was uh one of my favorite projects. Um a lot of the projects we do also like try, as James mentioned, with um Gatekeeper, try to get

Speaker 3: the public to help us do stuff. Um So there's a series of projects here.

Speaker 2: So yeah, one of those projects and I think one of our favorites, again, all built on Django, including kind of a contribution workflow using You know, things like uh gatekeeper I believe. And it has been this project political party time. This is basically when people get, you know, those those invites that say, Come and spend, you know, three thousand dollars on a you know, chick and dinner uh to support you know this candidate, those people can forward uh you know anonymously if they'd like uh forward those invites to us so that we can you know list them on this site and then you know aggregate that information into a place where you can see you know who's paying uh what to be where with who, I guess is kind of the long way of putting that. Um and you know this again was a a really impactful project. I think something that really got a lot of attention um especially around you know political conventions and things like that

Speaker 2: and kind of shines a light in an area that you know people don't really think about but that's like where influence it really happens is like you know at these dinners where people are eating uh you know a little their four hundred thousand dollar hors d'oeuvres or whatever it is. So

Speaker 3: being an advocacy advocacy organization, we try to get people to talk to their members of Congress. So we uh released a standalone package called Django Callbot 3000 , which uh is basically a tool that uh lets you create calling campaigns. And uh uh so you can like have suggested scripts for people to call. It helps uh people find who their representatives are. Uh we recently added the start a hangout thing. So you can can just click it and it launches Google Hangout that does a voice call to your representative's office. So we've used this for a number of campaigns around legislation to improve transparency. And we also run something called Sunlight Academy that a lot of journalists use. So we'll do uh tutorials about

Speaker 3: Sunlight Academy about um how to access data, how to interpret data, uh all sorts of things like that. Um and we publish all these here. People can come on, uh like mark the ones that they've completed and get And we've used uh Django to actually build Sunlight itself. Um I mentioned Django WordPress for our blog and our site. Uh we built a internal media management tool uh so that we didn't have to use the one built into WordPress anymore. Uh uh called Horseradish. Uh And then uh we also built our kitchen cleaning day schedule uh called Cleanosaurus Rex. Um uh it's actually mounted in a tablet in a kitchen in the

Speaker 3: the office that shows whose day it is. You can like reward them or send them anonymous nudges if they're not doing their duty. You get emails when your day is coming. up and stuff. So this this is like the critical piece of sunlight infrastructure here.

Speaker 2: I mean transparency can apply to the kitchen too. Yeah. Really see it see who's doing a good job and who's not. Daniel's doing all right.

Speaker 3: So um all good stories have an M. N. Shyamalan style twist to them. So this is the part in the talk where we reveal this one. James and I are both actually actually dead. Which which not quite true, but uh as of two weeks ago neither of us work for Sunlight anymore.

Speaker 2: So we should probably make that clear. But Sunlight, you know, uh you know, we really wanted to say, you know All this work would not have been possible. Everything that Sunlight has done and will continue to do without us is you know something that you know really everyone here can take some pride in and you know you've made you've made your your country, you've made your state, you've made your cities better places, hopefully, because of all that work. And so, you know, thank you for that and happy birthday, Django.