The Texas Tribune: Making Data, and State Politics, Public by Ryan Murphy

This video features Ryan Murphy at DjangoCon US 2015 in Austin, Texas, USA.

The Texas Tribune: Making Data, and State Politics, Public by Ryan Murphy
0:41:33
Published November 3, 2017
229 views

The Texas Tribune: Making Data, and State Politics, Public by Ryan Murphy

The goal of everything we do is the same – how can we produce something useful for the citizens of Texas that enable them to be better participants in their state government?

Our News Apps team is responsible for the building and maintaining of editorial-focused data explorers. Django's ease of use has made it possible for us to architect both robust back-end systems for managing the government data sets that power these apps, and to build compelling interfaces to the data for our users to find their own stories.

More details on the three projects we'd discuss:

The Government Salaries Explorer is our most popular explorer. This project manages the payroll data we've collected of more than 300 thousand public employees, providing a peak behind the curtains into how tax dollars are being spent. It required a system that could standardize the many different formats a public agency may release its payroll information to us, but also remain easy to use for all members of the team so updates happen in a timely fashion.

The Texas Legislative Guide was our spin on a legislative bill tracker. Instead of placing all the focus on the bills themselves, we instead created a platform for our reporters to provide context on the many topics and issues that come up during a legislative session. While we still have the capacity for users to search for bills, the site's bigger focus is on the potential changes this session's legislation may have on the state.

And finally, our upcoming revamp of our Public Schools Explorer will be a Django app. This project is currently in it's very early stages, but it's on track to be released by DjangoCon so there will be plenty to show by then! The challenge – how can we take Texas Education Agency data and turn it into something usable for the citizens – and parents – of Texas? We are building an interface that makes it easy to compare and districts and campuses to one another, opening up state data that has always been public but frustratingly trapped within complicated web forms and paper printouts.

While this proposal is more focused on "what we did with Django" vs. "how we did it" – although that will be touched on as well – I believe the work we've produced is a testament to the impact we've been able to have on the state and its citizens thanks to the support of a system that works well for us.

Help us caption & translate this video!

http://amara.org/v/HHB1/

Summary

Ryan Murphy explains how the Texas Tribune uses a small, editorially embedded news apps team to turn government data and state politics into accessible public tools. The team chooses the simplest technology that fits—favoring static sites when possible, and using Django when its admin, models, forms, validation, GeoDjango, and PostgreSQL support rapid development and reliable data processing. He describes projects including salary and prison explorers, the Texas Legislative Guide, and the Disappearing Rio Grande, while stressing tight deadlines, open-source code, maintenance plans, and eventual sunsets for temporary applications. Murphy also says Texas agencies have gradually become more responsive to public-records requests, though journalists still need to challenge unnecessary barriers and charges.

Key takeaways

  • The Texas Tribune separates its editorial news apps team from the platform team so developers can focus on reporting and public-facing projects.
  • The team prefers static applications when possible, but Django is valuable for quickly modeling data, validating government records, and providing administrative tools.
  • GeoDjango and PostgreSQL are especially useful for projects that compare data across Texas cities, counties, and the state.
  • Projects should have a maintenance and sunset plan so abandoned applications do not remain costly or break unexpectedly.
  • The Tribune’s issue-based Legislative Guide helps readers follow policy topics even as bills and their numbers change.
  • Government agencies are increasingly creating dedicated staff and processes for open-records requests, although persistence is still often required.

Summarised automatically from the transcript.

Chapters

  1. 0:16 Introduction to the Texas Tribune Ryan Murphy introduces the Texas Tribune, its nonprofit model, state-politics coverage, and public data projects.
  2. 7:15 News Apps and Platform Teams He explains how the Tribune divided its technology work between the editorial news apps team and the platform team.
  3. 16:31 Choosing the Right Technology Murphy discusses when the small team uses Django, Flask, Node, static-site generators, and other tools.
  4. 18:02 Django for Data-Driven Journalism He outlines the benefits of Django, GeoDjango, PostgreSQL, the admin, models, forms, and validation for handling government data.
  5. 23:30 Project Planning and Sunset Strategies The talk turns to tight deadlines, maintenance costs, and planning how temporary projects will eventually be retired.
  6. 25:31 The Disappearing Rio Grande Murphy describes building a Django-powered project for a reporter’s journey, including data collection, satellite tracking, maps, galleries, and WhiteNoise.
  7. 30:34 Texas Legislative Guide He explains how the Tribune shifted from bill tracking to an issue-centered, reporter-curated guide with a defined endpoint.
  8. 34:29 Government Salaries Explorer Murphy discusses separating the salaries app’s data and presentation layers and documenting data-cleaning workflows in open-source repositories.
  9. 36:47 Public Schools Data Project He previews the planned redesign of the Tribune’s public schools explorer.
  10. 37:39 Questions Murphy answers a question about whether Texas government agencies are improving their data access and transparency practices.

Transcript

7,599 words · auto-generated Show

Automatically transcribed, so expect mistakes in names and technical terms.

0:16

Speaker 1: Hi everybody. Thank you for coming and joining us here in Austin today and kind of taking using your Labor Day for hanging out with a bunch of other Django folk. Um so yeah, so I um work at the Texas Tribune, which is a uh start advancing my slides. Um News outlet here based out of Austin, Texas. We cover uh state level politics and policy. So we're we're really interested in anything that's going on at that big pink building just down the road I'll be explaining in just a second what news apps means and kind of the bigger picture of the uh Texas Tribune and how our kind of technology groups are organized.

1:03

Speaker 1: But just to touch a little bit on the Tribune itself. We were founded in 2009, so still relatively young as far as news organizations go. Um we launched um on election night um in November of 2009. Um and uh we were kind of we were founded as intentionally as a as a nonprofit um and that was that you know is funded through um membership so it's a little bit of the kind of PBS and um MPR model. Um we have foundation support, we have um Events that we host that we charge admissions for. The Texas Tribune Festival, for example, is one that's that's coming up here in October. Looking at tribune people back there, yes, October.

1:50

Speaker 1: Um that is actually here um at the uh UT campus and HT conference center. So um if you're around, you should come check it out. We have the uh largest state house news bureau in the United States, so that means that we have more people who are covering state politics um than any other organization um in the US. um which has been was one of our initial kind of claims to fame and one of our kind of goals when we were founded was to um fill in the gap that we saw was um kind of left open as as a lot of the other other news bureaus kind of were were you know figuring out where everyone was going. Good news is a lot of these other organizations have definitely

2:36

Speaker 1: rebuilt a lot of that team. So good news for Texas and good news for state coverage. If you if you've never heard of the Texas Tribune or um just maybe you may know a couple of these things, but um we have a c kind of a a few different projects that we've that that we tend to be known for. The Salaries Explorer, which is one I'll touch on a little bit later, is our One of our kind of biggest traffic draws and one of our biggest um actual data applications that we have currently has more than 350,000 state employees within it from more than 50 different agencies in the state. Um this is something that we're constantly updating, constantly um

3:22

Speaker 1: adding new agencies and refreshing the data in The kind of goal or purpose of this project has always been to kind of show the citizens of Texas where you know where their tax dollars are going. It's always been kind of a you know one of our kind of bread and butter um experiences that we've always had at the Tribune and it's kind of carried it through. I think even I think it was 2010 actually is when we when that when the original version of this came out. The Personers Explorer, same kind of idea.

4:09

Speaker 1: We have data that we collect from the Texas Department of Criminal Justice. um which has done you know has this big database of everyone that's in the currently in the state prison system. Um and we've Taken that data, we've cleaned it up, we presented based on kind of crime breakdown by housing, you know, by the um unit that they're located in. Um this is updated month by month. So as people leave the system, they get removed from a wrap as well. But the kind of goal of this was originally to present, you know, make this something that was a lot more user-friendly than what the state currently has. And still has. It's very, very kludgy. It's more kind of just a search box and hopefully you get lucky finding, you know, looking for someone that you want.

4:55

Speaker 1: So this is kind of indicative of a lot of of of the the work that we do at the Tribune is is taking these large data sets, these large um state government produced data sets and turning them into things that are um More consumable and more user-friendly. One of the Tribune 's kind of main tenants is always been You know, and it's kind of a you know a basic news journalism tenant um is is is kind of serving as like the translators of of kind of the wonkiness and turning it into something that um the people of Texas can use and and and understand. and and our work with with data is is a big part of that. One of our other um this is not necessarily a data project but definitely one of our um

5:41

Speaker 1: other uh kind of claims to fame was was our uh coverage of the um abortion filibuster here um in Austin back in 2013. Um this is our last legislative session. Um just it was almost by stroke of luck that we happened to be um that session had got a a agreement with the state to stream to live stream the House and Senate chambers. And Uh we we you know they've they've always had the ability for to do this, but it was a horrible, horrible real player powered monstrosity back in the um one of the back rooms of of the Capitol. Um and we we'd we'd got to organize kind of organized with them to uh get it turn it into something that could actually be on YouTube.

6:27

Speaker 1: and be streamed through YouTube, which really opened up the audience and a bit in just the the ability for people to actually see this stream. And again, just just kind of just so happened to be that that was a very good year to have that live feed and have that live look into what was going on in the House and Senate. That was a very, very long night. But this was kind of our pushed us into doing something very similar this year as well. um and really kind of change the the dynamic there that that it's it's now really under s expected and and and understood that this is something that you know people will be able to watch all across the state. So today, kind of what we're what I'm what I'm hoping to kind of get

7:15

Speaker 1: uh to kind of tell everyone about is Like as as a news organization and as a pretty small one, like how do we kind of go into our decision making about when and when not to use Django for our projects? And and when we f if we do decide to use it, like what goes into making those decisions in terms of how we're going to architect it and how we're going to um choose what components are part of it and approach the deadlines and kind of turnaround for that. Um how we make that s that small team work in this capacity and I'll be kind of explaining the dynamic of that team here just in a sec. Um and then I'm just gonna talk about the a couple of these projects, the salaries database will be one of them, um

8:02

Speaker 1: that I think are just kind of good representations of uh the processes that we we took and how we went from Starting with starting with kind of day one brainstorm meeting and got to final product. And of course we'll be saving some time for questions at the end. And as I'll just kind of add this too. If if there's like I'm as I'll explain in a sec, I'm there's a news apps team and there's a platform team. So if there's very technical questions I may not be able to address. There are three of them right back there. So if they so do not feel afraid to ask those questions because I'm sure one of us will be able to answer it. So news apps, what what what does that even mean? Um so it was

8:47

Speaker 1: actually right at the end of last session, um the 2013 session, that we decided that um the kind of the arrangement we had before is that we had a technology team that was responsible for a whole lot and still are and do a great job. But was getting spread very thin by kind of the editorial demands, but also the demands of the business side, the marketing side, membership And kind of looking at how the rest of the industry was going where they were kind of making these dedicated teams to producing editorial products, we knew that this was something that we also needed to do. Um so uh Becca Aronson, who is one of my um co-workers and one of the co-leads of the News Apps team, I um and uh Travis Swisgood, who is also here, um

9:34

Speaker 1: who was with us at the time, we we kind of split off and and kind of deemed ourselves the news apps team. Um and our job was to be kind of dedicated to what, you know, anything that editorial needed plus um these kind of more editorial focused um data projects like salaries, prisons, and and other ones that came up. Um so we do still have um our kind of tech platform team. Um this is their our the gang back there. They're responsible for building and maintaining our content management system. And they're the ones that work very closely with marketing, membership and business to kind of get their needs done. They've been working tirelessly on the um Tribune Festival um side and and launch.

10:21

Speaker 1: And um also are responsible for keeping the ship afloat. Um they're they're the ones that that keep an eye out on deployments. what we've um and work closely with news apps to kind of if as we as we get to deployment phases with our projects um we'd often um turn to platform often specifically daniel back there um to to make those things happen. Um and I apologize for what's what's next year. I just I I always left out every conference every talk that talks about Docker, which I'm gonna briefly mention it has to have that has to have a ship container thing. So I I wanted to continue. I didn't want to break that that thread. So um there's been a lot of great ship container art that's come out of this product. So, news apps.

11:06

Speaker 1: So, we are actually on the editorial team. So, we are we are technologists that are embedded with the reporters, um, with the multimedia team, with the art department. To do development based around that. This would be a good point where I kind of stress like we're sitting in the same room as the reporters, we're all together. That's also true for the tech team. So um the the Tribune we've we've always tried to really keep those walls um or take those walls down and not let it be a very much kind of Technology is over here and news people are over here. We're all kind of working toward the same goal. Um we're always in conversation, always talking. So, but we're we're the team that's that's kind of dedicated to doing the development work for anything that comes up for editorial.

11:55

Speaker 1: Depending on the time of year, depending on the kind of scope of the projects, we we kind of morph into different kinds of um kind of arrangements. We do also work on graphics that you'll see running in our stories. We build interactives. We do the data explorers again like salaries and prisons. And also work on the kind of the big bigger series features that uh you know are more kind of text or multimedia based, but we're kind of the web development team that that puts those together. Um and we also are responsible for working on um data reporting which is was which is uh definitely kind of Becca and I's background We've we kind of came from being more kind of data reporters for the newsroom and and transitioned into

12:41

Speaker 1: doing more development. But we still do that stuff as well. So as data sort data sets come in, the reporters need help with kind of parsing through a bigger data set or just making sense of it, um that still kind of falls under our um uh tent. Um just to kind of give highlight a couple of them and I'll these these links are all be included at the end just to kind of give a list of things to to kind of touch through. But just examples of of some of the things that we've, you know, we're kind of we focus on. Um faces of Death Row, for example, was our project that looked at the um the kind of looked at everyone who is currently on death row in the state of Texas, um which is is uh I wish I would have looked up the kind of the stats on that.

13:26

Speaker 1: But um At the time, this is an older photo, and I cut of course cut off the number. Um, but more than 200 people are on our on death row in Texas. Um and we we went through the painstaking process of getting um the mugshots for for all those individuals, which was was was something that the uh state agency was not not at at first really really uh wanting to release. But you know, we wanted to to present um this this large group of people that um that are just kind of been some people more than forty years have been just kind of in this system um and and and you know give kind of resurface that conversation. Um kind of more on the interactive side. Um this

14:11

Speaker 1: top right here is our at kind of Just a snapshot of our reservoirs explorer. This is our kind of just live tracking of the state of the reservoirs in Texas. Um this is something that we um not Django, but this is something that we've we've had for three, four years now. Um and we every day get uh fresh data that's that is keeping track of the state of every reservoir in Texas. And this is one of our kind of more wonky but popular apps that a lot of kind of our audience is interested in water issues will will very, very quickly tell me whenever it's out of date. So we have a small but passionate audience for that. And the Texas Legislative Guide, which is at the bottom right, which I'll touch on again in just a little bit

14:57

Speaker 1: , was our our big project at the beginning of this year to um kind of as a combination of data and editorial to to uh kind of approach the coverage of the legislative session a little bit differently. So one thing I wanted to kind of touch on was this kind of leaving the break out breakout of the teams part, is another part of the reason that we have kind of the explicit platform team and news apps team is There's this formal, informal kind of line in the sand for editorial. Um that we, you know, we intentionally keep it that way. Um and while we We all like everyone in our in the organization and we all get along great. It's it's definitely kind of meant to be a um

15:44

Speaker 1: You know, a separate, you know, there's a reason why news apps doesn't do membership and doesn't do business. Um, you know, it's it's it's it's You know, we f we feel better about knowing that that the teams that are dedicated to editorial don't touch any of that stuff. Um and that's that's been kind of our our another big part of breaking out um the two teams into separate kind of groups because we wanted to you know be able to maintain that. And it of course doesn't mean we don't still, you know If technology things come up that you know we want that platform needs help with, we'll help with. Um but we definitely kind of try to stay as far you know as much as we can away from the content aspect of that. Um So um as you know being a talk at DjangoCon, it'd be assumption that we're we're using Django all the time.

16:31

Speaker 1: Um but but no not actually. Uh we're we very much um you know we have just a I just went and found a bunch of random things. We actually do use all of these though, flask, middleman, node. You know, we always want to make sure that we're approaching everything with the right tool. Um it is it is if an app can be a static app and has no database, has no app server, it is a fantastic, good day and I will f I will fight for every ever every project that can be that way. Um because it cuts maintenance down to to practically nothing. Um just hoping that S3 doesn't go down. Um so because the team is so small, you know this is this is really important that we don't let everything kind of blossom into a big um Django or big just any like database driven project because we

17:17

Speaker 1: there's only so much time that we have and we're also kind of on the hook for maintaining all of these pre you know the salaries, prisons, all these other apps that um don't really have an end. Like they will they will continue in perpetuity until I guess the Tribune is no more, which um it's hopefully not anytime soon. And we know we want to try to save the bigger tools, the heavier lifting for the projects that are that you know more suited for that. So of course it kind of pivots into the question of why are we doing Django at all? Um I think Rails is okay. It's fine. Um Flask is is definitely simple and you you know if we need to just kind of have something that's sitting in front of a endpoint, that's great. And static gen

18:02

Speaker 1: site generators have just exploded in the past couple years and it's just kind of continuing down that path. There are some news organizations that will have committed to only doing things that way. And I I think that that's that's that can be a challenge if you're just really refusing to ever touch a a server or a database. But um Um but for us, you know, it's it's the the batteries that are included. That that Django just makes it really easy for us to to turn projects very quick or at minimum explore a project very quick. You know, they're very obvious one, um, Django admin. Um we do, you know, I I I do think that we've in some of our situations we've we've leaned on the admin a little too much, but

18:49

Speaker 1: um But for internal projects it's been fantastic because you get that basic kind of CRUD set up for free. Um we've been able to to kind of do you know, spin up an app really quick just for data entry, um, just for working with reporters um that have you know something that we know we can't scrape, something that we can't do. Um Just kind of process through as like a spreadsheet, but we want to you know ensure that we have kind of the validation um in place for every step of that that process. Um Um often, you know, almost it's if there's a very good chance that if we ever are touching Django, we are using GeoDjango and Post. js in some capacity. Um almost it

19:34

Speaker 1: pretty much anything we've we've done um has this kind of just GIS element to it. So um often we're looking at things at the county level, at the city level, at the um comparing you know the state totals to local totals. Um and you know the real power uh of of this combination is that we we have to s we get to spend a lot less time thinking about how to manage that data, how to turn it to something usable, we can just load it, model out, load it and go. And that's that's that's definitely kind of a a good um this cuts out a lot of that that very distracting figuring out how to organize things. Um many of our um kind of we have so we kind of have two different kinds of kits that we have built out.

20:21

Speaker 1: We have our our Django. Powered kind of templating that or setup that's meant for you know your more traditional um apps where you have the database and you're loading all the data there and then generating pages from that. But we also use kind of our version of our static site generator, which is written in Node that uses a templating language called Nunjux, which is almost identical to Ginja2. It's actually kind of crazy how how they've how it sys it emulates it so closely. But the advantage is that you know there between the Django template language and Jinja 2, there's not, you know, there are differences, but Because they're similar enough, you know, we're able to get designers and other developers to really kind of, when they're needing to build pages, the

21:09

Speaker 1: mental jump is not very far. Um, you know, we've there's that's they can kind of go from working on a static app and looping through data and you know building lists and then come to a Django project and do the very same thing. And it's not not a big loss of time for them to figure out the differences between the two. Certainly comes up, but it's great for just having that that breakout. For me, one of the best things is just being able to do the data modeling and it's probably my my weird favorite part of doing anything in Django is building out the models because that's it uh feels like Legos almost to me for some reason. But you know it 's great because we can we can use this again, it's

21:55

Speaker 1: all those built-ins, all those um kind of affordances that you can get from from just you know modeling out what what the data is gonna look like, all the validators, all the forms get come along with it, um, and then using those forms and validators for the loading of government data. So you know we have you know we know you know we get we can use the built-in Django kind of logging systems to to kind of throw up that flag and say, hey, there's something funny here. This date doesn't have there's only three characters in a year You know, there's examples like that that um you know when we you know doing a lot of things kind of with Excel, sometimes we don't always we wouldn't always necessarily come across those sort of things. It was it would very easy for it just to kind of kind of pass through and and not notice that something was wrong.

22:43

Speaker 1: And because it's so easy to spin these up and so easy to um kind of build out those those those kind of structures for this this you know we can we can we know you know it's worth that extra kind of like hour to build out something um just so we get all the extra um additions. And of course, Django being um you know birthed out of a you know news organization, we've um you know we have a great community that that we can work with and interact with, you know, if there's challenges or if there's um you know just just watching what everyone else is doing. And I I put this up here just because if if anyone knows much about the kind of the news layout or kind of landscape in terms of platforms and things that that that

23:30

Speaker 1: New York Times has always been very traditionally Rails. They use everything. I mean they have there's Go back there and everything. But I was just very excited to see that that Django is starting to make its way in. So hats off to you, Jeremy Bowers. So for how we kind of we approach um our projects, um you know we we we know that often that our timelines are going to be very, very tight. Um it's it's very rare that um there's one exception which I'll get to, but we there's very rare that we have kind of a long like year, six months kind of span that we're gonna get to really work on a project, take the time, tweak it, push back for a month. Um certainly happens, but often you know there's all these other kind of elements that go into it.

24:19

Speaker 1: um or it's just something that we need out quick. Um if if a project is going to not l exist forever, like one of the, you know, one of the major data apps, um, they must have a sunset plan. We have to have a process for um eventually turning that server off. Um because we've is I think we've seen a lot, we even we've experienced and we've seen in a lot of other organiz news organizations that you know the the ease of building it is exciting and and you You get it up and it's running and everyone's excited. But then eventually, you know, some you know if it's not something that's that's top of mind being updated on a regular basis, someone's gonna forget about it. Um and you know, worst case scenario, it goes down. Um other case is it's just bleeding money, um and you're not, you know, no one's

25:06

Speaker 1: no one's really noticing that. So you know we always want to be sure that we have a plan for kind of just turning off the servers, turning off the database, and letting it continue to exist in just kind of a frozen state. So the first of the projects I was gonna just was gonna touch on was the um disappearing Rio Grande. So we um and I'll actually because I can't slip click on that. But um so this project was um one that was pitched to us by a freelance reporter um named Colin McDonald. Um he uh used to work at the San Antonio Express News um and was was had been itching to find some

25:51

Speaker 1: pathway to um traveling all the way down the Rio Grande River. Um and he wanted a place to publish this. He wanted a place that he could kind of maintain kind of a running blog of what his experience was. Um he was going to have um an award-winning photographer traveling with him. Um and he would already had set up a Kickstarter and was was getting that funded and reached out to us and said, Hey, do you want to be the partner on this? Um And when he was at the Express News, he actually um kayaked all the way across the um Texas coast. Like went from bottom to the Louisiana side. Um he hiked across Big Bend in like twelve days. Um so the credentials certainly checked out. He this

26:37

Speaker 1: he was very serious about. um taking um starting all the way up in Colorado and coming all the way to the Gulf. So this was a project that kind of, you know, regrettably kind of snuck up on us because we've we um had kind of went back and forth about the feasibility of getting it um you know being the partner and and how that was going to work. Um this was kind of in that weird little void when we were um kind of the post-transition and figuring out how we were going to kind of own this project. So but you know we knew that this this was something that was going to rise to the level of being a Django powered project. So for the you know what we needed to have is that we needed a way to um, you know, was actually very cool was that Colin was going to do a lot of data collection as he was traveling down the river.

27:25

Speaker 1: Um he was um taking water measurements every night. So every every they were they were camping along the river when they were in places that they were allowed to to camp on or got permission from the private owner um property owners And at every point he was doing a water measurement to kind of test what the state of the river was at that time and at that location So when we were building this out, we had to build something that was that supported um that that was easy for him to to get that data into the system. Um and there was an extra level of, you know, he was gonna be often you know, sleeping in a tent and was not necessarily always going to have a good enough connection to just be sitting there typing numbers into the Django admin. Um so you know it had to be something that he could email off and

28:13

Speaker 1: get um if if if someone to kind of help him you know get all that information in. Um what was probably to me the coolest part of of his um kind of trip was that he had this little little clicker essentially that was a a G sent out a kind of ping up to a satellite that said, this is where I'm at. This is my light long right now. And we were we were able to work with the kind of creators of that system to get access to their API and get that feed. So like as every time he hit a point, we would know within we were checking it probably like every 10, 15 minutes, um we would know within 10, 15 minutes exactly where he was at. And we're using that data to kind of build out these um

28:59

Speaker 1: these kind of bigger interactive maps that that that showed like where he traveled d during that day um and kind of collected into this big like bigger map that was looking at what um you know his whole kind of trajectory as he traveled down the road. We also had to kind of do the media uploads, build sortable galleries, because again, this you know didn't want to waste this fantastic photographer that was traveling with him. And probably one of the biggest things, and this this this will, you know, there's I the one kind of product or like app suggestion I will make and and advocate for was um uh white noise, which has been uh one of our kind of you know great, great tool to have for um for these quicker apps when we don't have to worry about you know

29:46

Speaker 1: building out um static file servers for everything even though it you know it's gonna list exists for two weeks weeks. Um white noise um wraps that up into um when Django's uh serve. So it provides um That uh kind of same functionality but it's both wrapped in. Um it was originally kind of or not permanent originally, but it's been was is always kind of pitch as a good partner with like Heroku um because you get that kind of you don't have to set up all the separate hosting for the static files. But for our purposes it's been great because again it's so simple and straightforward. Um, you know, especially if an app again, it's only gonna exist for a couple weeks or a month. Um, it's not ever gonna really have the load that we know we need to like build out this kind of multi-redundancy.

30:34

Speaker 1: s set up. So this this is one of the stars of every every app these days. But want to just kind of plug it there real quick. The Texas legislative guide, which I showed off showed just a the picture of a little bit earlier, was again like was our most recent kind of bigger Django project. Um and and our approach was that we've we've uh had covered a couple sessions and usually we you know we had our standard stories and coverage that was going along with that and we had built out in the past. Um bill trackers effectively. So using the state's data to um to to kind of like have our own f interface on the state's kind of the state of the bills

31:20

Speaker 1: um in session. which worked and and was good and it was definitely a much more enjoyable kind of way to interact with that build data. But when we were kind of approaching this this this this way but this new session, we you know we wanted to try to figure out a way to do that differently. Because we kind of had approached, you know, we felt like we had approached a lot of those previous sessions, like, okay, build tracker, good, we're done. But but for me, and this was kind of a personal kind of challenge I had with that, was that you know we often in stories would talk about these bill numbers. It would become SB20. Um and then every time a reporter wrote about it, it would be SB20. But if you came into this the session a month or two in, that build number means nothing. You know, there's there's no

32:06

Speaker 1: there's there's you know it's it's disin it it's uh not attached to the actual issue. Like you know, for some people it you know rises to that level that they recognize what that is, but if you're not someone who's following this day-to-day, um that that build number, you know, could potentially you know you have to go hunt and figure out what that actually means. Um and also, as you know, the challenge in Texas is that issues on bills, just because a bill dies doesn't necessarily mean that its issue went away. It can then pop back up on an amendment that you know they found a thread of kind of connection to and then the very same language could end up somewhere else, but now the bill number's different. So we wanted to approach it, you know, again, less less about the bill number, more about the actual issues that we were that that were being covered.

32:51

Speaker 1: And we really wanted to let the reporters be content experts that they are. That that that it you know it was less about kind of trying to aggregate all this data into one place that someone can just go look at. giving a platform to the reporters to actually kind of maintain this evergreen look at what each one of these issues were you know we're going through. So um So like um campus carry, for example, like you know we you know we were covering that very digital you know constantly, but you know the the language and the bill that was doing that kept moving, it kept traveling Um but you know instead of us making about that build number and then having to explain every time that it changed, um, you know, we instead just let it be this kind of block of text that the reporter was curating.

33:39

Speaker 1: um and building kind of this list of stories that they had been writing about it and let that be what informs um you know what's happening with that issue versus um Just kind of having to try to draw all these lines and every single story in like three paragraphs at the beginning of like, it used to be here and now it's here. Um And also, we really wanted this to be something that had a designated end. You know, we wanted this to be something that um that that that was purposely dedicated to being you know a resource at the end of the day. So like at so once session ended um it would be kind of like frozen and and be a source that you can go to and see okay what happened in this session, what what happened with this issue that I was following.

34:29

Speaker 1: kind of conclusions to to those to those things. Um about one minute left. Um so just to touch on quick government Excel Explorer. Um this has been um one of the oldest apps that we have um back to twenty guessing twenty ten actually there's been variations of it that have existed so it's it's kind of actually a question mark when it fur when you would consider it technically um there. But this is the our one that we are constantly updating and maintaining. Um going to give the proper shout out to Travis Westgood and Dan Hill who who did a bulk of kind of the front work on it. Dan and I uh were the ones who kind of closed it out. But um

35:15

Speaker 1: A big part of of what we were kind of going for with this um is is really wanting to abstract out the um kind of the data elements of it and the presentation elements of it, which is actually um two separate Django apps that we we built for this purpose. Um and the goal there was to you know knowingly like approaching it as you know there's a there's a front end kind of element of it, the user facing, reader facing but also wanting to be able to use that data internally and have kind of a the the data app in this case um that could be repurposed for anything that we wanted to hook it into. and not necessarily have it have to travel with the front-end elements of that. I can't get that link going but I'm running out of time anyways.

36:00

Speaker 1: So um but again just to just to touch on what the self-documenting so um the repo and actually all the repos for all these projects I've talked about are actually um open source and on on our GitHub But you know, we wanted to make sure that um like there's code that's written to clean up every single data set that goes into the salaries app. Um and kind of a byproduct of it being a public repo is that we wanted it to be both self-documenting but also transparent so people can see kind of how we walk through um cleaning up each agency's data set and kind of normalizing it to to fit in the kind of the modeling that we had done. And next on our agenda is is redoing our public schools app, if anyone's has seen that, um, which is kind of looking at the um

36:47

Speaker 1: looking at data for every public uh school and district in the state of Texas and letting people find their um find their school and see where it ranks in relation to schools near it and with the state. I was actually hoping to show some of that here today, but we've we I was very I was way too optimistic about both time and also how far we would get along working on that. So um I will just ask everyone to keep an out for it in the coming next couple months. So I'm actually gonna qu fortunately at the end here, so um let anyone ask questions if they'd like. This we'll be making this presentation um public and sharing it, but these are links to all the GitHub projects and um things that mentioned um during this, and thank you.

37:39

Speaker 2: Thank you, Ryan. Uh looks like we have two or three minutes for questions. I have one actually since you've been working with all these government agencies over the last few years, have you noticed any like change in behavior? on their side are are they more kind of aware of like exporting data? Have you noticed like are they getting better at you know being more transparent and making data more available to the public?

38:01

Speaker 1: Yeah, um it's a great question. Yes and no. I think that you know our best example of this is certainly with the salaries app, um which We've we went through a lot of and I was kind of fortunate to be part of a lot of those conversations, but you know we we kind of s when we first started that we had we interacted with a lot of agencies agencies that just said no like we don't that that's not a thing we want to give you They actually can't do that as we as we found. But um and you know it's kind of like you know, we it's I'm trying to think of the best analogy for it. We've done the motions enough that, you know, they kind of understand, you know, okay, look, don't play the game of of oh well that's gonna be hard because we've already fought that battle and here's a list of everyone we fought that battle with

38:49

Speaker 1: and you're you're wasting your time. Um and here's an AG opinion that we've won that battle. Um So, you know, like when we first were going through a lot of these, you know, we would get like like, oh well we have a mainframe. And in in the state of Texas, you actually can charge a lot of money for for getting open records data out of a mainframe. It's It's there because you have to reserve time. Um and I believe it was I want to say it was UT Dallas. I'm not 100% on that. Um actually it wasn't them, but it was another agency that we'd like Essentially asked we got to the point of where like prove that you actually have a mainframe before you charge us you know eight hundred dollars and it comes it turned out they did not have one and did not know that they didn't have one, which was the most amazing part of that. Um

39:34

Speaker 1: They just it was just a charge that they were this knew that they could make and um no one had ever really pushed them on it So a lot of the agencies that we you know interact with now, like the prisoner database, for example, um, that's an arrangement. Like we've that we've already like arranged that. Like they know we can get it You know, they do charge us for it, we pay, it's it's a it's a transaction that's that's understood. Um in general, kind of across the state, I mean I think that I think that you know that this is becoming, and it's not just in Texas, like it's becoming more of a thing. It's more, you know, people are asking for this data, not just even in and news outlets. Um and a lot of places that we've interacted with over the years that used to just, you know, it kind of float if if you don't assign someone as your open records person, it ends up just being like the head of the agency

40:23

Speaker 1: And a lot of these organizations now have dedicated people, dedicated teams to fulfilling them. So again, you know, we're fortunate that in Texas, and this is true in Florida too, we actually have a pretty generous open records law. Um it's it's a lot of kind of no one wants to be, you know, the best way to kind of get back into other people is make things more open um when you're in politics, which is fantastic for journalism. Um and we've definitely taken advantage of that. Um so I I think yeah we've you know there's there's certainly still humps, but um I think it's it's it's become a thing that where we we even have agencies reaching out to us to kind of like, hey, we want to make this available How can we do this better? So, you know, at the end goal, they're wanting to save time and money. Um, we want to do that too. So it it's it's it's um you know, I would never say it's 100%

41:12

Speaker 1: there, but um I definitely have seen improvement over the past couple years

41:17

Speaker 2: Well, thank you, Ryan.

Questions this talk answers

What does the Texas Tribune’s news apps team do?

The team is made up of technologists embedded with reporters, multimedia staff, and designers. It builds editorial web projects such as interactives, graphics, data explorers, feature pages, and tools for data reporting.

Discussed at 11:06

How does the Texas Tribune decide whether to use Django for a project?

The team chooses the tool that best fits the project, preferring static applications when no database or application server is needed because they are easier to maintain. Django is reserved for projects that benefit from a database, fast application development, the admin, forms and validation, or geographic data tools.

Discussed at 16:31

Why does the Texas Tribune use Django?

Django’s built-in features let the small team model data, create CRUD interfaces, validate incoming government data, and get projects running quickly. GeoDjango and PostGIS are especially useful for projects involving counties, cities, and other geographic comparisons.

Discussed at 18:02

Why should a news application have a sunset plan?

Applications that will not be maintained indefinitely need a plan for turning off their servers and databases. Otherwise an abandoned project can break or continue costing money unnoticed; the Tribune may instead leave it available as a frozen archive.

Discussed at 24:19

How did the Texas Tribune build the Disappearing Rio Grande project?

The project used Django to accept field data, including water measurements, even when the reporter had poor connectivity; data could be sent by email. It also consumed a satellite-tracking API to map the reporter’s route and supported media uploads and sortable photo galleries.

Discussed at 27:25

How did the Texas Tribune’s Legislative Guide improve on a traditional bill tracker?

Instead of organizing coverage primarily by bill number, the guide organized it around policy issues, which remain meaningful even when bills die or their language moves into amendments. Reporters maintained evergreen explanatory text and related stories, leaving a useful reference after the legislative session ended.

Discussed at 31:20

Have Texas government agencies become more open about sharing public data?

The speaker says transparency has improved, though obstacles remain. Agencies increasingly have dedicated staff for open-records requests, some now ask how to publish data more effectively, and repeated challenges have made agencies more familiar with the Tribune’s requests and legal rights.

Discussed at 38:01

Presenters

Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.

More videos from DjangoCon US