Django Origins (and some things I have built with Django)

This video features Simon Willison at Django Birthday 2015 in Lawrence, Kansas, USA.

Django Origins (and some things I have built with Django)
0:36:10
Published July 16, 2015
1,650 views

Simon Willison
http://www.pyvideo.org/video/3659/some-things-ive-made-with-django
https://djangobirthday.com/talks/#django-things
How "The CMS" became Django and then what happened later.

Summary

Simon Willison traces Django back to his 2003 internship at the Lawrence Journal-World, where he and Adrian Holovaty moved from PHP to Python and built a framework because existing tools did not support their approach to URLs, HTTP, CSS, and rapid development. Django evolved while powering local news sites and ambitious projects such as Little League coverage, then helped Willison build fast, data-driven journalism at The Guardian, including a crowdsourced analysis of MPs’ expenses and the Guardian Data Blog. He also recounts side projects including Django People, Wildlife Near You, DevFort, Bugle, and Lanyrd, along with the practical difficulty of maintaining shipped software. His central argument is that Django matters because it makes it easy to build and ship useful, unconventional things quickly.

Key takeaways

  • Django began at the Lawrence Journal-World as a Python-based replacement for PHP tools that could not support the team’s preferred web practices.
  • The framework was shaped by real applications, including Lawrence.com, weather pages, sports coverage, and interactive local news features.
  • Django enabled rapid newsroom projects such as a map of British National Party membership and crowdsourced analysis of MPs’ expenses.
  • Willison’s side projects show both the creative possibilities of shipping quickly and the long-term burden of maintaining user accounts, infrastructure, and old deployments.
  • He argues that Django’s lasting appeal is that it helps people turn ambitious ideas into working software and release it quickly.

Summarised automatically from the transcript.

Chapters

  1. 0:06 Introduction and Django Origins Simon Willison introduces the talk and explains why he is beginning with Django’s origin story.
  2. 0:54 From PHP to Python He describes taking a year-long internship in Lawrence, Kansas, and moving from PHP and MySQL toward Python.
  3. 3:13 The Lawrence Journal-World He introduces the newspaper’s ambitious technology culture and Rob Curley’s mission to build distinctive products.
  4. 4:52 The First Django Websites He recounts building Six News Lawrence and rebuilding Lawrence.com with the early Django codebase.
  5. 7:11 Django’s Early Philosophy He explains the rapid development of the Little League project, the search for Django’s name, and the “wouldn’t it be cool if” approach.
  6. 11:00 Yahoo and Astro Newsology He discusses leaving Lawrence, learning the limits of research-and-development work, and building Yahoo Astro Newsology.
  7. 13:26 The Guardian Newsroom He describes joining The Guardian and collaborating with journalist Simon Rogers to turn newsroom data into publishable projects.
  8. 17:13 Data Journalism Projects He covers the BNP constituency map, the Guardian’s crowdsourced expenses investigation, and the launch of the Data Blog.
  9. 24:31 Django Community Side Projects He presents Django People and the collaborative Dev Fort projects, including Wildlife Near You and Bugle.
  10. 31:29 Lanyard and Startup Life He explains how Lanyard emerged during his honeymoon, became a startup, and was eventually acquired by Eventbrite.
  11. 33:49 The Tiki Bar and Shipping Software He shows Eventbrite’s production debugging tool and reflects on Django’s value as a way to build and ship useful things.

Transcript

7,298 words · auto-generated Show

Automatically transcribed, so expect mistakes in names and technical terms.

0:06

Speaker 1: Okay. Well, um good afternoon, everyone. Uh so My original talk title, as you'll see on your programs, was Some Things I've Built with Django. But then I realized that we're here in the birthplace of Django, celebrating a 10th birthday of the framework, and nobody's told the origin story yet. So I've switched things around a little bit. I'm gonna talk a about the origin story of Django, and then if I have time, I'll do the sort of self-indulgent bit and talk about some of the projects I've shipped since then. Um I think Jacob's Jacob's introduction sort of hit on some hit on something I've never really realized about myself. I do love shipping things. The follow-up and the sort of long-term thing isn't qu I'm not quite so strong on. And that was sort of came to focus when I was putting together this talk and realized that basically every project I'm to going to show you I had to dig out of the Internet Archive.

0:54

Speaker 1: But that said, um I will tell you the I will tell you the origin story of Django. So um for me um the story starts very much like Jacob's. Um this is I was reading RSS feeds back in 2003 And I saw this um this entry on Adrian's blog um talking about a job opportunity for a web programmer or developer in Lawrence, Kansas. Now I was I'm I'm I was in in England, I was at university, but my university had just given me the opportunity take a year abroad, uh to take a year out to do a sort of um like internship year in industry. My girlfriend at the time was off to Germany to do her year in industry, so I was like, well, you know, do I stay at university? And then this comes along. So I got in touch with Adrian and said, you know, could this work as a sort of year-long internship instead? And he was reading my blog

1:39

Speaker 1: and I was reading his blog and we sort of knew that we aligned on a bunch of things. Um so we thought we'd give it a go. Um now If you look through this job ad, you'll see there that this is all about expert knowledge of PHP and experience designing and maintaining databases, particularly MySQL. So this was a PHP and MySQL gig But um when we moved out, when I I sort of arrived in Kansas and um we quickly realized that we were both kind of kind of over PHP. You know, we'd We'd both built substantial systems in PHP, and we were running up against the limits of what you can do in PHP and have your code still be sort of clean and maintainable. And at the same time, we were both reading um Mark Pilgrim's blog. Mark Pilgrim had been posting a whole bunch of stuff about

2:26

Speaker 1: sorry, I'm just going to try and get uh mirroring turned off, if I can Do that, uh display preferences. And Mark Pilgrim had been publishing Dive into Python and making a really strong case for why Python was a great web language. So, you know, we decided that this was the thing we wanted to do, but we both had very strong opinions about how you should build websites. Things like URL design matters and we care about the difference between GET and post and we want to use this new thing called CSS to lay out our web pages. And none of the existing Python web frameworks really seemed to let us do what we wanted to do Now, before I talk more about that, I'll um back up and talk about the organization we're working for, the Lawrence Journal World. Now David gave a great introduction to why this is an interesting organization.

3:13

Speaker 1: You know, we're talking about a newspaper with a circulation of about 10,000, like a tiny newspaper. But with a world-class newsroom, huge amounts of money being funneled into it, and like employing full-time software developers to work at a local newspaper in In Kansas. And part of what was going on here was this guy. This is Rob Curley. He's been mentioned a few times before already. And yeah, Rob Curley set this unofficial mission statement that we build cool shit. This is something that Adrian would certainly never say. It's not really something I'd say. But this is this is Rob through and through. He was a fantastic showman. And um This was really the appeal of coming out to Lawrence was seeing the stuff they'd already built and the ambitions they had. This is Lawrence. com. This is actually the Lawrence. com written in PHP that Adrian had built as the sole programmer at the Lawrence Journal World

4:02

Speaker 1: And you should check this out. Like to even today, this is the best local entertainment website I have ever seen. This has everything that is happening in the town of Lawrence, Kansas population 100 Every gig, every venue, all of the stuff that's going on. Um and we and it was all written in PHP, and it was kind of a It was a very clean PHP code base, but it was really stretching the limits of what it's sensible to do using PHP 4 back in 2003. So we had this goal when we started using Python. We wanted to eventually rebuild Lawrence. com using Python. But in order to get there, we had to, you know, first build we didn't even know it was a web framework. We called it we called it the CMS And so when we started working on Django, the first thing that we shipped was actually this website.

4:52

Speaker 1: This is the Six News Lawrence. Six News is the TV channel here. Six News Lawrence weather page. And I think this is pretty cool. So Dan Cox, the designer, was a fantastic illustrator. And we actually have this illustration of the famous Lawrence skyline. with um with each panel could be displayed in different um with like different weather conditions depending on the weather. And in case you if you if you're not from Kansas, you might not have realized that the weather is a big deal here. You know, you have never seen more excited weathermen than when there's a tornado warning and they get to like go on local news 24 hours a day giving people updates. So we put the site live first. This was the the first ever uh Django website. Um we then did the rest of the sixnewslawrence. com website. And this um Adrian reminded me this morning um the launch of this was actually delayed by a week

5:40

Speaker 1: Because the most important feature on the website, which is clearly the photograph of the news people who are on TV, they didn't like their hairdo 's. They were like, they they literally told us we couldn't launch the website until they'd had their hair redone, had the headshots retaken, and had a new image put together. But you know, im image is important for these things. Um so anyway, we we did that, we did Six News Lawrence, and by the end of my year in um year uh by by the end of the year I spent out here in Kansas um Adrian had rewritten all of Lawrence. com as well. So this is the Lawrence. com powered by Django. And one thing I think is interesting about this is um when you when you talk to like the David Heinemhansen the um about Rails He'll tell you that Rails is a framework that was extracted from Basecamp.

6:26

Speaker 1: They built Basecamp and then they pulled out the framework that they used and and open sourced it I kind of see Django the other way around. Django is a framework that was built up to create Lawrence. com. Lawrence. com already existed, so we knew what the web framework needed to be able to do. And we just kept on sort of iterating on Django or the CMS until it was ready to produce this this this application this site here. And for me, the moment I realized that we were onto something special was actually when we built this thing. This is a classic Rob Curley project. So Rob The ro Rob was the boss. He had the sort of crazy ideas and he didn't care how you implemented them. He just wanted this thing done. And he came to us one day and said, you know, the kids little league season is about to start. Like kids playing Softball

7:11

Speaker 1: uh I think it w softball or baseball, I you um I whatever the American kids cute kids with with bats thing is. Um So he said, kids little league season's about to start and we are going to go all out. Yankees. We're going to have Player profiles and schedules and photos and results and uh you know we're gonna have um the ability for parents to get SMS notifications whenever their kid scores. And we're gonna have 360 degree like interactive photos of all of the pitches in Lawrence, Kansas, that these kids are playing playing games on. Um they actually did send a a couple of interns out with a like rig to take 360-degree virtual panoramas of um Fenway Park and Lawrence High School and all of these different places.

7:59

Speaker 1: And he said, and it starts in three days, so you've got three days to put this all together And we pulled it off because Django, even at that very early stage, had all of the primitives you needed to build 360 degree interactives. That was that was all down to the interns. But we had all of the pieces we needed to pull this together. So we when we were working on it back then, we called it the CMS. A few years ago, Jacob found a, well Jacob was still at the Journal World, he found a wiki page with all of some of the names that were being brainstormed for the open source release. And some of these are great. There's Brazos, I don't know where that came from. Webbing, physique, Anson. This is my favorite name, I think this is what I propose, is the Tornado Publishing System

8:45

Speaker 1: And the reason is that I was a really big fan of um office space, and if we had the tornado, we could produce TPS reports. We could have a TPS reports feature, which I thought would be amazing. But unfortunately, this being Kansas, the ide the the association of tornadoes isn't actually a positive one. Uh private dancer, FizGig, Lavalia, Pivi. Yeah, um I I'm very, very pleased that they picked the name that they did. Um So we have one of our philosophies was uh was build cool shit. The other philosophy we had was um was what we called a wouldn't it be cool if So basically, there were no user stories or careful sort of specs or anything. We'd all sit around in the basement and then somebody would go,

9:32

Speaker 1: And they'd say something and we'd pretty much if we thought it was a cool idea we'd build it and we'd ship it that day. And my favorite example of would it wouldn't be cool if this is a classic aging one is um wouldn't it be cool if The downloads page on Lawrence. com featured MP3s you could download of local bands. And seeing as we've also got the schedule of when the bands are playing, why don't we h feature the audio from bands who you can go and see that week So this page will say, hey, OK Jones are playing uh on seven uh on uh Thursday at the bottleneck, get their MP3, listen to the radio station. We had a little MP3 widget in there, go and look at their band profile All of this stuff. And you know, this is um really these kinds of features are what you get when you take 1970s relational database technology and use it to power websites, which

10:20

Speaker 1: I'm amazed. Back in 2003, that's still in the news industry, that still felt incredibly cutting edge. But you know, it worked. And um that's That philosophy has sort of followed me through the rest of my career, which is sometimes a good idea and often means that you're left maintaining features that seemed like a good idea at the time and quickly become a massive pain. So anyway, um I left uh I left Lawrence. Uh I just realized I set my timer for 29 hours, not 29 minutes. I was wondering why it wasn't ticking down very fast I have 25 minutes left. Okay. So after I left, after I finished my internship, I finished my degree in England and then ended up joining up with uh Yahoo. Um I was actually working out of the Yahoo

11:06

Speaker 1: UK office, but for a R<unk>D team in the States. And I was there for about a year and a half. One of the things I learned is that you should never go and work for an R<unk>D team because the problem with R<unk>D teams is you never ship. I was there for a year and a half and I basically have nothing to show for it in terms of actual like shipped features. We built some very cool prototype. types. And actually, after I left, one of the projects I worked on, uh, Yahoo Fire Eagle did end up getting spun out and turned into a real product. But generally it was um It was a sort of lesson in in not taking on RD positions. But there is one project, the first project I built at Yahoo using Django that I wanted to demonstrate. This was for Yahoo's internal hack day. And um So Tom Coates and myself, we were working together.

11:52

Speaker 1: We decided that we were going to build a mashup because it was 2005, and mashups were the cutting edge of computer science. So we figured, okay, let's take the two most unlikely Yahoo products and combine them together and see what happens. My original suggestion was that we take Yahoo dating and Yahoo Pets. But I was told that actually there was this thing called Dogster and this other thing called Catster, which already existed and did exactly that. So the next best thing, we went for Yahoo News and Yahoo Horoscopes. And what we ended up building, and again, this is the first Django application within Yahoo, was Yahoo Astro Newsology. And the idea was you um you take the fit news feed from Yahoo News, you pull out anything that looks like it's a celebrity's name look up their birthdate, use that to look up their horoscope, and then combine on them a page.

12:40

Speaker 1: And in a massive stroke of luck, we built this the week that Dick Cheney shot his friend in the face while out hunting And um Dick Cheney's horoscope for that week um says, a very close friend who means a great deal to you has found it necessary to go out of their way to tick you off. You're not just angry, you're furious. Before you let go and let them have it, be sure you're right. Feeling righteous is far better than feeling guilty. And so if if Dick Cheney had only had access to uh had only been reading his horoscopes, maybe that whole situation Would have ended very differently. Um so after Yahoo I spent uh a while doing sort of consulting and things, uh mainly around OpenID because I was determined to make open ID work.

13:26

Speaker 1: I was absolutely convinced that if if if OpenID didn't take off, just one company would end up owning single sign-on for the entire internet and that would be a total disaster. And um with hindsight it didn't quite happen. Facebook login looked like it was going to do that. A few years ago, but these days there's enough sort of variety out there that I I don't feel like we're all we all down to our Facebook masters. Um but after but you know I was enjoying freelancing and consulting and so on. And then I um Ended up going for coffee with somebody who worked for The Guardian. The Guardian, um I'm sure you've all heard of The Guardian. It's one of England's most popular internationally focused uh newspapers. It's a very fine publication. And um I realized that I really missed working in a sort of newsroom environment.

14:12

Speaker 1: And I was incredibly jealous of people like Adrian, who'd gone off to the Washington Post and was doing data journalism there. And uh um Derek Willis as well, who bounced from the Post and the New York Times. There was all of this cool data journalism stuff going on. And the Guardian's pitch was basically We've been building a CMS from scratch in Java with a giant team of engineers, and we've built it and it's really cool, but we're, you know, we're not shipping things quickly. We want to start exploring this idea of building things much faster to fit in with the um with the news cycle. And that was a very, very tempting thing for me to get involved with. So I went to work for The Guardian and um The Guardian have a really interesting way of doing onboarding of new staff. The way they do it is they set you up on sort of coffee dates with people from all over the organization.

15:01

Speaker 1: So one day you'll be having coffee with somebody who sells ads, and the next day it'll be the like the deputy editor of the newsroom, and the next day it'll be a journalist somewhere. And each of these people will sort of talk to you and then they'll suggest other people for you to meet up with So over the first few weeks that you're there, you meet a huge variety of people. And time and time again, as I was talking to people, they were saying, you know what, you should go and talk to Simon Rogers, this journalism in the newsroom. This is Simon Rogers. I went down to talk to him and I we had this fascinating conversation. So Simon is a journalist. He works in the newsroom and his specialty was gathering data for the Guardians Infographic. Because the items are paper. They post, they they have graphs and charts and all sorts of things that they like that they publish. It turns out that Simon was the journalist who knew how to get those that data out of

15:48

Speaker 1: Basically any source you can imagine, if you wanted data, he would make some phone calls, dig into some government government contacts and things, and he'd get those raw numbers. And all of the other journalists thought he was a bit weird because he liked hanging out and editing Excel spreadsheets and stuff So I said to him halfway through this conversation, just out of interest, what do you do with those Excel spreadsheets? And he's like, oh, I keep them all on my hard drive, and showed me this folder with hundreds and hundreds of meticulously researched like news properly like citable news quality spreadsheets full of data about everything you could imagine. And they lived on his hard drive and nowhere else. And I was like, have you ever talked to anyone in the engineering department upstairs? And we sort of made this connection. And um so from then on we had this collaboration going where he would get data and he'd sort of funnel it to me and see if we could

16:37

Speaker 1: see if I or someone else in the engineering department of the Guardian could do something fun with it And so that was some of the most rewarding work of my career because it's it's journalism, you know, it's news, it's stuff that matters. The deadlines are ridiculous. If if a news story breaks and you can you and it takes you three weeks to turn around a an in a piece of data journalism around it, why did you even bother? Um so it's and it's perfect for applying Django to So the first story I got to work on at The Guardian was um was about was a actually one of the early WikiLeaks things. This is before WikiLeaks was was like massively like high profile. Um but quite early on WikiLeaks leaked a list of all of the members of the British National Party, basically the British Nazis.

17:22

Speaker 1: They they they they leaked a list of all of their names and addresses. And The Guardian is an ethical newspaper, so we're not going to just publish 18,000 people's names and addresses, but we wanted to figure out if there was something we could do that would make use of that data but wouldn't be violating anyone's individual privacy. And so what we did is we took all of the addresses, geocoded them, figured out which parliamentary constituency they were they lived in, and used that to generate a heat map that's actually called a coropleth map. I think, of the UK showing where the hotspots of of BNP um activity were. And this works because in the UK a parliamentary constituency is They're designed to all have around about the same population. So if you just like make the color denser for the larger numbers of BMP members, you get this really interesting heat map of the country.

18:10

Speaker 1: And what was really cool about this is that I did this using um using SVG, because we have an infographics department with Illustrator who who are good at working with SVG. And it's very easy with an SVG file with the right sort of class names on things to um set colours on different different regions. And because we'd produced it in SVG, we could then hand it over to the print department and the next day it was out in the paper. It was like print it was a a printed thing on paper on like dead trees distributed all over the country, which I thought was super cool. Um so That was the sort of that was the first data journalism project that we did at The Guardian, and it really helped prove that given Given the right data sets and like the right tools

18:56

Speaker 1: and a bit of freedom, you can do some really cool things. But I think my favorite project I worked on, The Guardian, um was uh oh actually there is a there is a relevant story to that as well. So the first few times I did this, I did it by hand. Then we had the the Guardian's first hack day and I was like, well okay I'm I'm gonna build a little self-service tool for our infographics journalists to like dump in a bunch of CSV numbers and get one of these maps out of it. So I built this tool. I didn't have anywhere to, anywhere official to deploy it. So I just ran it on my Linux desktop under my machine underneath my desk. And they started using it and putting things in the paper and I kind of forgot about it. And every now and then I get a a little feature request. A few years after I left The Guardian, I ran into someone who who worked there and he was like, yeah, you know that that thing that you built

19:44

Speaker 1: So we had to keep your desktop running for six months after you left, and then we had to like convert it into a VMware instance And as far as I know, my desktop is still running as a VMware instance some somewhere in the Guardian. Which sort of ties into the Simon database, I guess. You know, the the hard thing is building building stuff is easy. Keeping it keeping it going, it turns out is surprisingly difficult But this was my favorite project at The Guardian. This was a there was a scandal in the UK a few years ago where it turned out that UK members of parliament had all been fiddling their expenses. And actually the background on this is that they're the lowest paid MPs anywhere in Europe And it seems like the culture had become that you become an MP and on your first day somebody sort of takes you aside and goes, look, I know the salary's terrible, but here's how to fiddle your expenses and make up for it.

20:36

Speaker 1: So this was a scandal that was brewing for several years. The Guardian had actually filed freedom of information requests to try and get these expense reports because they were pretty sure something dodgy was going on. The government had dragged their heels on releasing the documents. And then just when they were a month, a month before they finally released the documents, a rival newspaper, the Telegraph, managed to get hold of a leaked copy of all of these expenses And so the Telegraph had 30 days lead on all of the other newspapers to dig through and try and find the dirt. So when they did release the um expenses 30 days later. We had a race in our hands because we needed to analyze 20,000 odd pages of documents. Actually here it says 450,000 pages of documents. in order to try and find

21:23

Speaker 1: try and find anything left that was newsworthy. And so we tackled this with crowdsourcing. We stuck up a website, we told people Come to the we told Guardian Readers, come to this website, hit the button, we'll show you a random page from someone's expenses, and then you can say tell us if you think it's not interesting, interesting, or we should like sick an investigative reporter on it And um one of the smartest things we did with this is we added a feature where you could put in your postcode, we'd figure out who your MP was, and then we would show you their smug press photo, you know, their sort of smug face next to all of their expense claims that they'd filed. And this was incredibly effective. People were like, ooh, ooh, you look so smug. I'm gonna get you. And um once we we put this up and like within eighteen hours

22:10

Speaker 1: our community had burned through hundreds of thousands of pages of expense documents. Um trying to find this stuff. And again, this was built in Django. We had, I think, five days warning that these documents are coming out. And so it was a total like we I think I built a proof of concept on day one. That was enough to show that it was possible, so I got a team with a a designer and a couple of other people to help out. had it ready to go when the document dump came out um on on on that Friday. And it was pretty successful. We dug up some pretty interesting stories from it. And it was also just a fantastic sort of interactive way of engaging our community and and you know the whole crowdsourcing side of it was was super fun. So I guess the thing I've learned from that is that oh my goodness it's fun working for newspapers.

23:00

Speaker 1: And um actually if you uh the Lawrence Journal World sadly no longer has its own technology team, but there was a period a few years ago where they were doing some cra tracking um data journalism work, things like uh tracking uh what the University of Kansas had been using its private jet for and letting people explore the data around that and so on. Um the other thing we did with the Guardian, this is going back to Simon Rogers, is he had all of these spreadsheets on his hard drive and we're like, okay, we should really try and publish this stuff as raw data because Living on your hard drive under your desk is is is a is a crying what shame. And the idea we came up with was essentially to start something we called the data blog and publish them as Google Spreadsheet. You know, we spent a while thinking, well, you know, what what's the best format to publish these things in?

23:45

Speaker 1: And we're like, well, they're in Excel. Google Spreadsheets exists and is pretty good. Let's just put a few of them up as Google Sheets and see what people do with them. And it turns out that was enough to build this really fun community of data nerds around the Guardian's data blog who would build their own visualizations. They dig into the data, and it meant that we could um we could get all sorts of um like you we could get so much extra value from the work that we were already doing to gather these numbers for the newspaper. Stuff was was that was super fun. Um now while I was working at the Guardian uh I also got into the habit of um ship building projects with my um my girlfriend at the time now my wife uh natalie um because so natalie and i have quite um we have skill sets that uh

24:31

Speaker 1: Fit together very nicely, you know. She's she does uh front uh she's a front-end web developer, I do backend Django stuff, I do just enough ops to be dangerous. And so between the two of us we can we can we can build websites The first things we worked on together is a site which I think some people here should be familiar with called Django People. The idea was just, you know, the Django community appears to be quite big now. Let's try and get people to stick a pin on a map and tell us where they are. Still exists today. It's actually at a jank up at Django people. jjango people. jjangoproject. com thanks to a large number of people constantly bugging me at Django Cons and saying, look, just Give us the code and the data and we'll get it set up somewhere so it can continue to work. And that's great. I'm really glad I did that because this is the one project

25:18

Speaker 1: that I'm showing you today, which is still available on on the web somewhere. But Django Project, it was really fun. And we learned from the thing we learned from this, my wife and I is that we can work work together really well on things. The other side project we did was much more of a collaborative effort. This is probably again this no is no this no longer exists or at least it's no longer up on the web and I'm deeply sad about this because it's my favorite thing I'm going to show you. Um but before I show you the project, I'll show you how we built it. Um we were at a bar camp in London with a bunch of our nerdy friends and somebody showing photographs of this Napoleonic sea fortress that they had rented out for for the weekend. from an organization in the UK called the Landmark Trust, who basically take historic buildings and turn them into vacation rentals

26:06

Speaker 1: as part of the work to restore them Um and we were like, oh, wouldn't it be funny if we like rented a castle for a week and like all of us went out there and we built stuff together And then we were like, that wouldn't be funny, that would be fricking amazing. So we rented a so we rented this place. Um this is called Fort Clonk. It's in the Channel Islands, halfway between England and France, and I think it cost something like $2,000 For the week, but you split that between a dozen people and it's like youth hostel prices to stay in a frickin' fortress. Um So we got a bunch of people together and we went out there and we just spent a week. We called it dev slash dev slash fort, we called it dev fort , we spent a week just building something together. And the thing we ended up building was called Wildlife Near You

26:52

Speaker 1: And what Wildlife Near You does is it solves the eternal question, where is my nearest llama? And again, this is a it's a crowdsourcing uh system. So the idea is that you go to wildlifeNeU. com and you've just been on a trip to like a nature park or a zoo or something, and so you create a trip report saying I went to the red kite feeding station and I saw a common raven and a common buzzard and a red kite and you um import any of your photos from Flickr and so forth. And you build up this profile saying here are all of the places I've been and my favorite animals and things I've seen. And then once we've got that data set, we could solve the problem. You can say, search for llamas near Brighton. And it'll say your nearest llama is eighteen miles away at Ashtown Forest Llama Farm. Um and it'll show you pictures of llamas

27:38

Speaker 1: and All of the Lama things, right? And we had species pages. So here's the Red Panda page. Seventeen people love red pandas. You can see them at Toronga Zoo. Um and then the the the sort of Our most viral feature was we had all of these photos of red pandas, but how do we know which is the best photo of a red panda that we should highlight on the red panda page? So we basically built hot or not for photographs of um of wildlife. So it'd say, which Marmot photo is better? And you'd say, well, clearly the one on the right. And it'd say, okay, which Skunk photo is better and people would I was looking at the logs and people would go through hundreds and hundreds of photos and you'd get scores and you can see oh wow my my marmot photo is the second best Marmot photo on the whole website.

28:23

Speaker 1: Um so that was really fun. And then um we eventually took it a step further and said, Okay, well, this is really fun, but this is a website that you have to type on, right? And meanwhile, mobile phones are now getting HTML5 geolocation and stuff. So um So can we can we go a step further? So we built owlsnearyou. com. And what owl all owl near owlsnearu. com does is you type in a location. And it says, your nearest owl is 49 miles away. It's a spectacled owl at London Zoo. It was spotted one year ago by Natalie. And um if you had went here on a mobile phone, if you went here on a uh device that supported geolocation, it doesn't even ask you where you live. It's just like, oh, okay, here's your nearest owl. And we I think we ship lions near you and monkeys near you and a couple of other domains, but owlsnear

29:10

Speaker 1: you. com was was always my favorite. Um so looking at this now, I we should really get this stuff up and running again. It was frickin' amazing. Like this is this for me is the killer app of of all killer apps. Um so we there have actually been a bunch of DevForts since then. One of the things we learnt from DevFort is that Building applic if you want to do a side project, doing one with user accounts and logins and so on is a frickin' nightmare. It actually took us almost a year after we finished on the fort to finally ship. Wildlife near you because there were so many complexities and then we had to moderate it and keep an eye on it and so on. So if you look at the more recent DevForts, they've kind of taken that to heart and now they try and ship things which just work and don't require ongoing sort of users logging in and all of that kind of rubbish.

29:57

Speaker 1: But one of the other projects I wanted to show you um Uh that came out of a dev fort was something called Bugle. And the idea with Bugle is Bugle is a Twitter-like application for groups of hackers collaborating in a castle fort or other defensive structure Who don't have an internet connection. This was basically to deal with Twitter withdrawal when we were all on the fort together and we didn't we didn't and we had an internal network. So Bugle, looking at it now, we we could have been slack man. We could have we could have been valued at two billion dollars. Um yeah, Bugle is like an internal Twitter clone with a bunch of extra features, like it's got a paste bin and to-do lists and all sorts of stuff like that. And um does anyone here know Ben Firstman? I think quite a few people do. Excellent. So Ben Firstman was out on a Dev

30:42

Speaker 1: Fort and um I did a wood I did a wouldn't it be cool if on him. I said, wouldn't it be cool if all of our Twitter apps and our phones Talk to Bugle instead on the network. And so if you go and look on GitHub, I bet this doesn't work anymore. But he did add magic Twitter support where you could run a local DNS server. redirect Twitter to Bugle and we we cloned what he cloned enough of the Twitter API that like Twitter apps would work and would be able to bugle instead. So that was really fun. And everyone should went to actually we we wanted to do a dev fort in in in America. You don't really have Castles and forts that you can rent for the most part. If anyone knows of one, please come and talk to me. Because um there's a distinct lack of defensible structures, at least of the of the kind that we're used to back in Europe.

31:29

Speaker 1: So I'm running out of time, um, but that's okay because uh the most recent project, uh Lanyard, is something which uh Most people here have probably encountered. I will tell a little bit of the backstory of Lanyard because it's kind of fun. Lanyard actually happened it was a honeymoon project. So my wife and I got married. This is Again, the wildlife new influence, I think, came in and affected our wedding as well. It was a freaking awesome wedding, seriously. You know you can get in England you can get a man with a golden eagle and a barn owl and various other b b birds to show up for about $400 for the day. And and then you get to take photos like that, so it's great. So anyway, we got married, we quit our jobs. I had to leave the Guardian because we wanted to spend the next year

32:14

Speaker 1: year or two of our lives just traveling around the world doing freelancing work on our laptops and so on Uh we got as far as Morocco, we were sort of six months in, when we contracted food poisoning in um in uh uh Casablanca. And we were too sick to keep on traveling. So we figured we needed to like, you know, and it was also Ramadan, so it was really hard to get food and stuff. So we rented an apartment for two weeks and said, okay, well, since we're stuck for two weeks Let's like finish that side project we've been talking about and ship it and see if anyone's interested. So we shipped Lanyard, which was built around the idea of helping people who use Twitter find conferences events to go to What we hadn't realized is that if you build something around Twitter, especially back in 2010, it instantly goes viral amongst people who use Twitter. Um so that ended up sort of cutting our honeymoon short.

33:01

Speaker 1: And we actually applied for Y Combinator from Egypt and ended up spending three years uh building a startup and like hiring people and doing that whole thing. Um the only thing I'll say about that is Everything in the pol Startups have to give the impression that everything's super easy and fun and cool all the time because people say, how's your startup going? And the only correct answer is, oh man, it's amazing, it's doing so well because everyone has to lie about the misery, pain, anguish, and stress that's happening behind the scenes. Um so it was a very interesting three years and we built some cool stuff and we w learned a lot and I don't regret it But um but do not take startups lightly. Um so uh a year and a half ago we ended up um selling Lanyard to Eventbrite and moving out to San Francisco.

33:49

Speaker 1: And um And Eventbrite, I've been mostly sort of on the management team building side of things, but occasionally managing to sneak some code out as well. The one thing I want to show you from Eventbrite, because I really want to open source this thing. is um again at hack day uh we built a thing a tool called the tiki bar, which is essentially like the Django debug toolbar, but it's designed to be run in production. Because the really tough things to debug don't happen in your like dev environment. They happen in production when you're hitting a hundred million row database or whatever And so the tiki bar is designed to add as little overhead as possible, um, but to still give you detailed sort of timelines of SQL queries that are executing and service calls and all of that kind of stuff. It's called the tiki bar because I really like tiki bars. And that Tiki the best feature is

34:35

Speaker 1: if a page takes over 500 milliseconds to load, the greet the eyes on the tiki god glow red in disapproval at you. If you want a demo of that, come and talk to me. I would love to get a few more instrumentation hooks into Django to make this stuff easy um easier. So really this has been kind of a whistle-stop tour of the sort of highlights of my career working with Django. And actually, in putting this presentation together, I realized that really it's that Rob Curley influence from all the way back in 2003. The reason I love Django is it makes it really easy to build cool shit and to and to ship it. And you know, swear words aside, I think that's a a reasonable moral to take away from this. So thank you very much.

35:28

Speaker 1: Do I have time for questions? Um

35:31

Speaker 2: we could take one question, yeah.

35:32

Speaker 1: Okay.

35:33

Speaker 2: Question? Would you consider missile silos?

35:37

Speaker 1: Oh yes. Uh the question was, would you consider missile silos for a dev fort? The answer is oh hell yes. If if you can find us a missile silo that we can fit a dozen people in.

35:48

Speaker 2: There's a dozen here in Kansas.

35:50

Speaker 1: That's and they're rentable? That's amazing.

35:53

Speaker 2: We can take one more question. All right, well, thanks Simon.

Questions this talk answers

How did Django originate?

Simon Willison and Adrian Holovaty created Django while working at the Lawrence Journal-World in 2003. They had moved beyond PHP and wanted a Python web framework that supported their preferences for clean URLs, correct GET/POST semantics, and CSS-based layouts.

Discussed at 0:54

What was the first website built with Django?

The first Django site was the Six News Lawrence weather page, followed by the rest of the Six News Lawrence site. By the end of Willison’s internship, Django had also powered a rebuilt Lawrence.com.

Discussed at 4:52

How was Django’s origin different from Rails’ origin?

Rails was extracted from Basecamp after the application was built. Django was developed in the opposite direction: the team built and iterated on the framework while creating Lawrence.com, whose requirements shaped what the framework needed to do.

Discussed at 6:26

How did the Guardian use Django to investigate MPs’ expenses?

The Guardian built a crowdsourcing site that showed readers random pages from roughly 20,000 or more expense documents and asked them to flag interesting material. Users could enter their postcode to see their own MP’s claims, and the community reviewed hundreds of thousands of pages within about 18 hours.

Discussed at 21:23

What was the Guardian Data Blog and how did it work?

The Data Blog published the newsroom’s previously private spreadsheets as Google Spreadsheets. This created a community of data enthusiasts who reused the datasets to make their own visualizations and analyses.

Discussed at 23:45

What did Wildlife Near You do?

Wildlife Near You let people record trips, animals they had seen, and related Flickr photos, then used that crowdsourced data to answer questions such as where the nearest llama was. It also included species pages and community voting to rank wildlife photographs.

Discussed at 26:52

What was Lanyard built for?

Lanyard helped people who used Twitter find conferences and other events to attend. Willison and his wife built and launched it while stuck in Morocco with food poisoning, and its Twitter-based design helped it spread quickly among Twitter users.

Discussed at 32:14

What is the Tiki Bar Django tool?

The Tiki Bar is a production debugging tool modeled on Django Debug Toolbar. It is designed to add little overhead while showing timelines for SQL queries and service calls, including a visual warning when a page takes more than 500 milliseconds to load.

Discussed at 33:49

Presenters

Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.

More videos by Simon Willison

More videos from Django Birthday