From junior developer to Django contributor: My open source journey
Published October 13, 2024
This video is from Django Day Copenhagen 2023 in Copenhagen, Denmark.
"Lightning talks" at Django Day Copenhagen 2023. See description and other talks at: https://2023.djangoday.dk/#program
The presenters explain that Python’s garbage collector only needs to inspect container objects, but its stop-the-world pauses can waste CPU when Django loads querysets and related objects; using `iterator()`, scoped GC disabling, or carefully adjusted thresholds can help. They recommend keeping a separate, near-default Django admin for developers alongside a customized back-office interface for data managers. Other lightning talks encourage accessible, colorblind-friendly interfaces, propose Python-based version-controlled permission declarations instead of fragile JSON fixtures, and promote low-effort Django Social meetups. The closing remarks thank contributors and volunteers, point attendees to the developer survey and forthcoming recordings, and outline plans for the next Django Day event and its social activities.
Summarised automatically from the transcript.
Automatically transcribed, so expect mistakes in names and technical terms.
Speaker 1: Take a seat. Then uh we are going to start today uh with Eivar, if I say it properly. Hi Evar. Iverse, okay. Then I give you the talking stick. Five minutes.
Speaker 2: Okay, we have five minutes so quickly. Uh So we all should know by now that Python uses reference counting to release the memory and that is happening most of in most of the cases Uh however there might be some circular references and for that there's the garbage collector. And the most important thing is it's stop the world. It means that none of your code will execute during that. And uh The implementation for that is that there are primitive objects which use just the reference counting like uh numbers and strings and so on, and then this Garbage collector only works for containers, lists, uh maps, dictionaries, sets, classes, and so on.
Speaker 2: So how does it work? Uh if we import GC and ask for thresholds, uh there's this uh Gen Zero threshold is 700. So what is seven hundred? Seven hundred is the number of uh allocated GC collectible objects at the moment. So it's uh every time a GC collectible object is allocated this There's an internal counter being incremented and every time it's deallocated it's decremented. And when it goes over 700, uh GC kick kicks in. So We can do turn on the debugger for GC and then if we allocate 800 strings so nothing happens is because the uh strings follow this reference rule and they do not create uh cyc
Speaker 2: circular references so there's nothing to check. However, if we do that same for 800 dictionaries, then the GC kicks in, it does its work for nothing because there's no nothing to be collected. And this often happens when we load uh large uh large it's 700 is it's not large, but when we load CSV files, JSON files, and everything, so the GC kicks in, does its work. Burns CPU cycles and does nothing good. Uh so what about Django? Well the query set made uh Decision that it will cache the result result uh internally. If you do not want to do that, you have to use this dot iterator.
Speaker 2: So, how GC interacts with this query set? So, here I have a simple model, it's just three fields, like out of field, decimal field, and integer, nothing fancy, and then uh I have this loop which actually kind of doesn't load objects. It just uh the intention is to iterate through 200 objects, so it looks like like kind of stream processing, but in reality because of this caching it loads objects and the get GC kicks in after loading around 200 models. And uh if we do the same with iterator, so there's no J C which is like it's supposed to be. And then if we get some kind of related models, so here I added a foreign key to the client, the same effect is if we if I had like uh
Speaker 2: column with a JSON type. So in this case it's sufficient to load hundred objects if I because I fetch also this related record. So What it means is that this GC threshold for Django RM object is around 200. So as you load 200 Django objects, the GC kicks in, does its work, and burns your CPU cycles and everything else. works slow. What to do with it? So try to use iterator as much as you can. Then you can increase this gen zero threshold but uh that leaves you open for cases when the uh GC will kick in when the memory is already used
Speaker 2: uh in in these circular references too much. My favorite one is this disable G C uh where you can annotate either use it as a context manager or annotate a function so that for the scope of the function GC will not be running and when the function ends so most of the objects will be released anyway because of the reference counting if and if there's some garbage left then uh gc will collect it. And uh there's also another approach is to turn off uh GC completely and this is something that Instagram is doing and they
Speaker 3: Okay, hi. Um yeah, so this is for like small and mid-sized Django projects. Um what's fairly commonly happens is um say you have a client and the client is like some kind of data manager and uh you build them this Django app and they need to administrate the data so you show them The Django admin interface, and they think, oh, this is just perfect. Um like 99% perfect. You know, they want some small things here and there. And so you start adding those things. Like here, for example, this is a uh a JSON field but I added like a custom widget so that instead of the JSON they see this instead, right? So and it's it's true false values for these various uh
Speaker 3: uh attributes. So the client is happy. But what's the problem with this Any idea? Like what is so the I mean and this can just be the beginning, no? Because now the the the data administrator wants some other features, they want to hide some models, maybe they want like It when I when I went into projects that others had built already, it often turns out that like oh if you click the save button it doesn't actually save. Instead it like sends an email to somebody that like it should be saved and Like there's more and more things are happening to try to accommodate for this data admin uh the data manager That make the whole admin interface not work in the default way.
Speaker 3: And so here, for example, if anything in this JSON field is not one of these attributes, it will probably not be shown. Like I can, I as the developer, I cannot Use that any longer. So what I've started doing is this. I uh so this is what I want to see, no? So I can actually see what is the JSON in there. So what I've started doing is instead of Uh modifying the admin interface for them, I simply add a second admin interface. So this is fairly easy to do. You make like a copy of the admin interface. uh class you subclass it and so you have now two so you have back office which I that's for the for the data manager and then admin that's for the developer and those who actually need to have access to like the
Speaker 3: default Django interface or something very close to it and that way I can have two admin interfaces. Yeah. Comments on this? Any other way of doing this? Better ways? Okay, good. Thank you very much.
Speaker 1: Are you last one? Okay, then I think we are missing two presenters. No , who wants to go for jungle social
Speaker 4: Thank you. Good spot whoever saw that. Yeah, okay, thank you. I'm gonna try and do this on time, this one. Okay, so I want to talk about Django Social. Just a quick show of hands, who here has heard of or has been to a Django social event before? Yeah. Cool. Um the idea for this whole thing started actually last year at uh the Django Day event. Um it was the first time that I suppose after a couple of years of being locked inside through the pandemic that I got in a room again with with Django people Love the experience. It was great fun. Um, and it gave me that bug for getting back out, being sociable and seeing people out.
Speaker 4: and about. Um, but I had spent ten years previously um hosting and organizing like a quarterly, monthly ish. meet up in London um which became quite a lot of effort every time to find speakers to find venues um at a sponsor already but it was It was just effort and quite a lot of fatigue by the time we stopped. And so the idea behind Django Social has been to have a an easy-to-run social event. It started off in London. We just went for a walk around. um Regents Park after work. Seven or eight people. Will was there actually, some guy from an eye hospital, a couple of foreign language students, somebody like the film Django Unchained came along.
Speaker 4: And um the idea behind it is that it's just having that in-person interaction in a city near you where Django is the common interest. And the conversation starter. Um, I think a lot of people, I think um Ronnie mentioned earlier that that the cologne Django meetup is the only one that has restarted um in Germany since the pandemic. And I know that's Similar across a lot of places. I know um Ben's mentioned that the Janga Denmark, this is the only meetup that happens really anymore. Um So it this the idea behind Django Social, there's the walk actually. You can see us all there. Oh, Adam was there, Adam Johnson. But we've also done Um we had a summer barbecue, we had a co-working day we've done a couple of times where people just bring their laptop and work in a room with other people again.
Speaker 4: Um with a lunch break and Beers after work, social, we've had a picnic, we've had a shuffleboard night, all sorts of quite easy to run, low admin events. The group itself has grown a little bit organically. It's now held in l in in London, uh, Bristol, Cambridge, Manchester needs a new organiser, Berlin. Um the group is there, just needs somebody to step in and organise the the first meetup. Um Porto, New York, and then this show. have been on the same line. Uh Raleigh Durham, we're having uh a Django social meetup the night before Django Con starts next week. Um and then hopefully they will continue. to to run those monthly afterwards. So why is it so simple?
Speaker 4: Um the idea well organiser fatigue, it's hard work. Meetups with speakers still happen. We're very lucky in London that there is a a monthly ish meet up with speakers with um interesting talks where um you can go and attend and learn but a lot of people c turn up anyway just for the socializing before that interaction the people side of it so um The future of Django Social, we've got a new website being built, but we use meetup. com where you can go on and find any of the groups that are are there. But how can you help? Join the community on meetup. com and find a group near you. If there isn't one, I've listed them all.
Speaker 4: So if there isn't one Um please just get in touch and if you have the inclination or you want to try and get people out and about in your city that like Django that you can meet up with and um collaborate on an event. Um just just get in touch and we can set you up with a group, we can advertise it, um, we can help you to plan your first event. Um And then we'll just ask you to plan another and another and keep it going regularly. Try to run it on a monthly basis. The London one we try and do on the last Thursday so everybody knows so you can plan your working week to be in the city. Um this is how to get in touch with me or with Django Social.
Speaker 1: Thank you. And one talk now about uh color blindness. Yes. And view full screen Have
Speaker 4: I turned that into presentation mode? Clay, maybe? Or would you And I guess just space bar to move it on. All right, thank you. Um the original plan was this was going to be a longer presentation, but uh life kind of got in the way, so it's
Speaker 5: um it's a five minute lightning talk now. So uh you can find me in all the usual places and I'm Mitch J Nich everywhere basically. Um so today this is more an awareness talk. I'd like to ask you to Consider the colour blind. Now there are four lines on here. Hopefully, people everyone can see the top line this time. I did this PyCon UK the other week and no one could see the top line. I've made it slightly lighter this time. It says this is pretty hard to read for most people. You probably wouldn't design your website to look like this. Dart grey on black. Line two and three are also fairly difficult to see if you happen to to be colourblind. And the third line, there are two different colours there.
Speaker 5: The only reason I know there's two different colours there is because I actually formatted it with two different colours myself. Otherwise I wouldn't have a clue, uh frankly. Out of interest, how many people have trouble seeing or distinguishing any of the last three lines there? Is it just me? Anyone else? else no one two one one two three yeah okay no not not just me then no good well not good but You know what I mean. Um so I mean this comes up all the time. Uh terminals can be particularly heinous, red on black, dark blue on black, black on red. Red is always error messages, so that can be particularly problematic. Um
Speaker 5: I've been told, oh just configure your terminal. Uh we can't we don't have options to configure colours in this CLI. And that's very nice, but you know, I've got 27 CLIs on my system and and you know, they all use a different combination. If I change yellow, then doesn't work for something else then, you know. So yeah, think about it if you're designing CLIs, if you're building something like that, just consider who can read it and preferably add some kind of configurability in to make it easy for people to change. change on your specific um application. And it's gone. And I kind of need the visual aids for this. There we go. Yeah.
Speaker 5: There we go. No. You have to stay there now. We'll see. Um yeah, okay, so two examples of of um uh data visualization and things. Left one's from a ticket booking uh website seats in a in a venue that's not a great example um ideally you'd want some other indicator other than other than just Just colours for this kind of thing. As in the one on the right, where you can see, okay, even if I can't see the colours there, this one's on squares, this one's got circles, this one's got triangles and X's, you can actually distinguish those things without having to rely on the colours.
Speaker 5: So there are there are ways around you can use colours but use something else as well. A few resources that are quite useful for this sort of thing, whocanuse. com. Um I'll show you what that looks like. in just one second. Uh colorblind augur on Twitter, they they do show some rather fun and heinous examples of of uh uh problems um from uh And uh wearecolorblind. com. Um incorrect American spelling. Apologies to the Americans if there are any. This is uh who can use, which is quite good. You can put your colour combinations in and it will tell you okay, people with this kind of colour deficiency will have problems seeing it.
Speaker 5: Not just for colour blindness, it's uh I think cataracts, glaucoma, things like that, other kinds of vision deficiencies are further down if you scroll down. It's just a screen copy. So I can't scroll. Um but if you're in any doubt about the colour combinations you're choosing, um chuck them in here and see what it says. It it's uh it's a very useful tool. I think more people should should use it. And finally here's just a few examples. These are all from colorblind. org on Twitter. Don with the lights on the router, that's oh god, I hate these things. They uh hold the button until the green light turns yellow or the yellow light turns green. Oh great, I'll be here till next week. Um Brilliant
Speaker 5: Um yeah, but it it's as you can see there's it's problems all over the place. It's sports sports is quite common and f football teams wearing similar kits and you can't tell who's Who's in possession at any given time? So it shows up everywhere. So yeah, just as I say, more an awareness talk. So please when you when you're building front end visualizations, whatever, please consider the colorblind.
Speaker 1: Thank you. Just put it in top side of my phone. Okay. Okay, hi. Uh second time me on stage. Um
Speaker 6: I built a very small thing um I want to share with you because there was one thing that annoyed me as much as building emails manually and it's about permission permission fixtures um who already have seen these slides. Um so um in most applications the permissions are very very important so they're like an integral part of the um uh of the application And um the um mostly you have many permissions if the system is or the application is a little bigger. And um the permissions have to be synced on every system you have, test, stage, and to all of your developers as well working on the system. So if somebody changes something, it needs to go on the system at some point. And usually
Speaker 6: there are jungle fixtures, it's a JSON file, and it's a huge JSON file. And this will export the ID as well because they are all records in the database. I don't need the ID, I don't want the ID. Um the JSON structure is kind of hard to manipulate, and it's very, very prone to merge conflicts, believe me. And it's really hard to fix them because you always need this importing, exporting crap. And uh oh sorry. Um and um yeah, so um this is kind of hard. So um I um built something um in our company toolbox. There's just many useful things in there. You can have a look. It's called um ambient toolbox. And I try to um use the um
Speaker 6: um use a Pythonic structure for this. Everything is under version control. And um yeah, let me show you how this works. And here basically is an example. This would just uh a file that live in a permissions package inside, for example, the account Django app. And um for the group editor you would have a permission declaration. Permissions always are based on the model. So you have, for example, for the article. You would have these three default permissions and per model, which the editor should have permissions to, you just add one of this, and then in the end you just merge them and um you can then um load them via a management command which you can use in your uh CI
Speaker 6: you can unit test the the command there's a helper for this If you uh want to avoid like putting all the permissions from all your Django apps in one place, um then you can just create uh kind of like um per app uh permission files and just merge them and then when you import them so you don't you have to you don't glue all the context together. And um yeah if you just need all of the default permissions, especially for kind of higher ranking permission levels, groups like managers or so, there's a helper which generates the four default permissions for add, delete, create, and View. Um and um yeah I don't have a slide for the uh management command, but it's very it comes in very handy.
Speaker 6: because then it will just install all the permission fixtures and give you a neat output on what has changed for which groups. And yeah, as I said, there's a helper. You can just add this to your unit test. It's just a one-line and it will automatically test. if you uh if this works and if you made any typos with your um uh with your permissions because if you look at this If you make a typo here or if this permission doesn't exist, then you wouldn't recognize until your code doesn't work. But this checker for the unit test will do such a thing for you. Okay, that's it.
Speaker 1: Thank you very much. And I guess now it's time for another drink.
Speaker 3: Thank you. We have a can can we get a a laptop up here with the closing words? Or are they on this one? Thank you so much. To the pile of computers.
Speaker 4: What can this talk be about? It's called closing words.
Speaker 3: Ah, there we go. Emil, do you wanna do it with me? Grab him grab a mic, come up here. I
Speaker 4: nod. Okay.
Speaker 3: All right.
Speaker 4: You're so polite.
Speaker 7: Sure. So we want to uh to end the day. Um I I haven't said much before any talk about like how amazing the talk is because it was obviously like gonna be an amazing talk uh that's that's why we come here um And that's why we keep doing this. Uh every time it's it's another encouragement. Um but now let's get really explicit about it, right? First slide. Thank you to everyone who got a corporate supporter ticket. And that means Yes.
Speaker 7: And There are six of them. And you can meet them all here. And uh of course this would have been a Kind of weird Dave, we didn't have speakers. Thank you to each and every one of you uh who showed up with the talk. Thank you for taking the time to write the proposal, to prepare the talk, all this invisible work of of creating talks that as like the uh lower part of the iceberg, right?
Speaker 7: So yeah, thank you. And we look forward to publishing your talks so that you can share them. Yes. Everyone that showed up, applaud yourself. Will it and yes. Do you want to say something about uh
Speaker 1: No, I don't want to say anything. You're doing great.
Speaker 7: Okay, okay.
Speaker 1: Keep on going.
Speaker 7: Well there there is one person in the room who can receive your applause and he's Sitting there. Fernando has been doing a streaming and also a lot of preparation uh the day before. So yesterday. If you need someone to do streaming for another event, maybe ask ask Fernando. Yeah, and and to the rest of the venue, of course, also. They're not here to hear this, but we are very grateful to be able to do it here at uh Union and we're very seriously considering doing it here uh next year. Also
Speaker 7: No, we we we wanted to put this little reminder that there is still time to uh fill in the the Django developer survey. Uh so go to the Django project. com and uh give your input there. That helps shape the future of of Django. It's a really important survey and And um you'll also be shared the results back uh afterwards. So yeah, make make a difference. Uh that that's a way to make a difference. Uh it doesn't take so much time. Yep. Um now um the talks will be online very soon. Also thanks to Fernando, like uh to to to edit this very quickly and and make sure that we can uh see it watch it on on YouTube and on another website so yeah please share them when they when they come up
Speaker 7: um Yeah. We don't know like what will be the date of the next Django Day Copenhagen. We know what will be the year. But um but we're gonna sort it out quite soon. Uh we learned from a past mistake to never uh uh announce a date before you have booked the venue. That's also incidentally why uh Uh this year it's in October. We actually tried to make it in in April but got caught up without a venue. Um yeah. Uh if you want to help organize, that would be super nice. Um just like come and ask us. uh we would love to have some help in in doing this uh there's a a bit a bit of work involved
Speaker 7: um i i would say that uh having the venue uh uh is already like a a lot of pieces of the puzzle. Um but of course there is a a a lot of practical things um and organizing can can also be fun. So like don't it's uh it's it's a good balance between like fun and work um so so uh yeah we we would love to uh to see some more people like to have some more shoulders uh to carry the weight um and yeah we will be back next year so uh
Speaker 8: and i would like to add like people who have been helping out today has been like a tremendous help so So um can we have a little applause for uh you please? That's just a good example of like it doesn't take much more than that, you know, just to kind of on the day help out with the the the actual execution. So like someone else can take all the stress of like finding a venue and booking in food and buying beer and et cetera, but just on the day host the lightning talks. Awesome. That's all
Speaker 7: thank you. What's the next slide? Aha. This is like the opposite of Django Social. An announcement that uh yes we have this meetup group. It's a little bit confusing. Uh we didn't wanna delete it because there's so many people in it, but uh we we we don't get any feedback from there. Uh it's kind of kind of weird uh so so don't use don't expect that by signing up here you will somehow uh receive events and news but uh we have a newsletter instead and please do use that um And uh when you bought a ticket, uh you were asked if you wanna receive it, if you said no and now you wanna change your mind, uh you can also find the link to sign up for the newsletter on our social media.
Speaker 7: Profiles. All right. Parte Please uh please get food before the party. We reserve a little bit of time here. It's six o'clock now, so in one and a half hour we uh will expect to be back here. So that should be enough time to go and get something to eat uh nearby. And um the big event during the party is the pop quiz. And the pub quiz has been prepared by Joseph. Super secret stuff.
Speaker 7: You won't know anything about the questions before hand you have to be there um and um aside from that there will be yeah drinks music we'll set up some tables. Uh unfortunately we couldn't get the nice uh uh leather cal uh balloons that you saw in the photo in in in John 's slideshow. Yeah we went to four different shops yesterday and and and they're not selling them anymore. That also means we're not doing a a group photo. after this but we have such a nice group for from last year it's completely impossible to outdo what's that 's the one you you want Is it by popular demand there will be a group photo?
Speaker 7: All right, we will do a group photo.
Speaker 5: Okay. Okay.
Speaker 7: Um yeah, so uh uh yes of course uh payment uh we have uh payment up there uh uh so so you can uh pay with card um and any of the backup options. can be considered um and we close at uh 12. If you want uh to stay at 12 uh the requirement for staying is to help to clean up But but that's that's okay. Yes, then you go home and sleep.
Speaker 8: No more slides. The clicker is uh I can't click more.
Speaker 7: I thought that would be like I'm done. Uh uh finish of the closing of the closing talk will be uh difficult. So
Speaker 8: I think that's it, folks. Thank you. Thank you.
Speaker 7: Thank you.
Speaker 8: Great.
Speaker 7: See you back here very soon. Um if you wanna stay, that also uh requires to help out a little A little bit, um, but do get food.
Speaker 8: Yes. How many people
Python usually frees objects through reference counting, while the stop-the-world garbage collector handles reference cycles. The collector primarily examines container objects such as lists, dictionaries, sets, and class instances, rather than simple objects like numbers and strings.
Discussed at 0:27Django ORM objects are garbage-collectable containers, so they consume the generation-zero threshold quickly. Loading related objects or JSON fields creates more such objects, causing collection to start at around 200 objects instead of the nominal threshold of 700.
Discussed at 3:40Use QuerySet.iterator() to avoid Django’s result caching, and consider temporarily disabling the garbage collector for a tightly scoped function or context. Raising the threshold is another option, but it can delay collection until circular references consume substantial memory.
Discussed at 4:26Create a second admin interface by subclassing or copying the admin configuration. One interface can be tailored for data managers, while the other preserves a default-like interface for developers who need to inspect the underlying data.
Discussed at 6:57Do not rely on color alone: provide configurable color choices in command-line tools and add shapes, patterns, labels, or other indicators to visualizations. This lets users distinguish states even when colors are difficult to see.
Discussed at 15:37WhoCanUse.com lets you enter color combinations and see how they affect people with different color-vision deficiencies and other visual impairments. The speaker recommends using it when unsure about a chosen palette.
Discussed at 18:01Declare permissions in Python files organized by application and keep them under version control. The declarations can be merged and loaded with a management command, avoiding database IDs, difficult JSON manipulation, and frequent merge conflicts.
Discussed at 21:16Use the project’s helper in a unit test to check the declarations automatically. It detects misspelled permissions or permissions that do not exist before they cause problems at runtime.
Discussed at 22:50Note: We understand that names change, people change, and bodies change. We respect each individual's journey and privacy. If you have any concerns about a video or need us to remove content, please don't hesitate to contact us. We will handle your request with care and promptly address any issues.
Published October 13, 2024
Published October 13, 2024
Published October 13, 2024
Published October 13, 2024
Published October 13, 2024
Published October 13, 2024