The art of (not) redirecting with Lorenzo Peña

Speaker 1: So, hello everyone. Yes, my name is Lorenzo Peña. I am originally from Olgin, Cuba, but for the last three years I have been living in Munich, Germany, where I have a two-year-old girl that is as beautiful as superactive And she's waiting for me and cannot wait to get back to her. For the past 15 years, I have been doing uh Python and more specifically Django. And right now in Munich I am working in a uh as a jungle developer in a Munich-based uh startup that is called Alasco. And we're trying to digitize the construction industry in Germany, in Europe, in order to make the whole industry more carbon efficient. So I would like you

Speaker 1: today to take a deep breath, take a look at this URL, and search in your feelings. How do you feel about it? If I may offer you some emoji recommendations, would you pick one of these please Alright, the reason why this emoji list is so biased is because it's based on my own emotions. And when I see a URL like this, it's far from joy. what I feel. I was wondering if we if it was a situation that uh only I had, so I had to take the things online and I made a a poll. on X and the question was basically when you use a web application do you care about how the URLs look? If you were to answer that today, what would you say with a show of hands? Would you care?

Speaker 1: Alright, thanks. Now we do care, but when we see one of these things, who should be responsible of in the first time creating them and in the second time making it a bit better? Who should be responsible for that? So I also had to take the things online. Unfortunately, this time the statistically significance of the results are not too high. So I would also like to have a show of hands here. Who right now in the web world is in charge of designing, deciding what the URLs look like? Would you say is the product people? Uh designers perhaps? Maybe us, uh front end, back end developers.

Speaker 1: All right, or maybe so something else, maybe the framework. Anyways. Well, you may be familiar perhaps with a bunch of memes about the difference between front-end and back-end. But when it comes to URLs. It's not specifically in any of these categories. It's more about the architecture of your system, the care you put on the code. Because your else, whether we want it or not, is a one part that is very visible about our code in the in the top bar of our application. So if we want to get really dramatic about it, when you look at the URL of a web application, you're looking at a mirror of the soul of the code of that application. And 25 years ago, when URLs were a pretty new thing, Tim Berners-Lee, the man who invented them

Speaker 1: had it very clear and thankfully the results of the poll more or less align how he envisioned the whole the whole matter around URLs to be in the future. So yes, he's saying uh in this article that is called Cool Eurists Don't Change. Back in 1998, that is the responsibility of a webmaster to design good URLs. And right now we unfortunately don't have that title anymore, but We software developers, web developers would be uh the ones considered to be the webmasters of today. And this requires, in his opinion, a lot of thought, organization, and commitment because designing URLs is a task that demands all of that. But to make matters a bit worse and a bit more complicated for us, Tim says in that article

Speaker 1: that the moment we put a URL out there, The moment we put a URL out there, we're meant to never break them because you never know who has a copy of that URL. And the fact that we can put a QR code like this, which points to that very article Under the same URL that was written so many years ago proves the point and serves as a beautiful example of what's what he's talking about, the immutability in a way of URLs. This, as a theory, is Very nice. One can melt when when one sees such a beautiful theory. But in practice, things are a bit more complicated. Because we know that naming things is hard and URLs are not an exception to that. And not only that, it's also that name things evolve in time.

Speaker 1: And the products evolve also in time and with them the URLs. Things that had a name today could tomorrow be merged into two different things or perhaps divided. And so it's natural that the name of things sometimes change So if we were to take uh Tim 's advice, Burbatim, it would be a very hard to impossible task to do. So there must be a middle ground for that. And I would like to make it perhaps a bit more practical in terms. And that would be design your else as best as you can and break them as little as possible. These could also be interpreted as this. We have to learn to love our ear elves. We have to learn to love them.

Speaker 1: And We also need to avoid 404s at all costs. Of course, one very rude way of abo of avoiding 404 at all costs would be to just return 4010, it's gone. And then we just leave the user hanging or we could just gaslight the user in a way return 422 unprocessable entity and make them think it's a problem of their own. But no, there is actually a better way. If we are supposed to never or avoid 404s at all costs, there is a much better way, and that way is predirects Redirects are everywhere. And what is a redirect? It's a response from the server that is coming with a specific response code, but it's also containing a location the user agent is meant to

Speaker 1: uh redirect to or to go instead of the other place that was visited. And we use them on a daily basis. Some some some user visits our website using HTTP and we have HTTPS available. What do we do? We redirect Or somebody perhaps visits a page that requires authentication and they are not authenticated, what do they get? A redirection. And same when they submit a post form and we redirect them to a different place so that A hidden refresh on the page doesn't resubmit the whole thing. But also when we do, for instance, short URLs for the sake of capturing the essence for people, or maybe analytics inside our sites. So that we can keep track of what pages are people visiting from our web applications.

Speaker 1: So there are a few status codes that are very redirect specific and Come in our help on a daily basis. And these you're probably familiar with this. These are 30302 found and 301 move permanently. The first is the one that we we use the most. For instance, every time a user posts Something using a form, they get a 302 and they get redirected to somewhere else, and this tells them that this redirection is not permanent in its nature. Maybe the next time you come to the same place you'll get redirected to somewhere else But then there is also 301, which means if you ever come back to this URL, you should expect that you will be taken to the same place because it's more of a permanent thing And this is more or less getting close to what we should be doing with URLs that are maybe not useful anymore.

Speaker 1: But I don't know if you know that there's also 307 and 308. And this, I think these are now capturing the essence that 302 and 301 were meant to have. And the the main difference in the semantics of these two is that they tell the user agent to respect the HTTP verb that was originally. uh use on the on the on the main request. So if you do a put and you get a redirect, the user agent is also expected to do a put to the second URL So this would be more useful when we want to deal with URLs that at some point are not going to be useful anymore, in this case the permanent one. However, there are some status codes that should exist, but sadly they don't.

Speaker 1: One of them is redirect and replace user bookmarks. How we wished there was a way to hack into what the user has bookmarked in their browsers in order to change into something better perhaps. But we cannot do that for obvious security and privacy reasons. Another one is 310 recursion base case. I don't know if you haven't been part of too many redirects problem, but maybe we just need uh a code for the base case, that's what it's missing. And the last one that should definitely exist by now is 311 never gonna give you up. If by any chance you don't know what that is. Break a leg, follow the link, just make your volume loud, no, not loud, uh loud, so that we don't all get alerted.

Speaker 1: So, yes, redirects. And here's a picture. That's me in the middle. Biting my nails, surrounded by colleagues, which are using the emoji-based identity protection. And yeah, the reason why they have such uh looks in their face is you may have guessed it because that URL. And at Alaska at some point our product had been evolving uh too too fast in a way And they the the quality of the product in the body of the document uh wasn't had nothing to do with the quality of the URLs that were reinput put uh above. So we have to come with a plan

Speaker 1: and we have to try to make the whole things better. We had to try to to make to to do a full refactor of our URL schemes and that was the idea. we embark into doing. Imagine this was the idea we had. Imagine if we could just copy all our URL structure as we have it right now into a separate file and you have to imagine that a single file will contain all the URLs. And then at some point take these new URLs and try to reorganize them in a way that it's more suitable to the experience the user is having with the product. Then at some point take all the old ones and make them redirect to the new ones. in a permanent way. And the less traffic they get, the better chances we have to retard them one by one until at some point

Speaker 1: all the URLs are the new ones. So we embarked in this journey and we actually did it. Did it work? Yes. It worked so well I was so proud of the work we have done that I actually prepared a mini talk and I gave in a in a meetup at my company. And what do you think was the only question I got at the end of the talk? Was it worth it? And my answer was yes, absolutely yes. It's so much joyful right now when I can experience the product. We put so much care into building, but also we take a look. at what the URLs are looking above and how our product is being routed in a way. And it's so aligned, so in harmony, so in synthony.

Speaker 1: So let me give you the results of our experience in that process in the form of the art of not redirecting or how to design the URLs because we had to go through the process of designing only URLs. And we use a bunch of basic principles. Number one, we use the sen of Python. It's amazing the fact that we have a sen as a language. And it's incredible how much applicability this Zen has even outside of Python, even outside of programming. If you were to read at least the first parts of the Zen, you would find a very helpful guidance into designing good URLs. And the other principle we use is the breadcrumbs principle. I know that it's not too uh fancy anymore to have in our websites these breadcrumbs that were

Speaker 1: uh the coolest thing a few years ago, but the fact that they existed at some point is very helpful in conceiving that the way these breadcrumbs works is aligned to the way the users experience the product. and had nothing to do with what the logic is behind, what the code is doing behind for their product. So what they're looking also above has to do a lot with what they are experiencing experiencing But then apart from the basic principles, we actually want to make that very concrete. And I'm just gonna give you a bunch of very opinionated prescriptions on how to design good URLs. Number one, they have to be readable. Because user at some point will try to read them, some people will even type them from memory.

Speaker 1: And if they are not readable, they're gonna have a really hard time. But they should also be predictable. Because if I am a user and I am able to guest navigate the site just by rewriting URLs. I see what your menu is and I'm capable of inferring what the URL is gonna be for a page that I haven't clicked on that link, that's a beautiful experience for me as a user. Maybe not everyone is as freak as I am in that regard. But it's also a very good thing to have. They also should be concise because there is absolutely no use in having so much redundance and this Sometimes happens when we try to include your L patterns one inside, we lose track of where we're coming from. And then sometimes it's hard to detect that we're incurring in redundancies.

Speaker 1: But we do ourselves a favor by keeping things very concise They also be as much as possible complete. Every part need to lead needs to lead to somewhere. And if there is no official page, for instance in this case, user settings does not exist. But there is security, there is users at least, we should make that URL redirect into one of the nested levels. They also should be consistent, and consistency is so important. I would say the moment there is a chance the web application goes international, it's no not too risky to say that English should be the language used because then trying to apply Spanish. or maybe German or maybe Portuguese in URLs to an international audience is going to be a bit hard.

Speaker 1: But not only the language should be consistent, should also the style be as such. And they also should be beautiful. And what is beautiful in a URL? Is it the fact that we can use camel case versus kebab? The fact that we have trailing slashes or not? Well, beauty is in part in the eye of the beholder. And if we at least if we cannot agree on what beautiful is, at least let's not make that ugly. Avoid file extensions and file extensions as much as you can And if possible, also query parameter-based navigation, which is not the way URLs were intended. And if you dare to put commas on your URL

Speaker 1: on your URLs The Django documentation saw you from the past and have very strong words for you. So please don't do it. So in short, we have to learn to love our ERLs. But the fact that we are capable of designing them well doesn't help us from the fact that maybe the current situation is not that good. So let me also talk about the art of redirecting or how to avoid 404s at all costs. We have a framework that is batteries included, and we have Django Contrib redirects Is something that comes with Django. And the idea is we can store in a database old URL, new URL, and Django takes care of the rest. If for some reason we get or we are about to get a 404, then Django searches

Speaker 1: In the redirects table, and if there's something that can be redirected, it will do it. However, this is only useful if URLs can be expressed very statically, if there is no dynamic component to them. The moment there is some the dynamic part of the URL then we have to get a little bit nerdy because the read arrest is not gonna be enough for us. So in our case we had the situation that we had created a couple of parallels URL structures. We had the old and the new, but because they were referring to the same views and they were sharing the same names, the only difference is how the paths were actually looking like We had to namespace the old ones so that we could have better control of the redirects and this is just an example of of of a real thing we had.

Speaker 1: This was an old URL that was a reflection of a product that didn't exist anymore, and then the new URL that was very aligned with the experience of the user. How would we do this? And the answer would be middlewares. Yeah, middleware is the the perfect way to do this redirection. So we need a middleware that is capable to redirect The old URLs into the new ones. And this is the basic syntax or structure of a function-based middleware And the two things to notice here is a shoot go-to-new function that takes the request and the response that was normally generated and based on that it decides whether to redirect to new or none However, we have a little situation here

Speaker 1: because we know that 307 and 308 exist, but apparently Django doesn't. And it would be a very interesting perhaps sprints project to make Django aware that these two types of free redirections exist and maybe even enhance the redirect shortcut to be able of not just saying permanent yes or no, which is 301, 302, but also respect the HTTP verb in a way so that it gets 307, 308. We're not gonna do that, but we are gonna take a look at this should go to new function What should be inside here? So it's a function that takes the request that is in common plus the response that was generated, and then we have to decide. Do we want to handle this response, yes or no? And the URL we found, does it belong to the O1, yes or no?

Speaker 1: And then we have to decide from the O1, which is the new one that will be the equivalent. So, yes, these are the three things we have to decide. Let's put it to our tool list and let's try to implement the we want to handle function. What could be reasons that we don't want to handle a response that is already generated? Well, if the response is a redirect itself, we certainly don't want to handle that anymore. We can leave that alone. But maybe the response was comp was was expensive to compute and because computing the response and then redirecting to a page that is gonna render the same thing is gonna be a waste of time. So let's not do that Regarding whether the URL is old or not, we have a magic component in in Django, which is the resolver match.

Speaker 1: Whenever the resolution mechanism has taken effect, the request is augmented with this resolver match, and there is a bunch of very nice things that we can ask this resolver match in order to know what's happening with the with the view that that got uh resolved. And we have namespaces. The fact that we put all your else as a namespace before is coming very handy now because this is how we determine That is an old one. And then finally, the find new URL. This again we're using the resolver match. And because we have the name that was matched, and we have the arguments and the keyword arguments, we're able to just reverse To that same name without the namespace, and this is how we get the new URL. There are two things that are missing here: the query parameters that should also be redirected, and the fact that we have two parallel structures and we have to keep them in sync.

Speaker 1: So we need to capture a non-reverse match. So by doing this, we kind of solve the situation that we had a problem before and we created a way to transition all into new without breaking the user experience. However, this doesn't help us in future evolutions of our software and we cannot expect that we're always gonna have these two parallel structures forever. So what if, what if, what if we could also express this is a vanilla path definition in a URL URL conf What if we could have an extended version of this path that just takes a bunch of old URLs that are now meant to be redirected into a new one? This would be a nice way to keep future enhancements going. So here we go again

Speaker 1: and let's try now with path with alt. This function is exactly the same path Django is providing. It's only receiving an alt. And let's try to build this function one step at a time. So we know we're gonna be handling multiple paths here, so let's get ready for that. And if there is a name and if there are all URLs that we're gonna be redirecting, is the only way we can do something here because otherwise without a name we wouldn't be able to find the proper URL. And if it turns out that we find and we have multiple paths here, we're gonna have to include them and if not we can just return what we had what we have as if it had been a vanilla jjango path. So the core of the function is well we're gonna need a redirect view that is computed on the fly.

Speaker 1: And then With that view at hand, we can create a bunch of paths on the fly that use that view and redirect into the original view that we had above. So a redirect view on the fly would look like this. We receive the name that we are meant to redirect and we use our magic resolver match again. In this case, we compute the full name and finally we do a redirection. Again, this redirect needs a bit of improvement and also don't forget the query parameters And this is how we can be very top-notch in the game of redirecting. So to summarize, we gotta learn to love

Speaker 1: URLs, we have to design them as best as we can. We have to try to break them as little as possible. And by doing that, we are very engaged in the art of redirecting, or maybe not. Thank you so much. If you want to keep in touch with me, those are more QR codes and the slides are there.

Speaker 2: Um thank you very much. Um I read your write-up of this talk. Oops, because you said don't break URLs and I do that all the time in documentation because every time you refactor documentation there's a chance that you're going to change the where something is. And I tell the people the technical authors I work with, don't worry about it. In a couple of days' time the search engine will find it and the navigation of the of the documentation website will find it and so will the search. So am I am I wrong? Am I do you should I not be doing that? We

Speaker 1: have to be pragmatic and we do it all the time. But in my opinion, the job as a good web citizen is to try to break the experience as less as possible, but just make it uh with uh trying to be wise on when when to stop because at some point there's not there is no point in in preserving this legacy URLs any further So use judgment, I would say, and try to be the best web citizen that you can.

Speaker 3: All right, we got about five minutes, but thank you, Lorenzo. That was wonderful. Let's give him a big look.