Unobtrusive internationalisation - Lars van de Kerkhof

Hello, uh my name is Lars and I work for a company called Haibiza. We built mainly web shops and but also regular websites using Wagtail. And I'm going to do a quick talk This talk is called unobtrusive internationalisation. So about that internationalisation, ik denk dat het obtrusief is Right. So what does it actually mean? Internationalization is making your site translatable into another language. So Wagtail has you covered.

In Wagtail internationalisation is built in. And for the page tree that's actually a really great solution. I wouldn't change anything about that. However, it's not always that your entire project is only built in Wagtail. Usually you also have models that are regular plain Django models and these need translation as well. So now, um I'm a bit old here, maybe even the oldest person in this room, because when I started coding Python, we did it with a chisel and the hammer in stone. But the first thing that I started doing when I started coding Python was actually internationalization for Django models.

And this time I think I might have it. So um let's talk about obtrusive internationalization. What I mean is that if you um have to make changes to your code to make models translatable. So in this case this is what Wagtail provides if you want to internationalize plain Django models. So you have to go into your code and Een decorator en een extra bas. Dus dat is obtrusief. Je moet modificaen in de code om. . . Add internationalization. A different method for translation of models is

model translation en in dit geval je moet code like this je moet het in een in een file called translation. py en ook jacken in de app dat je uh want to make internationalized. Yeah, that's it. Alright, so it would be nice if we didn't have to change any code. It would also be nice if we didn't have to change any queries. And it would be even better if the performance would be unchanged. And also if we had proper migrations for all these changes. So some of the things that I just showed you, they don't actually make migrations

voor de field die translateerd. So we need dat And then the last bit we want to make this work for models that are not part of the project code So models that live somewhere inside packages, maybe even models that are built in with Django. We need these nice things because we want to reuse code. We don't want to write code that can only be used in a context of internationalisation. We want to just Right, things that do one thing and reuse them in a lot of projects, if they are internationalized or not And we want to internationalize existing projects.

And we don't want to spend any time doing it. No effort please, that's what we need. So what are you standing here for? Go fix it, Drev. So let's have a look at how it's working. In here here this is uh all the code that's really needed to internationalize a site using our package. So we need to make changes to the databases. In this case we change the engine, the database engine. And we need to add a setting called translated fields. What you're seeing here is first the app label, so that's example, then the model name, it's called example, and then the field name, it's called second

And behind that you have to state if you want to have a fallback. So if you haven't translated the field yet, should we use the value from the main language? What you're also seeing is that we actually uh made the permission name translatable. So these are models that in are in Django and you can't actually translate the permissions. So these will always remain in English no matter what you do. They're not translatable. So, how do we go proceed from that? So this package comes with a couple of commands. The first command is called Make internationalized migrations. I-18N is actually

uh short for internationalization So what happens when we run this command? We generate migrations, but only for The part of the code that deals with internationalization. So here you can see that for the permission model that's built into Django We generate a couple of nieuw fields called name espagnol, name French and name Dutch. Now these migrations, when you take a Standaard approach they will go into the migrations folder that is in your site packages. That's not what we want. Not all sites have the same. So if you look carefully

you can see that the migrations actually go in the project. So the project is called Example. There's a folder I18N Migrations and then a folder for the app That's where the migration goes. Next up we have uh we have to apply the migrations So in this folder I-18N migrations voor alle apps die we translateerde field hebben, we kunnen alleen de migraties die met deze translateerde field Now we we also ship an extra command called A18N Migrate and it will just take the migrations from the original app. Take the migrations van de I18N migrations

folder, combine them and then run all the migrations. So that between if you have changes in your app, bijvoorbe in Django. You will find that out because it says there is a maybe a merge conflict in the migration tree. You need to add a merge migration. Maybe some field was added or something has changed to the field which you translate made translatable. The i make i ITN migration will detect all this and generate new migrations for you. So let's look at the queries that are generated. If you paid close attention you notice that I changed the database engine. And this is actually the principle. So In the database engine lives the query compiler.

In Django you write a query using Python classes. Deze Python-klassen , based on de ORM die je hebt gezogen , willen in SQL statements. En dit is waar we hoek in dit proces. Bep welke lange je kan , de columnname voor de field die translateerd. Dus als de film in de settings is, deze query voor je. So, at the third lijn je kunt zien Coalise Django Flatpage Titel NL en if that is null then please take DjangoFlatpage. title

So this will actually prefer the translated name and if it's not there it will fall back to the title. So in this case I made two fields translatable title and content. Um yeah. So here is why we don't have to change any code. Because we don't need to, because nothing has changed. If you run migrate, it will know nothing about the migrations, only I-18 and migrate will notice, but we do have the migrations. Okay, so here we can see what happens with the permission Aan de leeftijd kan je adres, dan land en kan delete country. En de model is.

On de tweede lijn zegt adres van gebruiker. Dit is Dutch. Maar can add useradres is still Engels. Op de rechter de namel van de permissie, zodat het zegt adres land kan land toevoegen. And this is code that lives in Django and I could still translate it and my migrations are in my project. So we have a lot of tests here and when writing tests we try to find the limits of what we can do with this approach. So here's some weird things that work. Filtering with the underscore underscore this works perfectly fine. Here you can see a test and it tests filtering on translated fields

should work just fine. So first we perform the the test in the base language, then we overwrite the language, change it to Dutch, different numbers come out. The next thing that is in the test is test aggregate. Aggregations on translated fields should work. So here's something very weird. First we make it we make an aggregation by summing rating And then we override with in Dutch and now we can sum over only the Dutch fields. So this query doesn't require any changing nowhere. Some even weirder things that work. So this query we annotate with an uh with an F statement

This is not the context to explain exactly what this means, but it means we put uh into a new variable called class De waarde van de field rating en we add 3 to it. En dan we filteren. Dat should be only rating less than 20. En dan weer de sum van de annotatie. And then we override in Dutch and we do the same and it still works because yeah, we only change the column name when generating the query. This is not rocket science. It it it should still work, so it does. The last test is also a nice feature because we actually have a mechanism to fetch specific translated fields. So what happens here is summing over translated and non-translated fields with F

and L10F should work just fine. So right here we can take the localized value and add it to the value from the base language. That's what happens here. So we take the Dutch value and the English and we sum it and we put it in rating total. So that works as well. Now, we have been using this for two years in production. And if you are interested, please have a chat with me or my colleague Gerit Jan, he's up there. And this is it.