The evolution of a Django Website into a radio automation back-end with Ernesto Rico Schmidt

Hello and welcome to the evolution of a website into a Ray automation backend. I want to tell you about how my first Big Django project has evolved over more than a decade. I'm going to try something different this time. Please bear with me. So let's go I am Ernesto Rico Schmidt. Rico is not my middle name. I am an electric engineer, internet software developer. I was born in Bolivia where I'm currently living after studying and spending half of my life in Graz, Austria. I've been using Linux and free software for almost 30 years.

I have programmed in all sorts of programming languages since I was studying. I discovered Python around 2003 and started using Django around 2008. I've been doing it regularly since 2011. I have recently started to learn Go. I work as a remote software developer for Radio High Synchron Grads. one of 14 free radios in Austria. I needed three attempts and it took me more than a year, but I translated Doc Hellman's Python 3 model of the week into Spanish. Free radius started In Austria as in other places all over the world

as Pirate Radios, broadcasting without a license until in 1997, when a new regional radio law was passed and and it ended the state broadcasting monopoly. Now they are recognized as non-commercial radios and received some funding from the states or they ordered their cities A free radio has fundamental differences from a commercial one. It is non-commercial and community driven, and as such it serves very diverse communities that don't have access to commercial broadcasting radios. Radio thinking Rats started to broadcast in 1999

for a month during the art festival known as the Starship Herz. In 2000, Radio Helsinki obtained its first license to broadcast 24 hours. Back then almost every show was broadcasted live. Now about 80% of the shows are either pre-programmed or rebroadcast from other radios. This is part of the program from Ryan Helsinki from June to September 2023. It's difficult to distinguish. them but the font colors signal the different type of show and the different color bars signal the rhythms they follow

This is the first challenge that commercial radio broadcasting software faces The initial goal of the project PV for programmab or program management at Ray Hersinki was to display the shows and the schedule in a daily and weekly overviews and to integrate them into the website. This led to a mixed solution in 2011. The website for ReHersing was powered by Clone and included all the program as HTML pages from the Django packet. This sounded way cooler to me back then and was something I was really proud of.

Please be kind for a minute and forget that rest was already a thing at that time. Yet another radio manager or yarn was originally developed at the radio fabric in Salzburg. one of Austria's free radios. It is currently in use at six free radios in Austria despite the development of the software being abandoned in 2014. YARM is a monolithic Java application that has one fundamental flow. It doesn't offer a remote interface. That has led to some radio stations to ask their host to upload their pre-recorded shows

and then to have The program managements download and manu scale them. This is the second challenge that commercial radio broadcasting software faces. At Radio Helsinki we use Rivendel, another open source radio management solution to automate almost everything in the program. Currently the main limitation is that if we are broadcasting some live stream from a remote location, we need somebody to be present in the studio all the time to start.

and monitor the stream. We have developed a simple interface for the host to upload and schedule their shows remotely. Currently at Radio Hersing we have about 120 shows that are pre -recorded or re-broadcast This situation led some of the free radio in Austria to start thinking about the joint project to tackle all these requirements that are more or less common to all free radios by developing a new open source software suite Then in 2017, the project Aura started based on the Redu Helsinki

's PV for program management and other components contributed by other venues. From the beginning of the project, it was envisioned with a distributed architecture in order to avoid the limitations that the arm has. This is the origin of the naming conventions of the components. Auto is one of the German words for car The different components of Aura communicate over their HTTP APIs The whole suite is made up of modular components which could be exchanged with other components if needed.

Every component provides a well-documented API. At the moment, the engine only the engine is developed with the API first approach. The documentations for the steering and the tank APIs are generated from the annotated code base. At the core of Aura is steering, the source of truth for the Quran. Steering stores the information about the program and it offers a REST API to manage this information It is intended for driving the broadcasts and to provide the data to be displayed on the radio station's website. Steering

also acts as an open ID connect provider for other components and can be integrated with LDAP if needed. The tank stores the playlist and the media for the playout for the shows including metadata if available. A playlist can contain a combination of media files and other sources like lane in or streams. The engine is the main playout component. It is in charge of playing the media and switching between the sources or studios according to the playlist

stored in the tank. The engine provides public and internal APIs and uses the liquid SOAP language for the playout. The engine also includes silence detection and optionally records the output. The dashboard provides the user interface to store the data in steering and tank. It currently allows program managers to create, update and delete shows and schedules. It also allows the host of the show to manage and organize their media and playlists. and annotate the time plots.

It will offer a different set of features and permissions depending on roles and privileges that can be configured individually by each radio station. Finally, there is plate. Play is a library of web components that can be integrated into the radio station's website to display the program, including what is currently playing. The program is modeled using shows, schedules, and time slots. The show part is obvious.

It includes the name, the description, the host and taxonomy like the category, the type or the language of the show. The schedule describes the rhythm of the repetitions of the show according to the recurrence rules. It is used to generate the time plots. A time slot represents the specific time that is assigned to a show. Recurrence rules govern how a show repeats in which rhythm, how often and how long it repeats. They are basically a wrapper wrapper around the

utiliser rule object Currently we support a total of 31 recurrent tools, including once, daily, weekly, p -weekly, four-weekly and monthly, Pmonthly, 3 monthly and 4 monthly. Most of them are not very useful for programming managers or even for listeners, but we support them anyway. For example, radiohealth sync in Grad supports only weekly, p-weekly and four-weekly recurrences. Rayo Ranch in Vienna supports weekly, bi-weekly, and monthly recurrences.

Now a small detour. Initially we supported what we called a pea weekly every odd number weeks and b weekly every even number week recurrences. It turns out this is a very odd recurrence rule. Why? Because every seven years approximately seven years the year has 53 weeks. 53 weeks This is according to the International Organization for Standardizations, the ESO. A week that has at least four days counts as a whole week. That means that if the first week of a year starts on a Thursday, it counts as a whole week and the result

is a 53 week year. The next time will be in 2026. The first day and the last day is a Thursday. This results in the first week of 2027 being outnumbered after the last week of 2026 also being outnumbered. That means if you have a show scheduled on odd numbered weeks, you will get a p-weekly show scheduled two weeks in a row Or if you have a show scheduled on even number of weeks, you will get no b-weekly show scheduled for three weeks. Fun isn't it?

Now let's go back The detection and solution of time slot collisions was a major headache with the original Django project. that to manage or better say try to manage everything in the admin interface. I was not part of the Aura project back then but luckily Ingolendeker studied the collisions that can arise and contributed to the detection and solution of time float collisions. Let's take a look at them The first type is a fully overlapping collision. Here we have a projected time slot

and an existing time slot that fully overlap. There are two possible solutions for this collision. The first one is what we call terris. We keep the existing time slot and discard the projected one. The other world pul possible solution is what we call ours. We delete the existing time plot and create the one that we projected. These both solutions are always possible A second type of collision is the partially overlapping collision.

Here we have a projected time slot and an existing time slot that partially overlap. In this case, the projected time slot starts before the existing one has ended. There are four possible solutions for this type of collision. The first one is what we call their start. We keep the existing diagram. and create a truncated template that starts at the end of the existing one. The second possible solution is what we call our start. We create the projected time slot and change the existing time slots to end at the start of the projected one.

The third possible solution is what we call Dirt's end. We keep the existing time slot and create a truncated time slot that ends at the start at the existing one. And finally the fourth possible solution is what we call our sensor We create the projected timeslot and we change the existing one to start at the end of the projected one. All previous solutions are also available. The third type of collision is a supraset collision.

Here the projected time slot is a supraset of the existing one. There is one possible solution that we call dares bot. We keep the existing time slot and create two new time slots around the existing one. And then there's the fourth type. is a subset collision. Here the projected time slot is a subset of the existing one. There is one possible solution that we call our spot We create the projected time slot and split the existing one

in two around the projected one. Now let's see this in action. I want to show you how the collision resolution works with a simple example I am running a local Docker Compose development environment here and I have created a show using this JSON file with a post request Then I also have this other JSON file describing a new scale for the show I just created

It is for Monday, October 16 from 10 30 to 1830 and it's only once. I can create this. New schedule with a post request. The response is a 201 created. HTTP status and a new schedule with a single time slot from 10. 30 to 1830 in October 16 are created. Now, if I try to create a new schedule describe it but this other file

from this for the same date but from 1745 to 1810 This will result in a time plot that collides with the one I just created. The response will return a four hundred and one conflict HTTP status and all the information about the collision including the hash describing the collision and the solutions available for this collision. If there were multiple collisions within the new schedule, then I would get a set of possible solutions for each one of them.

In this case, I'm going to replace the previous time lot with a new time plot that is shorter. For this all I need to do is add the solutions. json dictionary to the request where the Hash of the collision is the key and one of the solutions is the value. In this case it's going to be ours Now I get a 201 created HTTP status and the new schedule with the new time slots.

with new types that is created During the development of steering, the views and templates were removed from and Django REST framework is now used to provide a proper REST API to the components of Aura. At the start, six years ago, the admin interface had more than 600 lines of code. Currently we are still having around 90 lines, but the goal is to remove all of them. The models reached its peak at more than 1300 lines of code. Currently

we are below 500 lines of code for 17 models. Around 700 lines of code from the models are now part of the services. Most of them were methods dealing with the conflict detection and the conflict resolutions. We are currently slightly below the 1100 lines that we had with the Django class-based views. But now we are using Django Rest Frameworks, class-based news. And now we have more than 800 lines of code in the series.

This is possible thanks to people that have contributed in the past and the team that is currently working on Aura. Margarete Majao Falishka is the product owner for Aura. Christian Pointner is the lead developer of the tank. David Tratnik is the lead developer of the engine. Konrad Morfeld is the lead developer of the dashboard. Ole Binder is the lead developer of the engine recorder. Christian Pastel is currently contributing to the development of the engine API and the engine components. I am currently the lead developer of the

steering. Ingola Endeker contributed the collision detection and resolution and transitioned PV into a proper REST API. Andrea Ida Malkam Klauna asunas Czeki started the development of the dashboard Gottfried Geisbauer started the development of the engine. I want to close this presentation with a quote that I think does a good job summarizing where we are now with Aura and what is ahead of us. The possible is already done. The impossible we are doing it. For miracles

we need time. Marcelo Bielsa is a former soccer player, manager, and coach from Argentina. He has coached the national teams from Argentina and Chile. and will lead the Uruguayan national team into the next World Cup. We are currently working on the third alpha release of our 1. 0 finishing the APIs in the steering and tank components and the dashboard We expect to release the first beta in the first half of 2024 and deploy Aura at Radio Orange in Vienna and at Radio Helsinki in Graz after that.

I want to thank the organizers of this year's DjangoCon US for this opportunity. And I hope to be part of the Django Kong in the United States in person sometime in the future. If you are interested in the project or would like to check out what Aura currently offers or maybe give it a try or who knows if we contribute don't hesitate to reach out You can visit our site at aura. Radio. You can check our documentation at docs. auraRadio. And you can see what we are currently working on at our dot yet.

You can also reach me over Mastodon Thank you much very much for your time and attention and I hope this has been interesting to you. Thank you very much