Building Microservice Architecture for scale with Django

Speaker 1: Hey everyone, uh I'm I'm Ansar Kilani and it's a bright and sunny day here in Lahore and together with my uh with my colleague Tao we'd like to commence our talk on building microservices architectures uh for scale with Django By the way, as much as we'd like to be physically present between all you guys and you know some of the best minds in the industry We'd like to apologize for not being there. We were denied visas for traveling and we just wanted to make sure that our message and knowledge does get to you and we help the industry grow much better.

Speaker 1: A little bit about uh me and Daoud. Both of us are software engineers at one of the most prestigious software organizations in Pakistan uh which is called Arbisoft and like many other engineers and peers at the company we are active contributors to open index repositories And with the transition in architectural infrastructures over the last five to six years, uh we can uh say that we have been very well educated with microservices and how we can quickly scale them for any application ecosystem in the world So if anyone amongst you want to reach out us on LinkedIn,

Speaker 1: you can scan the QR codes available. Before I move any further, I would like to mention a quick word from our CEO at Arbisoft. Yasir. uh who has been a primary inspiration in our growth and to take part in the Django Con uh this year. say that he has been an incredible support throughout this journey and he holds us and the engine other engineers uh working at Arbisoft to the determination and commitment to the industry that he holds to the industry uh as a matter of fact. Coming to our agenda today,

Speaker 1: here's the itinerary that we will be following for the talk We'll start off by comparing monolith architectures with microservices, followed by some of the principles that we follow in engineering and scaling microservices. And we'll top it off with a case study from our OpenEndX migrations. With that being said, let's talk about monolith and microservices architecture and what makes each pattern so unique. As you can see in this chart, this represents the comparison between Google Trends of Monolith and Microservices architectures over the last decade in in the whole industry. And you can clearly see the domination in the popularity of microservices over the last six to seven years.

Speaker 1: Now what makes microservices so cool and why is everyone talking about it? Why is Is everyone thinking about uh replacing monolith with microservices? And why? Is it really that good? Let's find out If you're not very much familiar with the monolith architectures, let me give you a quick recap about them. A few years ago, everyone who wanted to create a new application would just build a single server and started contributing features to that application. Features can be some connectivity to the database, some business logic uh contribution that can be a UI-based uh feature, but at the end

Speaker 1: After a few months or maybe years of contributing to the same project, the code base became so huge and tightly coupled that it became impossible to differentiate between two components. Typically, user interface, server-side business logic, and database connectivities are all managed under a unified singular umbrella. And it became a traditional method uh to create any new applications all around. Now there were a few reasons why it was so popular It was easier to develop, easier to deploy, to scale, to test, and maybe debug.

Speaker 1: Over the past few years, we have seen dedicated technologies to optimize each aspect of the whole infrastructure. For example, we use React for front-end development. We use Python and Django for back-end services, data analytics. And maybe use Cypress and Selenium for uh testing and debugging purposes. But Monolit does not offer that kind of flexibility Utilizing dedicated technologies for each application that was not at all possible. And if you even if you are using a single technology stack like Django

Speaker 1: to do all the work. The development speed is bound to become slow exponentially with the growth of the application as you would need to rely a lot on the other counterparts of comp to complement your contributions and operate flawless. Eventually you can Maybe expect your application as a bake as a mammoth, hogging up all the resources on a cloud server deployment. But what about microservices? Actually, what exactly is a microservice? Well as the name states, microservices are a collection of small and autonomous services, each with its own self-contained implementation and business context

Speaker 1: collectively building up a singular application and in object-oriented terms A microservice would represent a specialization of each business context aggregating with each other for a large generalized product. Each service may communicate with one another using either synchronous or asynchronous protocol. For most of the services we use, HTTP Rest protocol is very popular. But each service must be deployed independently and have its own database. So it can all de

Speaker 1: be decoupled from each other. It's not a wonder that most large-scale organization use microservice architectures. for their products, you know. Most of these organizations uh are big tech companies like Netflix, Spotify, Atlasion, uh even Amazon and of course edX and Open EdX. Each having hundreds of microservices, if not thousands. In simpler terms, microservices offer six major benefits against the entire industry in in in uh against the entire monolith architectures.

Speaker 1: The code base became Highly maintainable and testable, as well as easily deployable, more like independently deployable Furthermore, each component of the entire infrastructure is loosely coupled and can undergo its own development lifecycles. Each service is organized around its own business capabilities and can be owned by each team separately. Most importantly, in a world of large complex applications, microservice can ensure rapid and reliable deliveries in the interest of scale and frequency One more thing that

Speaker 1: I would like to mention here is that microservices offer a three-dimensional scaling to each application. where we create a functional decomposition pattern by splitting things that are different as opposed to traditional two-dimensional scaling approach Speaking of principles, let me briefly help you with the two architecture principles that define our microservice. First and foremost are the 12 factors to engineer any service. Now these factors came into existence after constant and consistent iterative growth in development and deployment processes in engineering universe.

Speaker 1: Firstly we track one code base in revision, but experiment many deployments. We explicitly declare and isolate dependencies and store configurations in environment. We treat backing services as attached resources and separate build and run stages in the live cycle. Then we execute the app as a single and multi-stateless processes and export these services via court bindings. We achieve concurrency and disposability and remove disparity between development and production environments. Finally, we treat the logs as event streams

Speaker 1: and run admin processes as a single process. Together, these factors help us build our infrastructure foolproof in our life cycle. Furthermore, we use solid principles to engineer the core of Django applications inside a service. We start off by dividing services atomically so each service serves its own purpose. Furthermore, each component that we build, we create a level of abstraction So each entity is open for extension and close for modification. And of course, then we apply Liscoff 's substitution principle.

Speaker 1: The east class, each subclass inside a service is substitutable for its payment class. We make sure that no interface view is dependent on a method that does not Strictly use that particular method. Lastly, we divide the high-level components and make sure that these components rely on abstraction rather than explicit low-level modules Together, these solid principles ensure that our microservices remain as independently as possible and they cannot rely on external structures for scaling purposes. Speaking of scaling, I'll leave the floor to my colleague

Speaker 1: Daud, who's gonna come in and explain how we scale microservices. Thank you.

Speaker 2: Thank you, Ansap. I am Fay Daut and I'll be covering the rest of this talk, which is on scaling microservices and the case study of Open Edicks. Uh in scaling microservices, we'll take a look at a few of the items, which are authentication, packaging, and tooling. templating testing tools and documentation and see how how we can scale our microservices effectively. I will kick this off with authentication Now uh when we consider authentication in monoliths, it's uh very simple. Uh we have a single gateway where all sorts of users are entering and authenticating themselves. uh in terms of implementation we can uh really

Speaker 2: handle the special use cases or special uses specific use cases really easily in a single place And there are no major complications in with respect to managing tokens or cookies because we have a major major service that is doing all the work In microservices, that is not the case. We have dedicated services and users will be interacting with the service that they want to use or they need. So we can have as a user that is interacting with one service only, but then we'll have users who will be interacting with various services like we have here. So this complicated the situation

Speaker 2: a bit like who is the source of truth in here? Is it service over service one service two? Like who who decides that a user is authenticated and how is that communicated across various services because we have separate gateways uh the etiquette point for the users And if a user is authenticated in a service, how do we ensure that the other service also treats as authenticated user? These are some of the challenges that we have with microservices and authentication. That makes this a bit of a complex scenario. A few of the questions that you can try to answer to better decide the authentication mechanism, like do we need a single dedicated application for authentication or do we want to delegate that responsibility to each service or at least the back-end

Speaker 2: service? How will we manage account application if that is happening for the services? Like how we keep the information in sync or do we really need to keep that information in sync? decorating uh authentication with system specific or feature specific roles for the user so that uh we can customize the behavior on front end based upon that user role That will be important as well when the again your application is scaling and deciding that do we want a uniform authentication mechanism across all frontends backend or mobile application or do we want the different mechanism for front end different for back end different from my application or all of the sub application that you have so these are

Speaker 2: The answer will vary according to system. It's upon that devs or the architect to help answer this question. But these are some other thinking questions like where do we want to take the authentication mechanism? So if we take the example of a centralized or dedicated aut service and how we can do that with Django, so with Django and Django Rest framework. We have a variety of authentication mechanisms like session, mirror, jot, uh what this uh we can easily uh enhance jot to add user level permissions. Here we we can have a dedicated service where the user will log in and get the guest get the special token

Speaker 2: and then then you could then they can use their special token to make requests across services. It can be a front end and back end and then that service will verify the token without service and then drive their behavior Packages as plugin that is also a side effect of moving away from monoliths to microservices. So in monolith we have a big code base. every application is able to import code from another application. So the the application will be become coupled. uh but there there isn't a problem in importing the code that we need. But in microservices, since we have dedicated microservices It's not really easy to just import that

Speaker 2: code. The code that we are needing, it might be in another service. So how do we manage that? We would want to duplicate that code in each service. It works, but it violates the tribe principle and also it makes it difficult to communicate the changes in that util across services. So let's say if we have a piece of code that is used by 10 services, if we update in one, how do we communicate that to other nine services? The solution here is packages and plugins. What we'll lose uh do is that at least in terms of Jenguin Python that we'll create packages, publish them to PyPy and then install the packages where they are needed. Uh what it helped uh in that is that we'll have a single place for all the utilities that we need.

Speaker 2: Uh we'll just install in a service that must to that wants to use that functionality and if we need to do updates in the details we'll have a designated place where we'll do that change and publish a new board version. If a service is wanted to use the updated version, they just use the latest version of that package and all will be well A few use cases for that authentication is on top. So uh once you uh have decided authentication mechanism for your system you will have a a pipeline or way to get the auth mechanism or auth token from the service or a way to verify that the token token is valid. So you would essentially want that mechanism to be same across services.

Speaker 2: So you just have to create a package for that and import that across your services. Observability is another good example. Like if you're monitoring your backend or front-end application, you will have some sort of helper methods that you're using and you would want them to use across every service. That is a good example as well. User events are also good example of that. Django templates are different example, but if your system is brand oriented or brand brand focused, you would want your branding to appear on your hydro or photo. So that's where you can use uh templates in a base package and import that base template from that package in the service that you need.

Speaker 2: Testing tools, okay. Uh since in microservices communication among services is a key aspect We want to ensure that that services communicate with each other, they get the data and behavior they expect and they need. And whenever there is uh You could say violation of communication. We would want to be able to flag that behavior to the developer who is who's making that change So that's why uh to be able to decide on the testing pathway is important in microservices We have our traditional units and functional testing, but additionally, since uh we have multiple services that are communicating with each other, we would need a bit more than that. And that's where our integration and tool testing and contract testing

Speaker 2: Coming place. While the uh module or unit testing is focused on the smallest aspect of the code, the integration is mainly about how the services are communicating and If their communication expectations are aligned. So what we would want to do is that we would want to divide our indication behavior into our end-to-end testing and contract testing. And to N would involve UI and user experience in some way because we would want to test entire behavior that would span multiple systems. And we have Cypress uh really good tool for that syllaum uh we have a headless pause away we we can do this task contact testing is something uh different

Speaker 2: and It is being adapted into industry, but it essentially focuses on the communication aspect. So we have two applications, consumer and provider, and we want to ensure that their communications uh is documented in a contract so that's where the contact testing uh has been doing a lot of work with the help of the pack. io sprint cloud contact is another tool for contract testing so This triviant will become important as you scale your application because you would want confidence to deploy your application in such a way that it does not affect any of its dependents. So that confidence is only achieved when you have a proper integration testing mechanism in place. Now moving towards the bit of documentation stuff, which is the service documentation.

Speaker 2: Like when your services are expanding and you're adding new microservices for your business use cases. It will become very difficult to ad to keep track of who owns that services, what services that service depends upon, or what other services are dependent upon your service. uh identifying which APIs or endpoints are used, what data fields are used, that becomes a a bit of challenge when your services are growing So you would want different mechanisms to be able to document your ownership, document the service information, documents the data contracts. There wouldn't be a one size fit all solution. You have to use a combination of uh of multiple. So one could be that you can create a dedicated service repository.

Speaker 2: It can be a small Django app whose responsibility is just to hold the information of the services. their repositories where they are deployed, who is the owner, what service it depends upon, what other services use your service and etc. So you can have a small app for that You can have your public API documentations for your back-end application with Sword and Stopwight. Data contracts are something that is generated as part of contract testing and they contain the information of APIs that are used by a consumer. what fields are are are required and these container really unique details of the communication. So if you're already using contract testing, that part of documentation is also

Speaker 2: fulfilled. It will essentially help a new developer and also the old one, but particularly for a new one. Why? Because when they are onboarding onto your system To be able to understand the complexity of the ser of your own service in the entire ecosystem is really important. I think that's where the service repository and data contracts would come into play Templating, which is last part in the scaling, when your system is growing and you're adding new microservices, you would essentially want to be able to set up your new microservice in a matter of minutes. be it a front-end application, a back-end, or some sort of utility or some sort of plugin. We cannot afford to

Speaker 2: uh spend weeks to just to create a new service that uh that is deployable. Because you can see the benefit of microservices is uh is one of the benefits is speed. So once we have uh code structure established we can use packages like cookie cutter to uh to be able to set up new microservice. in a matter of minutes and also uh set up the infrastructure templates and use Terraform to to be able to deploy your services very quickly And again, this is something that once your system starts growing and expanding and becoming adaptable to microservices, you will need that. Because as your business expands, you will need

Speaker 2: new microservices. Now, just a little touch base on how OpenEDX engineered its microservices and where do we stand on it today? So in OpenEDX, we mainly have two types of uh microservices. One is IDAs or independently deployed applications. These are Backend Django application and then we have micro front ends. MFEs, which are front-end React Plus Redex application uh we have uh mobile apps uh uh as well but they they predate microservices era and these these are the outcomes of really the uh RD done to move from omelette to microservices And at this moment we have roughly 46 microservices with

Speaker 2: almost equal split of ideas and MFEs. So what's the journey? Starting when the edX started, we had our edX platform, which is the core and monolith of OpenEtx. house our learning management system, LMS, our content authoring system, our studio, the authentication mechanism, e-commerce, analytics, video modules and much, much more As a startup, it fulfilled our purpose, but uh when the system started expanding, we faced development challenges. edX platform alone has 2. 5 million lines of code deployment times were really

Speaker 2: high two to four hours and it was difficult to identify the boundaries of like where one app is ending and where the next is start it also caused a lot of scalability challenges as well because everything was just coming into one big box it became uh There were a lot of performance challenges. Application were tightly coupled. And one of the goals that we had in mind that we want to use real-time event thing using Kafka. That was not achievable there as well because there was so much uh development and deployment challenges. So how what's the current state? The transition from monoliths to microservices took quite quite a time because we had to like

Speaker 2: query define our ownership and identify where each app would go. And we wanted to do that without disrupting our current system so that our our learners and all the all the partners, they are not affected by that. So there were a lot of careful planning there. It it took us three plus Here time span, but uh right now we have a lot of dedicated repositories and deployment pipeline We have hundred 100k lines of code in each service and there is 10 to 30 minutes of deployment, which is really really small as compared to the two to four hours who deploy that monolith And with the clear ownership, we know which team is responsible for maintaining what part of

Speaker 2: system. So that transition has really helped in scaling our system. And we are on the path of scaling. And if uh if I were to connect this with the things that we have discussed earlier, the LMS is the big part in edX and LMS is acting as our central authentic provider. In terms of packages, we have specific edX Django packages and go edX and Django React. framework, event base, and some edX uh specific utilities as well. And as if we talk about testing, we have established Cypress for our end-to-end testing and we have been piloting as well for our

Speaker 2: contract testing. Cookie cutter is something that we have established for quite a while and we have uh dedicated process for that to create back end and front end and plugin application. This has become self-service that every developer can just down that download that cookie cutter and use it in a matter of minutes to to create a new device. But what with our model? It is still there, but it has been reducing in size constantly. Right now our back-end application are communicating with our platform where needed. But essentially it's the back-end

Speaker 2: application communicating with each other, the front ends communicating with our ideas and uh really honing that microservices uh architecture. Right, so that was it. Here are a few other references of the both the Microservices part and edX. Thank you for being a good audience and now we are looking forward to your questions. Thank you.