How we built a mobile application backend with Django in 10 weeks

Hi, my name is Cholo Shahin and today I'm going to talk about building mobile application backends with Django. A bit about me. I'm a software developer based in London and I'm working as a lead developer since 2013. I'm using Python and Django for many years now and also quite happy to be part of the lovely community. Other than really enjoying working with web application backend teams, I'm one of the co-organizers of Django Landom Meetup. Also an all-time Jung Girls organizer and mentor. I'm one of the co-organizers of the first

ever PyCon Turkey. and you can find me in Pilady 's London meetups. I'm also a proud member of Django Software Foundation and Python Software Foundation Yes. So each software development team is different. Some are small, others are bigger. Some are organized with stack, others are more mixed. Or a team can have dedicated product managers, UX, UI designers, or not. So uh this presentation is no more than my experiences leading a development team delivering back-end services for one mobile app.

And we have our own best practices, but we are far away from being the most experienced mobile backend team. However, I find value in this experience. does I wanted to share this with fellow Django Nords and also all those who is interested. I also like to encourage others in similar position as sharing the first-time experiences is as valuable as sharing years of experience in my opinion. So do it. Let's do it more. And also as a team lead and a prospect engineering manager.

In this talk, I will be focusing on my own challenges and uh that will be it. Please join me while I take you to a journey on mobile app backend development. So there are many different aspects I want to share with you in this talk, like challenges of uh delivering mobile backend for the first time, how mobile and web development differs , like working backward compatible. versus making the app, the project management challenges we had, uh choices we had to make, uh

more technical right, right. going RASFO versus GraphQL, setting up developing environments, marking up APIs, load testing, data management, planning the launch. I'll take you in a week-to-week manner into the 10 weeks of ours and then we'll conclude with some lessons learned. So week zero, start of a journey. The project started quite fast for me. Usually we do have some time before building a new app or starting a new project and preparation usually takes weeks.

This time I got switched from another project directly into the start of this one. So it's week zero. Me and my team is going to deliver the back-end services of our new mobile app. We are going to work with an agency. This is new for us. We previously worked with contractors, but this is going to be the first time we are working with an agency with an external team As a startup that has uh their internal development team, uh for us their practices are more customer-facing. So

we really liked it when they shared the story map. a story map for the mobile app with us, a practice we haven't done internally before. It's a good scrum practice and uh my team is impressed. And uh the story map is followed by a design review. The back-end team reviewed all designs and we have an action plan. So week one preparation week. After the hand all hands meeting in the previous week, we had our first engineering meeting and we met the mobile team for the first time. Both

parties agreed to work more closely. So uh we are going to have an engineering meeting every week to discuss and plan the week ahead. Each team has different work of way of working. We have two-week sprints. They are working with one-week iterations. They have a dedicated product manager and scrum master and a hands-on engineering manager. and the developers are more um they keep changing. We have uh uh uh vp and hands-on lead developers also a static development team a mix of back end front

end and yeah qa and Yeah, we set up Slack channels among organizations, which is a good practice as internally all commons are on Slack from our side, engineering or non-engineering. However, something we deeply felt the need is we didn't have a common project management platform. They are using Jira and we are using Farrow and they are using Notion, we are using Drive, Google Docs. On top of it, they are using GitLab and we are using GitHub.

So quite quite different project management tools. So each team, uh we decided each keep uh each team to keep using their own tools, uh, but I regretted that decision very, very soon. That w there was a big value for us in using the same tools, same platforms. We missed that opportunity from the very beginning and to this day we are still uh feeling the need uh and cannot make the change. Another big difference in ways of working was the uh detailed user stories and conditions of acceptance as we usually define for each component, each feature.

they were missing, they were using those. So we have started to work without user stories We have a uh legacy web app. It's Django Plus React , which is still live. Also many users on the platform. and an existing API. How to start the API works for the new mobile app Of course with some sprinkling. We upgraded dependencies, we removed some unused um apps uh this and that

which uh you think wouldn't shouldn't take a lot of time but yeah It does, it does. Do it often do the sprinkling, not only in spring. And um next we needed to define which endpoints uh are missing and what endpoints are required for each screen in the UI. After finishing that, um You will make some changes and added some extra necessary fields to the user endpoint. Yay! The project started, we are planning, we are coding, and we are still

not aware of our baby steps these are. While we were working uh on the planning of API work, uh mobile team was willing to play with the API. And uh we are using Jangres framework. It comes with its own documentation. Uh the browser API uh provides Software self-describing documentation of the uh API Uh the documentation for each API endpoint can be provided simply by visit visiting the URL in your browser and um that was working fine internally

However, our existing API was huge and complex and if we're going to uh continue with using browsable API, we needed to set up another document to point mobile devs to the correct endpoint definitions from a big list. Also, without the initial knowledge of the endpoint design, it was not straightforward to use the API with only the documentation that's available in the browsable API. We usually design um uh back end uh uh endpoints uh like back end engineers and front-end engineers together so they

there is a know-how how to use uh specific endpoint. So um For those type of uh documentation, we were using GitHub, uh GitHub wiki pages. But uh again with the external team that was uh not a um option. So we decided to uh use GitHub pages And um it worked quite well. It was the same syntax, very easy to deploy. And if you want, we could also uh put it on uh behind the authentication. And that worked out quite well.

That's currently my favorite documentation site. And yeah, I suggest you take a look at it. So week two, three, four uh back end development uh is going on. We are, I think by those weeks we were talking about heap, like an alternative to Google Analytics. how to monitor user events, which was different than web. And those are the weeks that uh We anticipated that back-end work is actually more than uh we spotted in the early more cups.

So the very first thing we needed to do was changing our authentication backend. This was a dramatical change. The mobile app uh the mobile app designs was using passwordless authentication with a mobile number and we didn't even have mobile number as a user field our app to other apps. We 're using user email heavily and phone number was only defined in some other models for notifications, etc. So we needed to add mobile number as a field to the user model. And We need to define a custom user model for that because we were using

uh country pod user model and I started searching and there are a lot of documentation not documentation blog posts and Tickets in Django uh website on how to spe switch to a custom Django user model mid -project. It is lots of refactoring, a big recipe. Refactoring includes deleting all migrations and creating new ones from scratch. It it is scary, it was scary. Um

and We needed to do it in a very limited time. We did it. It worked fine. It was scary, but it worked out well. And uh I quite enjoyed actually doing it uh with my hands a bit shaking, but nothing uh uh got broken, all the user data was uh safe and sound and yeah and the next thing uh was the unique field problem uh with mobile uh uh with mobile number uh being added to the user model and uh not currently

currently being recog recognized uh by the web app, uh we didn't have any unique fields in the user model other than user ID. And I was very concerned with this with its side effects. Like um lots of tests were broken. I mean we fixed those uh but like admin login was no longer supporting username because some of users didn't have usernames Maybe to use IDs to log in. Also, the web app was using Ot Zero login, which was done by email and our new users wouldn't necessarily uh

have an email starting their user models. But um Turned out okay. Uh apparently my concerns was um Abbott, uh yeah, I mean it worked out well. The second step uh was the authentication of API endpoints. Um web app which was webpack webpack powered react on front end uh uses session authentication with email and password login And all back end was beyond keeping that in mind.

So I was a bit worried that Could we just switch from session authentication to token authentication? What type of problems we were going to face? However, we used to work with on an endpoint for one client like two years ago. They were going to use that endpoint in their mobile app. When I was working on that project, I set up that endpoint with Django S framework token authentication. And thanks to Django S framework, defining token authentication for the rest of the endpoints was super easy. It saved more than a day.

With only little effort we had decoupled our Echo complication and this enabled the mobile team to start building the app. And we still needed to plan further. Next challenge was which authentication system to use? Some options were uh odd zero password is authentication service, uh Firebase phone authentication definitely , and I was desperately uh looking for a Django package to solve it internally. And then I found the Django

S framework password. passwordless package. It was a very well-written app and it did the job well. It integrates passwordless authentication in the Django RS framework. You can use email address or a mobile number. It's beautiful to work with DRF's own token authentication system. And uh in the back end it uses Tbilis to send the SMS, uh, which was also uh our SMS provider. or unrecognized emails or mobile numbers, uh create new accounts by default. So no We could authenticate users why more

numerous and yeah To this day I still uh like using this package and um it's it's it's a hand-on in mobile version of Django. So these weeks were also the weeks that our content team started creating manual data because app needs data. Needed to display data and we kicked off with a big, short, big, big, short list of curated data. This initial process was heavily driven by CSV files and data ingestion scripts

and then those are not enough. um Django admin came in very handy. Now uh we needed to set up a content aggregation pipeline to automatically feed uh to automatically feed fresh data into the app until we have enough content creators, owners uh to create content for us. On the other hand , we were working on the data models schemas for the app, and we were constrain using existing models versus defining

new and more explicit ones. Existing models uh meant moving all the tech depth with us and defining new ones meant leaving existing functionality behind We went with the new models and I never regretted that decision. However, I do believe the decision depends heavily on specific needs and analysis of each individual project. And the first one for me was to mock this many endpoints. We had lots of endpoints to deliver and only limited time. and um mocking up endpoints even uh without having the models defined enabled us to cope with the app releases

So the uh one major difference between uh web development and mobile default development is in mobile development there is no continuous deployment. Uh so uh they had the mobile team was doing weekly releases and if endpoint uh is ready by then it was going into the release and if it's not uh it meant another ten days So uh that's why uh we heavily mocked all the endpoints even though we didn't have uh anything working behind, just sending dictionaries.

in a minimal format work for us quite well and I highly recommend it if you need of such a thing Another major difference was the delivery schedule with all the teams I have worked on till today. Freezing the features was a big issue. For an agency with a strict deadline and in the mobile world, it was almost possible for them. So uh they shared this scan chart with us, a C sweep plan. So uh uh I would be they're going to work on a specific um module and um the and

is the at launch it looks okay it looks great uh however one th one thing they didn't take into account was uh the product was not ready in the back end not only the endpoints but like the the product itself and all the mocking and sprinting and all uh couldn't help covering that So we are four weeks into the project and mobile team is going to start coding next week. In these four weeks, we have finalized detail and back-end work. Now we have an idea but cannot plan

because deadlines are already set. This is going to be a real challenge Week five, six and seven. Both backend and mobile team is working on the app So we spent most of week five in a team hackathon, actually two of them. One uh team focused on uh one team was running a hackathon on back-end services for data aggregation. We were just four weeks into the project and we didn't actually know much detail about our data.

This hackathon helped us to understand well. Also, we created all the new models and endpoints during the hackathon. On the other hand, the other team was running a hackathon on the services to process data. By the end of the week, we aggregated a big dump of data into the database. So from it was like from zero, from no models, no nothing to a full database. And a version of the multi-level processing pipeline was implemented by the second hackathon team. It was a big, big

team effort. And uh the result was like almost product production ready, I mean for the first version of the app, data and pipeline. So win win. Working with uh front on font and tests For many years there is an inevitable moment where you start to discuss um how restful we should go. And the time was uh time has come for a mobile app.

The mobile team was complaining about the huge number of endpoints they needed to hit to display the feed. I mean they they were right. I think they were at least hitting seven, eight endpoints to collect all the data, all different levels of data from the uh REST API. So we were quite limited in terms of time, but uh though I uh There was no uh way out of it. I defined um I created a GraphQL API next to our Jung

Rest framework uh RESTful API And we use we define a query there to collect to um uh display the feed data and um Everyone was happy. It was not very difficult to set up the graph well and uh since the endpoint was read-only It was not very complicated. I needed to write uh middleware for the authentication. But uh other than that, uh it was uh it's more like uh not going to say easier but like uh hustle free uh than I thought

So uh we now have an draft all API mixed our school API And during those weeks we worked on several components. Those components are common in any social mobile app. You need to display other users data as an imitation mechanism because you don't want millions of users to end up in your database in day to you Because your app is going to be very popular, right? Uh there is a feed uh to display uh uh users will like and follow uh each other, their connections, uh

their recommendations. Uh several level of entities like user profile, public prof uh figure uh profile, organization profiles and um tags of course and uh data signals. So uh for most uh we used um uh we we uh we define our custom models, but I do think um uh there is uh I mean these are very common in every social app, so I I I'll expect uh to have if uh few a lot Django

page packages around uh those like solving uh problems or n data or service uh level uh issues in those um components. Sorry. There um there is no very well known or like uh out-of-box Django packages for most of those these components. So it's some food for George After planning the backend, we started to plan the role testing of the APIs and we used to use Rocus. We again write new scripts

and there were serious uh log issues. We moved uh to bigger dinos, we set up some autoscaling for the services and uh yeah it was good. Along the way , these were the weeks we started to create SEMS So we had a large amount of backend services to automatically aggregate and process data. There were two different pipelines. They were both automated. uh but we still needed manual annotation of the data. That was going to be done by

uh overseas company and uh We needed uh admin-like one-page CMS. So that's how the CMS project started. It started for annotation and used for many other use cases. Again, we we should have used a Django package, a CMS package, and that's that's another uh regret, not only me, but um the other team member that owned this project. We can move out of it and we need to keep uh developing it.

So towards the launch, still there are components that we need to deliver like settings notifications, user history, push notifications, again, and iPhones. some Django apps but didn't work for me out of box so I got inspired uh from some of them team uh team mostly uh defined uh new uh models and um uh functionality uh customized App submission. During the um

first uh eight weeks and after uh we used um uh Microsoft App Center. They used mobile team used the Microsoft App Center for delivering the um app releases which is um uh which is a simple uh interface you can see all the apps uh Android or um iOS uh that you are allowed to see assign to we can uh download and install the packages and uh but by the end of the 10 week like I think it was week nine uh my well team uh asked me to set up demo users for the App Store and Google Play Store review

process. I Didn't know that we needed to set up demo users so that the review process could happen without a SMS verification. And um the Django package we were using uh supporting that feature Otherwise I think we were in quite a big um trouble. Uh so that's uh That 's like be being unexperienced or like working in a mobile work for the first time. I mean we we We delivered Slack apps before, so actually we are

um experienced in delivering um apps and getting reviews, uh creating a even like a walkthrough video and everything for this time. Like we didn't plan about this and hope like the package we are using or supporting it it was undocumented. But yeah uh save the day and um yeah we extended the project we realized that we needed to make changes To the API, to the s uh data schema, and

some small, some bigger uh They they were all painful. So the first change we did was to rename OneField in the API. That was a request from the mobile team. They didn't imagine we would do it in that way that is back backward incomfortable. The app has broken As soon as we do the change and it failed the sign-off from Apple. So it came back from Lurious That was a easy fix, but not a great uh first-time experience. Um the second change was

a lot bigger, so it was one of the main data models we wanted to change. It was going to change like it was going to result in a dramatic change in the data schema Uh like it took us two weeks to plan it, to discuss it. And uh we worked fully uh backward compatible Nothing has broken but we fail to communicate the change uh heavily. So weeks and weeks and weeks after mobile team design team they were complaining that

some stuff changed around data models and they wasn't on top of it. They didn't understand, they don't they didn't know the details. So backward incompatibility was wanting, but then communications is also very very important in terms of changes. And another one was technically not difficult. So uh one management relationship uh that uh that was defined for many models like three four. We needed to uh edit it in our way it is uh Um

yeah, technically not Not the everyday job or not something that you want to do, but not very difficult. Um there's this separate database and states uh feature in the migration that I learned after this change. It went okay, uh but back then um Not the change itself, uh but uh some one detail uh in the change in in one of the custom fields we defined uh created a big deck tech depth for us created lots of b uh bugs later and uh back then we were unaware of the catastrophic effects that

actually uh we realized in investor uh uh that uh presentation. And uh yeah at the end uh we needed to revert and find a different solution. But yeah The changes on API or data models are inevitable when working in such a fast space with good planning and good communications, those changes usually pays off in the long term. And uh when they don't, you have no ch choice but to erode Okay, um before submission of the app

to the app store, um we used have the R dev API. I mean the app was pointing to the development API and uh API was querying the development databases Uh that's like using the the API gave us great great flexibility because even though an endpoint was ready to be consumed by the uh mobile app team for the voting purse purposes we had um uh uh QA and uh code review process and uh this enabled us to deliver quality code uh to the production uh without affecting the uh development

and without uh stopping the uh fast pace And we did the switch in week nine. I think we should have switched earlier. It went um Okay , it took a bit longer, like 10 days. Uh a team member focused on that. It was mostly data, but um Yeah, so we used uh Dev API until the very end and only switched the ProLaption API in the last two weeks. and this encouraged mobile team to start working with multiple environments. Later they kept using the development API.

during the the whole new phases and uh they didn't use our staging API uh but uh yeah collection API so Yeah. And uh moving towards lounge, we finalize the plant work. Uh at feed service i was in action now. Recommendations were going from vacant services, user invitations. or sends uh no more mocking and all data is coming from real backend services. The services is not is not perfect but uh nothing is broken. Yay Yeah, while mobile team was able to

deliver all as planned in six weeks. Our team was exhausted I was exhausted. My experience is a mobile FUI is more limited. compared to a web app UI. However, the back-end services and features are um not So um the overall feeling in the team was more towards like we hardly delivered, we still have lots to do But um all teams should self-care and celebrate success

Uh in the uh week nine and ten, we schedule time of for all team members. Uh we create the calendar so uh there not everyone was away before the launch. Everybody had to take an extra two, three, three days off to recover to have some rest and to celebrate success and I do suggest you keep this in mind and don't forget sooner is better after delivery is too late. Uh give yourself some self-care, your team some self-care, and don't forget to

celebrate success. And when I say success, we arrived at the lunch slides. By the end of the tenth week, we submitted what applications. to uh bought app stores and soon it was launch day So um lessons learned. This was uh a very, very um Beauty time for me. I realize it's not

uh so very different like building a mobile app uh backend than a non-monolith Django. It wasn't uh not so much different Apparently mobile teams and our team had different practices in place. I do think there is lots of room for us to work on necessary Django packages to support mobile app. development mobile required mobile app uh components. Um I found I found it very difficult to test and monitor the uh

mobile app , especially in in uh from a uh backends perspective Backward compatible development is a new territory for me and the team. But it's it's not that difficult. There are uh less complexity on features, it makes UX a lot more important uh user behavior uh was is harder to model and monitor we spent a lot of time. uh discussing and uh influencing it. And uh last but not least, mobile developers are at Frentry Point and I'm looking forward to continue working with them more in the future.

So that's all for today. Thank you all for listening and I hope you have a great Jangakong.