Technical Debt: Why it'll ruin your software by Luan Fonseca

And hi everyone. Technical depth and why it will ruin your software. And here you can see a really known monument, and it's located on Italy, more specifically in Pisa. And it's called the Leaning Tower of Pisa or just Pisa Tower. This building has a very interesting story behind it because it was built on the 12th century. and due to a soft ground which could not sustain the weight of the monument, it started to lean. Okay, the structure the structure was stabilized in 2001 after eight years of e-work. One fun fact is one of those reworks and fixes in fact made the leaning worse. Well, I don't know if you guys

already passed through this, but I already did a refactoring that makes the problem worse, so I can blame them. And now I hope you have two questions in mind. First one, why the Pisa Towers relates to technical depth? And am I Italian because I am saying about Pisa The first one you know through the to the end of this talk and the second one no I am not from Italy, I am from Brazil. Actually, I'm from a really a really distant place from Italy. I am from a city called Recife, where this accent that you are hearing is Portuguese. So Recife is a city that has the best carnival in the world, and at least seven people here will agree because every Brazilian speaker that hasn't this conference are also from Recife.

So it's really nice. Okay, my name is Luan Fonseca. I am working with Django since version 1. 2, almost like 10 years already, and I'm the founder of an open source project called speakerfight. com. Which we manage almost all meetups and conference for the Python community back in Brazil. And also I'm a software engineer at Lab Codes. which Lab Codes is a software studio that's based in New Brazil and we specialize in creating custom web product fitted for specific needs. And we believe that software should be built by fulfilling clients and user expectations. Our software is always aimed at solving problems and optimizing process. It's an incredible honor for us to be part of this conference, giving talks and

sponsoring, because I have a talk today, Nikki did that yesterday, and Renato will be on Thursday. Yeah, okay. Tomorrow. Tomorrow. Oh tomorrow. Thanks. And okay, the agenda of this talk will start with a short story about John showing the reality of an everyday programmer. Then we will go find out why technical debt is always seen as a bad thing and who should be responsible for that. After that we'll go through some solutions that are not so good and another and we can check another that can be healthier for these problems about technical debt. We will discuss if it's lucrative for the companies to deal with technical debt

and what word the guys that coined the concept of technical debt. And in based off everything like that we will discuss what can programmers get beyond profit on dealing with technical debt. Okay, John the programmer. This is John. John is a senior software developer, ours with a ton of work to do. Jones is a five-star programmer with years and years of expertise in Django. Suddenly, a new project appears on John on John 's desks. This new project needs to have a payment system, social authentication, and integration with a third-party shipping service. The due date for that, for everything like that, is what like one month.

But John, John is an awesome develop awesome employee, which always delivers his tasks. He went there, took the project and delivered in time. After that, when the rest of his team went there to review his code, they found that the Jones code have some problems, like inconsistency payment bugs. Sometimes the shipments weren't being processed and the authentication, the service authentication, were too naive on checking the social accounts. Okay, John knew that his code have some bugs, but when John was close to the due date on delivering that, another project appears on the John schedule. So he hadn't time to fix those problems. They had to deliver the project the way it was, but John

is positive and kept thinking, okay, in the future you'll come back and fix everything. Spoiler. We all know that this one will happen, right? So after the review the team found some problems on his code, like the code responsible for the payments on his app were not flexible enough to accept different coherencies. If the shipping service is down for any reason, the delivery could not happen. Also, user with deactivate the account, like if I deactivate my account on Facebook and try to sign in on Jones Project It will work, which doesn't make sense. Besides that, all the code was not tested as it should be. And that are some points that we call technical depth. And why is that?

Because as he chose a first-time solution because of the delivery date is too close, the code had some problems and the team and the team accepted. We are okay with some problems in the code because in the future they think that we will do some changes. In the future, in the future, if this business logic On the project changes, the cost of changing Jones code will be higher because people will need to fix new issues and everything that they accepted as a bad thing, as a not so good thing So that's the moment when technical depth was introduced on Jones code. What happened in Pisa Tower is closer to what we understand the technical depth. Started with a few problems and suddenly those problems escalated so quickly that past

some time it will fall down. The same happens with our softwares. with us usually is more visible during the process because through the development process we are feeling that some features and bug fix are taking more time than it should take. So We are used to think that having some level of technical depth on our software is a bad thing. I mean, usually it is. But we can see it as a strategic move because if we can have deliver some have fast deliveries because of some technical that we allow it to have it's awesome for a pro for a product. The major issue on that is when we forget that we did this in the past and don't come back to fix it.

Imagine that technical depths like the mac and cheese from yesterday. If you have like one or two, it's okay. But after the twelve, it may become a problem, right? So Accepting that it's a trade-off, we can move fast and pay the consequences for that decision later is a really common thing to do for us. I don't know how many of you already pay the consequence from that mac and cheese, but remember that the cost increases over time. So if you identify with any of those situations. on your day-to-day tasks, it's really okay as soon as we have our backlog, some dedicated time in the future to come back and fix. And that's a really common scenario on delivering new things on a day-to-day sprint That you accept that situation and okay, on the future you come back.

If you look on the Martin Fowler Technical Depth Quadrant, which I highly recommend you to check out later, we can see easily where John is. Because he's on the top part of the quadrant being being reckless and prudent because he knew that didn't have that much time to implement a better solution But also address that in the future he will come back and fix. So he's kind of in the middle. When we focus on the name that we are repeatingly saying, Technical depth, technical depth, we are guided to think that it's a developer 's issue. For some reason because we are the programmers and we created and it's technical and it's adept and we need to pay. Well, the problem is that sometimes

Software is a result of the entire structure of processes in some company. Also, the entire industry have changed this since Ward created the term in 1992. So as in John 's scenario, the problem of that not so good code that he wrote is more a management or a product team issue than his, because they don't give them enough time to improve to create a better solution. Because we know that Joe is a good developer, but he needed to be deliberated on delivering some pro on be deliberated on forgetting about some problems in order to deliver in the date that was scheduled to him. And so sometimes the bottlenecks

that are creating technical depth in our software doesn't come directly from the technical team. It can be from other other parts of the company. Like the management team may not be up to date on the development team speed and velocity. So they can schedule more things than your team has hands to do it. Also, the product team might not have a have given a big picture plan for the developers, so they can't they are not able to anticipate things on the architectural side. And even the UI and UX teams may have been like too far on the development process from the devs. So they can create unfeasible experiency experience on the design and might be creating

more bottlenecks to the development team to deliver. I don't know how I say that. Okay. Also, we can't forget that sometimes you need to blame yourselves for bad decisions because if you like like if you are choosing a new framework for a new project and and you just choosing the new pad framework that has And and or also we don't didn't give attention, the enough attention for a framework that is dying, and we still pick that We need to accept that was our cons our consequence or other accept that as our decisions. Or even if we know that our application is not well tested. And we don't do anything to fix it, it's also our problem and we need to deal with it.

Okay, how can we deal with Properly with technical depth. After they find out those problems we have in our applications, we need to see what are the possible tools that can help us to fight to fix those problems. Imagine that if they stop at working on the Pizza Tower and bring everything down to rebuild it The chance of the same mistakes and even new ones mistakes could appear on the new tower is really huge. Not even think on the amount of money that was spent to bring everything down and rebuild it Also in the Pizza Tower, if they just postponed problem by adding more people to try to hang the tower on the other side to prevent it from leaning. It will be not to be sustainable

and in fact if we relate this to software development is worse because if you add more people to work like okay we decided that the Core team will work on fixtures and bug fix and we'll bring a new team to work on technical depth. That's the worst scenario because the people that I get into, they don't have the context of your software They don't know the stack. They don't know how your team works. So your core team will need to explain then the context and everything like that. So that's where the worst scenario. Don't do that please. What happened on the pizza tower was exactly like that. The towers were leaning and then they decided to stop the tourism until it was fixed. So if we did that to our software, it can be a good thing, but it can be a challenge to convince your managers to do, hey, we will

spend three months just cleaning up the house and fixing technical depth instead of doing new features and book fixes. It can be really hard to convince but If you if you can do that is awesome. But we can emerge a new problem on that. If we have these moments of stopping everything and cleaning everything It is not sustainable because imagine that if you are spent like four months working and then you spent you stop one month to fix to technical diet and so architectural problems. After four years, four months, you need to stop again. So the best solution for that is to create a culture in your company of refactoring in order to avoid that stop

in fixing and stopping and fixing. Okay, if we just create a new board on our Jira or Pivotal or any task tool that your team uses. If you just create a new board for technical depth issues, after some time it may get invisible for the team. Because imagine like if you your day-to-day backlog It's already invisible. You can see just like 10 tasks on your backlog. Imagine one that's separate from the main one. So it's not a that good idea. Okay, what can we do? So however those are solutions that most people do, we can think on better solution. So what options are this?

We have some based on everything we see, we need to come up with few feasible solutions for dealing with technical debt. The impact is a process of identify, mark, plan, act, and test on technical debt problems. As it says, the first one is all about finding it. So we need to look our code and ask for the dev that has more awareness on the code where we can find some technical depth points. Then the second one, we need to make sure those candidates are visible for the team by creating text on a board marked as technical debt issues, for instance. And we should should also set a priority on those tasks. And if you are using Jira, for instance, Jira

has a mechanism that over time It's increasing the priority of these tasks. So it's really aligned with technical debt because technical debt costs are increasing over time, so we are working together. Okay, and basing on that, if we mark those tasks, we need to stick to a plan and really stick to it. in order to work in a healthy way to deal with technical debt in still delivering things, which is a the hardest problem I think. After that, we should act on decreasing, like fixing, really the fixing this those depths. And the most important part is the last one. Because as any refactoring process, we need to test that we didn't break any behavior of our software.

I got that strategy for a book called Refactoring for Software Design Smell, Managing Technical Debt. It's really good in easy book. Okay, if you don't know how many tasks you should put on the backlog of our team of your own company. We should just use Pareto principle where just 20% of the teams productive is assigning to those technical debt tasks. And the rest can be used for like features, bug, or any other technical stuff that your developers team works. And if your process have some big releases like we have two weeks sprints and one release, you can do a post-mortem meeting after that just to discuss, it can be a quick

one, just to discuss like the technical depth. things that we faced or introduced on the code during that sprint. That's the that's nice because you can make The candidates for technical tasks earlier, you make them visible earlier, so we can we have more have less cost on those tasks. And there those are three examples of possible solutions, but I highly recommend you to be really critique about everything and see what best suits you. Because you need to fit that on your day-to-day schedule or culture of developers. And let's dig deep on the economical view of cost of working and dealing with a system with uh

some level of technical depth. So this is a shard that I fall in love with from day one that I see this. There is a this talk from JB Reisenberg on YouTube you can find this. It's the economic of software design. It's amazing. One and a half hour, but you can totally see it. And that talk he explains the cost of the next new feature on your software. How can we decrease the uncertainty of that cost? Because if I go to anyone here and say hey, how much your team takes to deliver a new feature? We can say that. We say two weeks. I don't know. And then this chart explain

why that's so hard to to to be explained to have a real number on that. And this chart if we look on that it has a comparison comparison of two lines where one of those is creating a software from the beginning with design and with thought As we can see, the growth of the cost of software that were built without design, without thinking about refactoring, good practicing. And and design in general, it grows really quicker than the blue one Even if the design first software architecture has a higher cost if you compare the beginning costs, the blue one has a bigger cost in the beginning, but after for

through time, this amount is mitigated It really pays off. So if you write your software without design, you are betting that this dashed line moment will never happen. Because when this when that moment happens The cost of implementing the next new feature will be higher than rewrite everything on your software. So if you want to bet on that it's okay, but you can think about design and prevent that Okay, poor quality software has become one of the most expensive topics in human history. Last year, CISQ did a report on US software costs. Which was calculated considering legacy

code, massive failure, cost of trouble in cancelled projects. And for us to just have a notion, this amount It's about two-thirds of the total health expensive in 2018 on the US. That's the how much is the cost of doing bad software? Programmers spent on average about 3. 8 hours a day on debugging bad or low quality code. Or if a code is that hard to maintain or read. So imagine all that money being used for like better better wage, social initiatives, training, and personal improvement. We can use that money bearer. This is Ward.

This is the guy that coined the term technical depth. There's a short talk, about four minutes. of him explaining the technical depth concept and where it came from. I really recommend you to have to look on that. And he created that concept on 1992, the same year that I was born. using a financial metaphor to explain for non-tech people the implied cost of additional work everything that we know today about good practice on software, agile, everything like that. passed through word before. Also word brought brought the week concept like if you have Wikipedia today is

is because Word did it. in 1990 coisa, it's something. And also he's an eva uh activist on agile and design patterns. So if we explain his concept, we can see something like that. And the definition of the depth metaphor Even if we have a great code with the best practice applied, that doesn't mean that we have a low technical depth. What he's saying is that technical depth is not a technical thing. In the if the code isn't aligned, if you you do your code with Less decoupled classes and design patterns and everything, that doesn't mean that your code doesn't have technical depth.

Because the main argument on his concept Is that technical depth is about the domain language and your your software. If the code isn't aligned with the domain of your project, we will always be on a technical depth. Because we will always continue to grow a software without thinking and worse then stop uh without not giving back knowledge that we gain through time. I mean If you start to to work on a project on a new project today, after two years, you have a completely different knowledge about the domain. If you don't break don't bring that domain, that knowledge on domain back to the software, you are creating technical debt also. So what you have beyond profit, besides like saving money and delivering things fast.

There are several things that we can identify as point of improv of troubles on your life. Caused by dealing with technical debt on a day-to-day job. Atrophy of the cist of the team's technical ability is a really important one because Who he already had or still have some code running on a deprecated framework or Python 2? Like inimaginable. And it's really a common scenario for us and that's hard to deal with. Because we need to accept the consequences of that deprecated software. Sometimes we need to copy copy and paste code in your own codebase to avoid of some bogey or something like that. Or even worse, usually we fork the original project, uploading in your GitHub, and start to install it from there

and fixing from there. So we have a lot of difficult scenarios to work on a deprecated code. Besides that, we could face some compatibilities issue with new tools like if you want to use Hiroku or something uh product as a service deployment tool for a deprecated framework it will probably won't be that integrated as Django is with Hirok and things like that. Imagine also how hard it can be for someone to be reallocated in the market Like if you someone passed like five, seven years working on deprecated software and then suddenly he sees that he's

completely out of market and try to become a jungle developer. Imagine how much concepts we have in Django that that deprecated software just does it don't have it. That can be a really hard thing to be. Also, a lot of anxiety and depression. Technical debt brings a lot of weight to our lives. as developers. Because if you are consistently dealing with bad code, a test that looks like, okay, that's a two two two day tasks. When you start to dig in the code, you see that like two weeks, one month. So that's bringing anxiety for you because you don't know When these kind of tasks will appear or some bug or some

weird comport behavior on the application, so we start to get the answer because we don't know if something like that will come up over time. And a healthy growth. Does it make sense for a company to grow like 300% each year if or developer team? doesn't scale and had the time and hands to work on that velocity. It's really a common scenario for startups to glamorize excessive working hours. because there were some high peak of purchase in the application and then the dev needs to wake up like three in the morning on a Thursday just to restart the server because the memory was like out of quota.

In a realistic and respectful world, pragmatic world world, the machines could take care like if you have a load balancing before that that could Starting in turn off some servers, so you you don't need to be that person or that thing because the machine will do that. And as just said earlier, you are not going to be able to do cool things burned out. So need to take care And now the Pizza Tower is now sustainable. This is one of the rare situations where a bug becomes a feature because It will not fail anymore and it became a monument like worldwide famous. And even though it's still leaning, but it's leaning on a controlled way.

But not like because of the everyone that tried to push in. It's because they engineered a solution that will hang that. And the same can happen with John software. If we apply all those feasible solutions, we can have the same scenario on a sustainable software, which is the best case. And all those reference references that I use to write this talk is on this this GitHub awesome list. the technical depth concept, the talk from G Bay Riceberg, everything the book that I get the impact And also a lot of others reference and I really would like for you to contribute it with some architectural and technical that

links with videos, talks, books, everything. So that's it, thank you so much.