Cleanroom Software Engineering with Django

Good morning. Welcome to this session of software engineering with Django. My name is Francisco and my intention today Is uh that you get a very good idea of what uh thing room software engineering is. I don't know, maybe in the future in a party they offer free drinks for those who know what thing room software engineering is. or something. Well seriously my goal is uh uh you get to know can software engineering today and with together with its possibilities, disadvantages, and above all how JAM can be used in the context of this method. This workshop is more about functionality than performance or middleware.

I apologize for my English And I also apologize because I won't be checking my notes, so I won't miss anything that I want to share with you. And sometimes I'm going to be looking at the screen in my left. Okay. Swedest start. Our agenda today is as follows. Um The first is uh that we are going to see is do we really need another method or is uh Francisco, why you come and disturb our peace of mind? We are so happy with our methods now. The second item is what is clearly software engineering. Here we will see a theoretical

overview of the method, and then we will start an exercise. The third point is can the channel help with the method? And we we will see a practical uh application of the method using Django. The fourth point is can the method help Django? Can we reuse it in in our Django projects? And finally, we will have um part of for conclusions and questions. Okay Do we really need another method? This graph shows the adoption adoption of agile

practices. It is from 2015, the state of Scrum report, but still very valid and things haven't changed so much. You can uh see that almost 95 % of responding organizations, almost everyone uses Chrome, which is good. Four out of ten use canban. Two out of ten use lean development concepts. One out of ten use stream programming. Almost one out of ten use feature drive development.

One out of 20 use other methods like unified processor family and one out of 100 use other like Team Software Process, Waterfall. Authority is not a method, it's a lifecycle model. And also one of 100 use the dynamic systems development model. uh i'm sorry dynamic system development metal and the same for crystalline and crystal clear Now let's look at the other graph. This one is from um state of scrum 2017 -2018 and it talks uh about the value

what executive What executives value most about the STAM projects? Okay. So you're going to tell me what do executives know and They they may know nothing, but they are the ones that approved the budget. So uh let's see. If uh if we see here uh Seven out of ten appreciate the value delivered to the customer and six out of ten appreciate the flexibility and responsiveness of the method. And then we move to another part where four out of ten appreciates

quality, transparency. uh scheduling deadlines and visibility. And at the bottom we have cost, innovation, and improving with three out of ten. Well let's focus on quality. Quality is uh something that is uh of course appreciated in agile. Methods, actually the agile manifesto says that continued attention to technical excellence and good design improvements agility agility and it It is correct because nothing is less agile

than uh fixing defects all the time. Agile brought us many practices, but many good practices related to quality and like continuous feedback from Strum. or per programming for from string programming or clean code, um concept uh refactoring and testing practices like test -driven design, uh unit test, of course automation of tests And uh the whole part of devs. So uh it seems like everything is okay, right? Well, let's see.

This is a master thesis from Ipetro Kalane from South Africa And basically what he says is that Scrum is agnostic in quality assurance practices. And he sees this as an incomplete method. It does not provide any software quality assurance techniques and it is incomplete. And he concludes that the scrum needs to be complemented. It's like empty buckets which need to be filled. And it is not different from what the creators of the scrum says, that it is a framework.

Okay. So the important conclusion here is that the scrum needs to be complemented, especially with quality practices. And it is this is reflected in the fact that almost eight out of ten organizations um use uh complements from with another uh with other approaches yeah So how can we complement this Chrome with other practices? Well, uh one idea is to use extreme programming and peer peer programming practices or other methods. Let's see.

This is something from Papers Jones. Papers Jones is a specialist in software engineering. He specializes in measurement and he does a lot of publications about statistical studies. uh reliable source of statistical studies in in software development. So uh what does Kiper John says? He says that uh although Peer programming is uh good. Um it is more efficient to use static analysis than inspections.

Okay. Another conclusion is that testing only finds a third of the effects. That we when we when we use uh test cases, when we apply test cases, when we test, we also inject uh defects. And uh as an average uh we have an efficiency removal of eighty-five percent. And um For every 200 lines of code, we will inject about two defects. I want to confess I never liked the term

bug. Because it relieves us from responsibility. You know the term came about uh because there was a mouth eating the wires of the of this ENIAC computer and causing failures. a long time ago. And I like the term technical the better because it makes us think more about what is our responsibility. I won't dwell too much here. I just want to emphasize that effect are injected at any stage of the life cycle. It's the same if we use an iterative and incremental life cycle. There will be always analysis activities. and design activities

even if we uh do them at the IDE Agile methods have reduced documentation, but uh defect repair is a part where we inject about 80 percent 8 percent of the defects I'm not going um to go into much data on this slide, which is also a study of the niters. I just want to emphasize something that is obvious to other ones, that effects are more difficult to repair when we are already in production. And finally in this part I want to emphasize the effectiveness of code

inspections. If we look at the graph, the blue line. shows how we are getting the text the life cycle and how if we do not perform this is the field line If we do not perform inspections, we are leaving the factory mobile to the to the end. So, what is Green Room Software Engineering? Minroom Software Engineering is a method development method that emphasizes the production of high-quality software. In an economical manner, okay, without incurring in a lot of costs.

The important part is that it has its theoretical foundation in mathematics. And then it's just a curiosity, it comes from the hardware industry. uh the harbor production plants and is named after the clean rooms where you have to keep the clean air and avoid any contamination No, uh this is a short graphical can Roman Software engineering history. The creator of the metal is Harlan Mintz. Harlan Mintz, we used to say the only way for errors to occur in a program is by being put there by the alpha. No other mechanisms are known.

Programs can acquire bugs by sitting around with other buggy programs. He has his sense of humor. Harla Mills earned a PhD in mathematics at Iowa State University and became a professor of computer science at the Florida Institute of Technology. After that he became a research fellow at IBM and applied the ideas from the most influent thinkers. of the PR like partners, Niklaus Weir and Distra, along with ideas from mathematics to software development. Actually there is an hour with his name at the IEEE Computer Society

and his method to remote software engineering. was used with success um with by organizations like NASA and IBM among others. Now let's talk about the clear software engineering principles. The first one is that for this method, a program is viewed as a set of rules that express mathematical function. We will go into uh more detail later. The second one is the requirements specification and the sign is made via a special construction bricks colour boxes.

Third one is the this method emphasized code inspections using the technique of peer reviews as a way to find effects. For one is that it is the life cycle model, this method use is an incremental as well as an iterative development. For the testing part, the method used statistical setting and quality certification. So programs are rules that express uh mathematical functions. And when we specify a program ,

we specify a function that maps an input sequences to an output space. This will allow us to achieve completeness because we will map each element of the domain to at least one element of the branch. We will also achieve consistency because each element of the domain will map to at most one value in the branch. So we won't have two answers to uh to an input to put it that way and uh using the reviews we will also uh achieve um

correctness In mathematical analysis, the general concept of function application on mapping refers to a rule that assigns each um as we said each eleven of our first set um as a single element of a second set. If this look um very theoretical don't worry we will go uh to a practice later okay And let's talk about the construction bricks, the boxes. We said we will have three boxes, a black box, a state box, and a white box.

The black box basically has to deal with data, input data and output data. It is called a black box because when we design it, we don't care about what is inside. was what enters what goes out. The state box reflects the behavior of the system , which we can view as a markup chain. And finally, white box or clear box is the procedural part where we put the code that will create a response from the system. The black box. Here uh we talk about stimulus and responses. Which could be, for example,

menu option in stimulus could be, for example, a menu option selected by a user in in our system and the response will be an email sent or a template presented. Also could be symbols, could be a crumb job , launch, etc. An important part is the history, which is this age. We can see the important uh part of the history in example blackboard, which is a calculator.

If I enter the digit four , it is not the result is not the same if I enter um five times uh before or if I enter previously three plus before or two So uh history is important and also order is important here Now the state box. The state books can be implemented as a state transition algorithm. If we see it from the object-oriented paradigm perspective. We could have a single tone object implementing the state pattern for the whole system.

In the procedural paradigm, we can have a set of functions in a way to keep the current state of the system. Finally, uh we have the wider clearboards. In object-oriented paradigm, this will correspond to the implementation of the methods in an object Django it is implemented in our views and our models. This is where the logic of the code goes. Okay, we have a The instructions we know very well, which are here as though the code.

Now we say that here we use statistical testing because in this method the sewer is built as an classic model. Um there is probability uh related to each uh event, okay. We don't know if the user is going to do things in order. And we have we assign we know it with a certain probability and we know with a certain probability that the database will be available, etc. Okay. So uh for example we have a system with 10 um possible inputs

which in the worst a scenario could be in disorder, we will have more than three million possible permutations without replacement, which is impossible to test So what we do in this method is to develop a usage model of the system and then we estimate the probability of each transition. From here we will design test cases from the most probable scenarios to the least probable. We will also have to develop different usage cases. or use scenarios for different users or actors and for special cases. We can then represent the these

scenarios using macrochain models. Now, uh statistical sampling is probably using our industry, for example, in a bulk factory. You don't have to measure every bolt in order to find out the quality of the production, but instead you will use statistical sampling It is also useful when it is impossible to measure the whole population. Think about, for example, sea pollution. You have to take a sample. And in software, you know the saying that goes the only exhaustive test is the one that leaves the texture exhausted. Let's move to markup representation.

I won't skip stop here a lot because you are probably familiar with uh markup models. The markup model represents states of use connected by transition acts. that represents possible stimuli to the system under this. They are probably assigned it to each other. That's what we were talking about. And the probability represents how much um transition R can be selected from a state. For example, here you can see the probability associated with the transition from A state to the end state, which is 20%. Now

a little more about this team remote software engineering process. We will start with an analysis of the requisites. Then we will to the we'll move to the functional specification and in parallel we will develop the usage specification And there we can plan our iteration and start development. And Start to plan the test with the usage modeling and then we will move to the statistics called test certification. uh

i'm sorry i will uh like to give more time to testing to the the to this part of the testing part but it will require uh dedicated time Finally, this method is compatible with GI methods, with user stories, use cases. So am I I saw 29 tons and 110, unified process or object-oriented paradigm, vendor-oriented paradigm, and other other methods and disciplines. Okay, that's uh that's it for our theoretical

explanation. Now let's get into our sample. I will uh switch uh my screen now and we 'll share with you Another one. First, I want to say that I use this method with an organization where the software development area had five developers, a tester and a scroll master. They had a history of customer complaints because of defects and they also will struggle at the beginning of the project in order to find out what every developer

be responsible for. We then use a spreadsheet and like this one that we are going to see now. And it took us several days, but we could finish an especially a module, a module that will do a teacher assignment to courses. They they they want to develop the software and deliver it to the customer and the results were So good that even surprisingly they didn't have a problem with the distribution of job because the top-down approach we will see now And it was the customer, the first time the customer didn't find a defect. Now um

I ask you if it's possible to open your own spreadsheet and try to do it at the same time and try to find the answers by yourself. Let me um share it with you again because Now this for this example, let's say we are a company that buys uh sales used vehicle. Um Over the internet, we want to have a page on the internet

where the person who wants to sell his vehicle can obtain a previous valuation at the moment. To do this, you must register first. Once you register, you can enter the date of your vehicle, the year of the vehicle, the model, and the condition and the system will calculate a range of value you know from x to y um euros or dollars and when the owner consults the valuation he or she can schedule an appointment with a salesperson for a physical appraisal And the vehicle owner can reschedule the appointment to a date where there are available salesmen. After the appointment with the commercial

area, the owner can confirm that he or she is entrusting the sale to the company for He must sign an online contract and the company will notify the owner when the sale has taken place. The sale, the accountant department will make a depository minus the commission. Okay. From this description, you will agree with me that is it's not uncommon that this is the level of the tail that we get from the users , we will develop our requirements. We could use user stories or even use case if you want, but for this example, we will simply

use list of requirements. Now uh make a here a list of our our requirements if you can do it in parallel uh that would be great okay And the first one that I think of is that the system must calculate the value of a vehicle in a range. Okay. And we will assign it a an ID which is uh R1 requirement one and the second one for this part is the system must register users okay And we will assign it a

R2, okay. Uh let me see And what else? Um well the system must allow registered users. To consult the evaluation of the vehicle online, this would be uh requirement three and What else? The system must give the option to make an appointment with the salesperson. Okay. And this is uh this will be the Requirement four and so on. Okay. Users

can unsubscribe from the system and which is surprised. User may entrust the sale to the company, which is R6. And then What else? The company will notify the owner when the cell has taken place, which is uh R7 and and that that's enough okay so we have these uh these seven requirements that We they are at a high level, okay? So what the method says is now we have to um find out what are our uh inputs to the system

which are colored colored here um is stimulus or stimuli stimuli the plural and and then what are the responses to those stimuli okay So let's give an example. I'm going to use also a code here. It's stimuli one. uh stimulus one sorry and let's see uh what it could be well um let's say that the the owner of the vehicle because he's not yet uh A user he enters into our main page. Okay, we'll leave a column here for variables if we use it. And

then we will um See what an other stimuli we can well let's move to responses. What is the response that um the owner we'll get if he enters our main page. Well this is a very simple one. Let's say uh we'll show our landing page or our main page okay This is response one and uh we will have to annotate the stimulus. We'll see is um one five And so we will move to the next stimulus and which is The owner wants to register

as a user. Okay. And the third one will be the owner enter his data to register as a user. I know we have a module for authentication in Django, but still we have to take this into account. And the next one is the owner cancelled entering her data which could occur and and let's say the owner clicks on the link received by mail We send him a mail and he clicks on it to be registered and then um the owner logs in. Okay Once he logs in, uh

he can select enter your vehicle data. Okay. And What else can he do? He then registers the big vehicle data. Then he will select uh make an appointment, for example. Then he uh will make an appointment and with a salesman, then Or he could cancel the appointment later. Okay. That could happen too. Or um What else can he do?

He can uh change the appointment. Okay, we'll give an option to change the appointment. Also the salesman after he meets with the customer and checks his college, he checks the vehicle, he wants to enter a uh evaluation the the real the final evaluation okay so he selects enter evaluation then he enters evaluation Then user selects title sale because he has he needs to sign a contract. to so the company can sell his vehicle and the user effectively entitles the sales to the company. Then the salesman

confirms After some time the salesman sells a car, so he confirms sale. And uh Then what can happen is that that will be the end of it. Okay. So we will assign an ID to each of every one of these. stimulus okay just um identify them and now we can move to responses okay we saw that for example uh if the owner enters into our main page the response is showing our land page and what other response we will have. If he wants to register as a user, then we will

show our registration page or or we could notify him that he's already logged in. Okay. And What else? Uh what other response? If he enters his data, then we will show the situation said mail or what if he's already registered or the mail is already registering the system then we will show him an error message We have both possibilities here. And then we can use a variable that is is um let's call it is um Ready

just as an example but um are um variables Then uh next um next response we show him a login page and when after he clicks on the mail Then we uh shows the page for initial evaluation. We another response, we show him the result of this calculation We uh show the page for appointments and after he selects uh schedule a time then uh sends the email to the user and salesman

and Also, if he cancels the appointment, we will send a mail. Those are typical uh responses that we profiling this type of system. If he wants to change the appointment, we could tell him cancel it first because we need to cancel it and then make another appointment. And uh if the salesman selects enter evaluation, we source the evaluation page. After he enters the data, we notify uh the final valuation and then the if the user selects

entitled cell then we show him an entitlement page and we If he uh he signs the contract, then we send mail with the contract data to the user and we uh if the salesman selects confirm cell we will show the page of the confirmation of the cell if um The sale must confirm the sale. We will confirm and send information by mail to the user. And that's it. Okay. So we will send a code here for every response. And we will we can put the stimuli related to

every response, okay? And now we have finished with all the possible uh stimuli and responses of the system. This is our top-down approach. Now what are uh our valid uh sequence of actions? Okay. Uh we want the the final. uh the happy path is or the is that the owners goes in order so when he enters into our main page he wants He selects registering as a user. He enters his data and

the data is correct. And he clicks in the mail at the time he receives it and he logs in, then he enters, he selects the option of enter your vehicle data. He enters the data and it's correct. He makes an appointment. The salesman then goes to visit the client and then enters evaluation And then the client and the user selects and title sales, hint title shell to company, and then the sermon sells a hair. vehicle and then he confirms shell and that's it. Okay, that's a happy path.

But um is it um is it uh realistic that would expect the happy path to occur Or are we very optimistic? For example, can the user try to register twice? Yes, uh I know authentication systems can take a bit, but it's not here. Um can the user try to change an appointment without uh having confirmed one um it could happen or what if the salesman try to enter evaluation but the Client the user hasn't

entitled the sale to the company. Um What other scenario can you think about that goes out of the happy path? Okay , it can happen. And the important is if the happy path changes and the user takes another path Do the response changes and like in the calculator. Is it me that is very pessimistic or are we optimistic and thinking that everything we will have an as plan. Have you ever had a friend or family that

ask you for help with your with the mobile or with the computer and you say just restart the system I have done it many times. Why is that happening? Because sometimes the systems, the state of the system is inconsistent. And if you reset it, then you get to a consistent system and state and then everything works as as banned. But of course if we consider that uh we are being too optimistic and we should uh take into consideration every path possible, then we get into another problem.

Okay. Because If um let's see that if the owner enters into our main page and then wants to register as a user, that's uh the stimulus one and the stimulus two okay that's uh uh two stimulus at the same time okay that's um And but if we consider uh three, then we have three possible stimulus, and we have to take into consideration that we could have like And three

Okay. Uh for example, we we have uh here that it could change and Stimulus two could be at the beginning or stimulus three, and we talk here about permutations. Okay. And if we only have uh 10 stimuli or inputs and Only two we consider two at the time, for example, let's say here two at the time Then we are talking about 90 possible permutations without replacement, of course. But if we want to consider three at the time like we had before, we have 720 cases.

For at the time it's more than 5,000 possibilities and so on. So what do we do? Are we going to ignore this scenario? Are we going to um Trying to find out which are the most probable paths that the user can take. So our system Don't get into an inconsistent state. Well, uh fortunately, uh Clean Room Somewhere in Unity has a technique for this. It's called sequence development. Okay. So um let's say let's ignore this for for a moment

okay um and the technique calls it the length the length is the number of a stimuli that you consider at one time so We consider one here, one here, one here, one here. So this is um of len one. Okay And but if we do this, um we have this possible uh sequences at the beginning okay so one two or three that's this are the stimuli we have here okay and then um If we consider the length two, we will have the same, but um length three will

have uh combination permutation of three and so on. Okay. So um The technique says that then we must give an a response for every every stimuli. Now uh let's put uh a special simula here which I call uh S0 or limit initialization and then uh we will try to give a uh response for this okay uh let's say that we we need to initialize the system and also it serves us as a starting point okay so that's um And that everything that happens before this, we will have a response that we would call illegal.

It's a special response. And so the first one except the for the The initialization is uh illegal. Okay, and the response here that we don't want women initialize is uh we want to show our landing page. So um the method Google have to uh give uh an answer to any of this. And What does the technique help us with the number of permutations? Okay, uh for the length one, we will consider each of them, each one of these before the system initialization is illegal So we won't consider

them. Okay. So let's go to length two and then we have system initialization. Then we can consider uh some sequences uh they are valid okay let's see here the zero and is one which is uh the owner and their synchronous patient the owner wants to register Then we have response one. This is uh we will show our landing page and so on and on. Okay, so we will um But uh the technique says that we only consider uh stimuli that are not illegal or has an equivalent. Okay If a sequence has an equivalent, then we will have to process one, we don't repeat, so it won't go further to another length.

But this is how the technique goes. And also the technic the state uh the states of the system could be the uh stage we are now like showing our main page or the response we're giving at at every moment and the white boxes is uh we will put here the code for um every state and then we will have uh our code okay so that's how it works with uh uh spreadshe okay now um we're running out of time and now i want to share with you

uh The answer can can Django help us with this, okay? So See I'm going to share my screen and then uh sure um okay we have a database here then we have the the the stimulatic here that um we is the same ones that we saw there we have the responses Okay, uh here um including illegal and null which are special cases and we have uh the requirements of course that we saw and that's

okay and uh here uh Django helped us uh as a database of course it's better to You found that this way then in a spreadsheet, okay, but now let's go to the uh best part, okay So if we see the site here and these are seven requirements, and then we have the stimuli here, okay, and and the responses We see that we have the name of the response description and we have the requirements related and we will see now the sequences. So the sequences

is what we were seeing before, okay, and then um we have S0, S1 S2, S3. Now I have put a response for all of them. We said that nothing that happens before system initialization is legal. So all of these are illegal responses. Okay And then we have uh a responsibility response with Sherro, which is show our line uh display our home page. Okay. Then uh but where the help comes from Django is here because we have 20 sequences here at level one or length one.

But uh once all of them are finished and I'm going to finish uh this one and You say yes, okay. Then we see that we have 40 sequences, okay? So the system has calculated and for the only valid one here we have the the next sequence length okay so we have um uh a zero as zero okay which will be for example a non -response then uh we have uh zero s one a zero S2 and so on. Okay. So the system will allow me uh to calculate and elaborate the

The sequences once I have initial level. So it will help me like this. It will also help me to find out which sequences I haven't yet uh result okay and what are uh the legal sequences what uh if we have node sequences or or um So uh in and the most important part the sequences there are the going to be the the happy bodies the canonical sequence. Here we have like this one.

This is um But the sequence uh these are not because they are not uh finished yet. Okay. So uh we will see here only the home Now um let's uh see another example. I will um let me change the location here One another one and then um here we have another example we have um This is for another application. We have uh here 196 sequences, okay. And we have the canonical sequences that are only 29. This is only 29 cases that.

We have to take care of and that's all um And we can also see which sequences are illegal. We can check what sequences are have an equivalent response. So we don't We don't process them twice, we we make the equivalent to the to another response, okay? And so on. So this is how the San Django has helped me to use this method, the supporting. Okay Now for the for the part that can we use Django for this

in a project? We have this generic code. item that or menu option that is going to take everything that we have here and it's going to generate the state transition um Code and also will take care of all the equivalent and individual responses. So if if he finds this uh sequence Oh 114 in this case, then uh he will make it equivalent to this one and so on. Okay. And If we do generate the code, then we have

um we have the this code that I'm going to show you now because we're running And with this we will finish. Okay. So um we have uh generated this uh this code okay uh i won't get into much detail here but uh we this is all automatic generated this is the the state transition algorithm okay uh it it receives uh a the option from the menu okay and sometimes an item id And then we will have a dictionary for the

sequences that are canonical, they are valid at the end. And um the function they are going to call. Okay, this is all automatically generated. Okay. And and we also um would see if if we have to change the status after the response, we'll take care of it. And then we will also take care of the sequences that are illegal or if they have an equivalent. Okay. So let's see one of the responses. For example, this is uh a response that shows the login login page

okay and this is uh something that you know very well better than me but um it recurns returns uh a page and with some parameters This code is uh we we put it first in the in every response. So we uh we take it there and we only put it here But we have uh for show logic page we have uh the the sequence related. Okay, so What I did is to put the state of the system into the session. I don't like it very much, but

so we're taking care of the session, the state of the system in the session. So every time an stimula is received, uh it is uh added to the session and then we we see how to what to do with it. Okay So now for the final part, let me show you what what this looks like. This is a system we're going to deploy up soon, and this is a real case. So we have uh here like uh Options and every time we select an option, it goes through the state transition diagram and

I'm sorry, the state transition algorithm, and then it it makes a response that corresponds to to what we have. For example, if we We have an illegal response. It would show us here that it is illegal to the user. And so we will take care of all the system consistency. And I think I'm out of time, so um thank you very much for for attending this uh workshop and