Mocks: Do's and Dont's - Julius Seporaitis

Speaker 1: But so we can start. Um and uh first of all, thank you for a very interesting talk. Uh mine is gonna be much much shorter. And uh one thing that I learned uh apart from expedition is that St. John's Day is uh celebrated what in one country that I know uh but more so I didn't know that Anyhow, but going into the talk now , a few abouts. And the mandatory blurb about me is that I work as head of engineering at a startup providing digital marketing recommendations. It operates mostly in the United States but has engineering team in Lithuania Where I am from. Before that,

Speaker 1: I worked at some better known companies. And if you like to read, I'm trying to come back to regular blogging. Finally, this is my first attempt at public tech talk circuit. So if anyone would like to give feedback, I would be very happy to listen. About the talk itself, uh, these slides were part of a bigger deck uh for internal knowledge sharing. Um The talk is named Mox, but the gist will be on how mocks relate to test scalability and code maintenance. Most of what I will talk about comes from experience, but there are some parts where I myself am

Speaker 1: actually a recent ponder. In the end, I will share a few day-to-day pitfalls related to mocks that may not be obvious to anyone who is starting out. So I hope this talk will have some useful tidbits for varying ranges of experience. And uh so mocks. Uh in terms of test maintenance, um avoid mocks as is reasonably possible. Mocks are deceptively easy to use, but for the long-term maintenance, they will become a source of waste. uh when it feels like working with the tests takes longer than uh actually implementing the feature being tested. A good test should verify that uh

Speaker 1: given a set of uh Inputs expected output can be observed. In most cases it should not really care how the code under test implements it So the first problem with using mocks is they leak implementation details into the test. For example, to test that a function A calls some function B, it is very easy to replace uh function B with a mock. But uh imagine t 10 tests like that and uh change comes removing B and it will require abating uh those ten tests when in fact it uh actually should not The second problem is evident from the previous example. Using mocks

Speaker 1: focuses the test case mock on the behavior of the function contract but on the fact that mock is called or not. And if the mocked out function, the B in the previous examples changes, the test won't just won't catch it So what are the alternatives? And uh well using the real thing underneath the tests would give the most confidence that uh the thing works, but it is not necessarily always viable due to performance um or complexity or some other reason. in which case writing a fake that is tested to have an API matching as closely as possible to the real

Speaker 1: service is probably going to save a lot of time for you Stubbs suffer largely from the same problems as mocks. They do very similar things and it is very hard to ensure that they behave as the real system or as a fate. But mocks must serve some purpose because otherwise why would there be a part of the standard library or have uh popular packages? So let's take a step back and think of what types of code can be under test. And here's one way to answer this at a very high level. state changing code that produces

Speaker 1: side effects and changes the state of the system. For example, send email, save a record. and non-state changing that most often just returns information like get user or get the file. I hope the idea is quite clear as it will become relevant in the next slide. Uh so the answer to the question uh before when to use mocks. Um well use mocks when you need to have some dependency return a value to get a state changing function into a specific state But only when real or fake dependencies are unavailable. And coming back to the terms from the previous slide, state-changing and non-state-changing functions

Speaker 1: It does not really make sense to test non-state changing functions using a mock. Consider this that if a function getUser returns user object has database query results mocked out. What does it test? It actually just tests that the mock and return statement were called. And hopefully I planted the seed in your mind about not using mocks, but you just made the trade-off and have to use them. So here are three most common pitfalls. One of the problem with mocks is they do not check the interface contract.

Speaker 1: This means in the test anything could be passed as a parameter and everyone would be none the wiser until something breaks in production. As you can see the test actually passes, whereas assume this is production Part breaks. That is because the mocked uh out function doesn't did not just uh pass parameters However, this issue is an easy fix by passing autospec through to the mock constructor. It will do a basic check if the inputs conform to the function contract in terms of whether positional arguments, required positional arguments are passed, whether no unknown keyword arguments were passed and similar.

Speaker 1: It will not check uh tag types though. Second uh Splitfall and an easy way to spend more time looking at the test output than necessary is by using uh mock. anyplaceholder Um so because uh it shows the difference uh between two uh asserted objects. Uh When you test the real data and expected data, it actually compares string representations of those objects. So in this example, you can see that two Differences are reported where C and B and

Speaker 1: N E and D. However, in reality, there is only one. C and B uh because any would match D. And the this uh example is quite innocent and trivial But anything more complex, very big hierarchical dictionary that you need to just skip parts of, and it may waste a lot of time talking from experience. And finally, the most frequent quote related to Python mocks is uh probably mock the right thing. Uh it is also one of the easiest to get frustrated about And in here

Speaker 1: there is an example service module and example test module. And as you can see, the first test function tries to mock the right thing, but it will not work. And the second example marks the function in the module in the service module where it is used and it does work but it uh it prevents reusability of this mock and will most likely lead to a lot of uh copy-pasted but slightly adjusted uh code. And the reason this is happening is because uh at the top of the file, the service. py, The function getScore is imported into the

Speaker 1: module scope. And when run falls it For all intents and purposes, the get score function is part of the service module, not the project shared dotils. And uh I won't go into details about uh Python scope, but uh if um I really recommend uh Reading the documentation on that and uh getting familiar with it, a lot of things will become clearer. So, uh how to fix this Thankfully the solution is quite easy, which means mock the right thing. And it means import module and call the function on the module. Uh I hope you can see the difference here.

Speaker 1: Um and the way it the reason it works is because the function is called through a module reference. So instead of function being inside service, we take utils and we call the function on utils. And uh the example test that you can see here, it actually mocks the function inside the tiles and it actually works. Uh and that is it. And uh Sorry for a slightly abrupt ending. One thing I would like to acknowledge is that I was actually a big fan of uh Mox until very recently and uh converted after reading uh this excellent uh uh recently published book from Google.

Speaker 1: So the talk somewhat uh disappointed you. I hope this free book recommendation will pay for it this time many times over. And apart from that, I look forward to your uh questions and feedback. Thank you.

Speaker 2: Thank you very much, Hulas. So does anyone want to post a question in the Q<unk>A? Um do you have a question, Marco?

Speaker 3: No.

Speaker 2: I think it was a very clear talk. I have well one other suggestive resource on that third problem you're talking about, about why your like uh mock doesn't work. There's actually a a blog post that I always point people to by Ned Batchelder called Why Your Mock Doesn't Work. I'll post the link here.

Speaker 1: Excellent then.

Speaker 3: Um okay while we are um waiting for um questions oh we have a couple of questions Okay, one suggests from Thomas, no question, but uh useful tips, thanks. Then Gregor, oh we

Speaker 2: Tom Granton.

Speaker 3: Okay, I think uh

Speaker 2: I think these are just in the wrong chat box.

Speaker 3: Oh, from Dump we have what is your preferred testing framework now?

Speaker 1: Uh PiTest. And uh in the mocks that I have used, I actually use standard library uh mock patch utility. Um however I sort of only recently skimmed back, uh went back and skimmed through my test box and uh relearned again, I would say, about Monkey Cache, and uh I actually find it uh very useful and uh very uh easy to use but like i said uh i changed my ways very recently and tried to avoid mods and tried to uh talk uh my team into using fakes And it did not require a lot of uh convincing as well.

Speaker 1: Everyone's seeing the benefits quite quickly

Speaker 2: Could you give us a little bit more information on what inside this Google software engineering book convinced you?

Speaker 1: I think the main insight that uh made me change my mind was that I actually noticed a couple of times where people would do one their intricable feature and then actually spend uh time changing went to tests and because we have quite uh quite quite high test coverage and I just didn't like it and uh on at the time I just didn't know how to solve this. Um And in this book there there is a whole chapter dedicated to uh testing, unit testing, testing, and actually like large integration tests. Um And uh one of those chapters had very, very convincing um

Speaker 1: arguments uh that basically So one way to avoid this requirement to go back and change all the test cases is to use fake. And remembering my experience in Wi-Pan, we had used face a lot. And as long as they match the real thing, we do not have to go. Uh through all the dust changing them.

Speaker 2: Nice. Um we have a question from Adam Steele. Oh well, do you want to leave these questions, Mark? You start.

Speaker 3: classic interview questions so like yeah uh take note if you are like looking for a job right now uh what are the main differences between mocks fakes and stops

Speaker 1: um So I don't know if you still think see the screen. But I anticipated this question. Um So I don't know if I have to read it to everyone, but if you can make a screenshot, I will maybe try and share this slide deck later as well. But this is the difference.

Speaker 3: Okay, um Yeah, another question from Agby. Can you give one of two problems that you experienced with mock?

Speaker 1: So like I said, uh one problem was that uh Um our test coverage is quite high and we used Mox a lot, which meant that any small change um to the implementation would require updating a bunch of uh a lot of tests essentially 10 sometimes 20 and this is just time wasted because uh not time waste time wasted and a noise because um you make a change you implement a feature and you usually test it uh yourself And can't tell whether uh you covered

Speaker 1: a lot of edge cases or maybe it's good enough. And then We're having to go uh back and fix 10 or 20 tests just to make the tests work uh is just uh un unreasonable waste uh that should be removed. And the this morning I I saw um an excerpt from Ken Beck post. Ken Beck uh works at Facebook and is quite famous programmer and uh something from that excerpt really struck a chord with me. He said that Facebook if a test fails and the system still works, they just remove the test.

Speaker 1: Maybe I'm not that rabbitful, but it does give food for thought.

Speaker 3: Well, yeah, on this subject, I don't have a question, just want to share my opinion. I think that the golden rule with Marks is to test external dependency. Like for me, the classic example of Mark is when you need to uh use an HTTP library to call an external endpoint. You can mock that because that's external to the system, but