Modern Web Testing: Going Beyond Selenium

Hello everyone, my name is Dmitry Vinick and I'm a developer advocate on the Facebook open source team. Thank you for joining my session today. Today I will talk about modern web testing and how to go beyond simply using Selenium with your test infrastructure. So let's go. Again, thank you everybody for joining. And as I said, today we'll talk about modern web testing. And what do I do? So I give you some of my of my credentials. My name is Nietzsche Set. I'm an open source developer advocate. What it means is that we are working on this. a Facebook open source program where we are trying to empower diverse communities through open source. Since you know this screenshot has been taken, we actually updated our website. So I invite you to go check it out at opensource. facebook. com And you'll find our videos, blogs, and the projects that we are advocating for.

I'm also focusing on mobile in particular, Android, iOS, and hybrid development like React Native And as I mentioned, some of the projects like Letho that helps you to build declarative UI frameworks, Fresco, and Flipper for debugging on mobile applications. But less about that and more about the open source. And again, I'm very much passionate about open source, hence I'm talking about testing today. Testing using open source technologies. What are our goals for today's presentation? I always like to establish goals because they will help us to establish our agenda. And first and foremost I'd like to uh for us to choose the right test context and know where we're working with. Then we need to choose right level of testing.

As I'll introduce you, hopefully you already know of test pyramid and how to actually apply it to web context. And last but not least is choosing the right end-to-end test approach, because that will be the most valuable, I hope, takeaway from this presentation is how to do end-to-end testing that satisfies your developer's needs. whether using Django for development or anything else. We will try to take an approach of an end user, regardless of what you are using in the back end or even front end. Alrighty, so let's get to a bit more details for this presentation and how do we usualize? That's an important question to ask. So we're all on the same page here. As I mentioned before, the test pyramid is usually the very good guidance and what people are introduced to when we are talking about testing.

First level, the lowest level and the widest of them all, meaning supposed to have the most number of tests. Unit tests. They are just testing a single function, a single unit of work. They are the cheapest to write and maintain, the fastest, but they give you the lowest level of confidence in your overall application, the system level. Then the integration test take place. This is where you test multiple functions in play, multiple components, how they interact with one another. And last but not least is end-to-end test. Going through the user scenario, your end user scenario that they will go through while using your application. And they are usually the most expensive ones, the slowest. But they give you the highest level of confidence. Hence you're supposed to use very few of them compared to other levels.

But how do we usually test web Real test pyramids, you know, I like the test pyramid as I just showed you, in ideal scenario. Reality is very much different. What people usually end up using is ice cream cone pattern, hourglass, or cupcake. So, ice cream cone. The ice cream cone it's basically inverted pyramid where you actually have the fewest number of unit tests, then you have service or integration tests, more of those. and a large suite of end-to-end tests. I actually worked with the companies before where this would be the reality and that's usually the reality for many. And on top of this cone you would have manual testing, which again is not very much scalable for large teams and large applications.

Our glass is similar. Again, you have uh actually lots of unit tests, lots of UI tests, but very few service tests. We can argue whether it's good or bad, but that's reality for many. And the cupcake. Cupcake is very much like the ice cone. Again, it's very much like an inverted pyramid. You have fewer unit tests, more integration tests. Lots of automation UI tests and a massive number of manual testing. Again, it's very much like the um the ice cream cone. But the idea is it's an inverted pyramid. Meaning that the pyramid that we all like to talk about is not the reality for most Uh so we need to change that. You know, we need to change these uh anti-patterns that some people would call it. I call it a reality, but I always like to hope for the best and you know um

advise us to change that. And the way to change it is uh first to actually test something. And what is that something? Throughout today's presentation we'll have this simple app, to-do app that usually people use for Lots of different UI frameworks. You know, you can find an example of to do with Angular, with React, with Svelte, any number of projects, any number of frameworks. And this is a good app for us to test. It's basically you add in the to-do, you check mark it, you delete it, you filter items by completed or active, etc. So back to testing. Now I introduce you what we're gonna test throughout today, and now let's actually see what we're gonna do. Um there are a couple of scenarios I want to introduce here and go through for today's presentation.

A first case would be imagine if you have a backend using Java and for many testing would be Java. You know in case of Python, some people use Python backend, they will also use Python for testing. I would say that there is a problem with this approach where you try to match with the backend for actual testing for the language that you use for testing. Uh because ultimately for web for web apps you're seeing it from the end user perspective. Even if you're writing end-to-end tests, even if you're writing unit tests, it doesn't matter. You don't have to match your backend with a testing. code and so that's why I'm suggesting us to move away from being bound to whatever is in the back end uh and just go away from it

and rather focus on hybrid model, domain-driven development, and user-centric testing. When I'm saying these words, what I mean by hybrid model is that Many companies in the past couple of years they went through transformation where they used to have testers as a separate team from developers So developers would write some production code, they would throw it at testers, testers will test it, give the feedback back to developers, and there will be like a fairly massive feedback loop, fairly slow one. And we want and people have been trying to, companies have been trying to avoid that, and they called everybody now software engineer and using this hybrid model where developers are the ones who test and testers are the ones who develop. There is no separation anymore.

It means that you are trying to use the software engineering principles even with the test infrastructure, but you are emphasizing the test expertise that testers bring. So it's not just the fact that testers don't do anything and developers just do the important work. No. Both parts can actually bring something important to the table and we need to find both of best of both worlds. This is what hybrid model is. And by again going away from a backend testing and just focusing again on the end user , this is why hybrid model is so important. This is the main-driven development principle that plays a role here as well The management design, uh one of the main ideas there is ubiquitous language. You're trying to speak the same language with your developers as you would with your end

users. When a user talks about an account And in the back end you call on it user. java or uh something else. There is this misalignment, there is that miscommunication that happened. You want to actually match what your end user would care because nobody cares what kind of great pattern you use at the back end. Ultimately it all ends up of what your customer gets. So customer is what we care about really here and we're trying to speak the same language as they. That's why instead of back-end testing, we do front-end testing even on the lowest levels. Because context matters. You don't want to do this context switching of how you test internally when you talk to the user. And again, it's all about keeping it user-centric, and I've been repeating that over and over again throughout this

couple of slides. If you're still not convinced why I'm saying that making sure that you match your back-end language with your testing language, modern web testing, focus on the web. It 's brings up the power of JavaScript. And I'll be emphasizing that regardless of what you have on the back end, throughout your application, throughout the pyramid, unit, integration, and end-to-end, you can use JavaScript. That's what Web is. for many of us is based on. You know, it can be PHP, it can be something else, it can be hack, but really JavaScript for majority of the web outside of like WordPress and such. And that's what I will be focusing on today. When it comes to power of JavaScript, I'll talk about assertion libraries, process libraries, and

enterprise libraries. So assertion libraries. Things like JAST, Jasmine, Process Libraries, it's behavioral-driven development libraries like Cucumber and Enterprise libraries like Apply Tools for testing. It helps us help us with visual testing And speaking of language of the web, JavaScript, in particular I'll talk about Node. js. So when it comes to Node. js, it's isomorphic. It basically helps you to write code for server-side and the front end. for the UI, then it's fairly flexible, obviously, and it's case customizable. And it all comes down to the uh power of uh NPM manager as well. You have packages. So you can basically reuse something that other people build and simply import

those packages and use them, whether it's for testing or development code. But going back to the pyramid, web test pyramid, it's very much the same as you know any other pyramid would be. It's uh with unit test, low cost, high speed, and Still quite a low level of confidence in the overall app. But for the unit testing, for the web unit testing, we'll use Jasmine for an example to showcase how it can be done for your app So um let's say for our to-do app we have this simple function that takes an input, it converts it to JSON and basically returns you know name and state here. Name of the to-do and the state active in a completed etc

When it comes to actually writing the test with Jasmine, we first initialize the test suite, create to-do item, then we create a test case where we accept valid test data We initialize the to-do item here. We basically pass some data to test, some data that we know of, brainstorm ideas, and the state is incomplete. And we do simple validation. We make sure the name matches what we expected after the app process it, after the function in this case, process it. That's what the unit test is. You're testing a single function Create to do item. And Jasmine lets you do that. And I didn't care how that function is processed on the back end, what kind of services are being invoked, with what. what kind of language you have at a back

end. Ultimately your end user wouldn't call the service directly. Your user would call this function on a front end, on a JavaScript site. And that's what I'm gonna test here. If I go up the level of the pyramid, an integration test in the web context, I would use Just, an amazing framework. It's a test framework. They have so many things there. I'll give you one particular thing called snapshot testing with Jest. It's a Facebook open source project by the way. So imagine you have this React component. That's kind of an old pattern of writing it, but imagine that we do. We have this create to do function. Create to do component, I would say, that has a rendon render function that returns you a button and it generates certain action on click

and has the label create And it also has some sort of styling attached to it. I know people usually don't combine comp you know component and styling, but this is for simplicity reasons just to show you visually what a component might look like in case you never seen React before, which is totally fine. And let's say if I were to test with just, I would create a test suite. I would say create a to do button is what I'm trying to test here. I render a component, basically I load it, I initialize it, and I call it, you know, by what it is, create to do. And I convert it to JSON and I save it as we call a snapshot. What snapshot is, is basically this uh piece of code. It renders a component, it renders everything in that component

as the web would see it. You know the function, you can see it's undefined because it's in isolation here. Name, the class names, uh, and styles. And if there was anything to be changed to this component, we would catch this after we validate these snapshots So to reflect on these two levels of the pyramid that we just looked at, every step has only one focus, and it's developers. Even though I kept saying that user-centric testing is important, I keep talking about those slower levels. of testing and their focus is still on developers. Good for some tests, this kind of focus, but not for end-to-end tests. For end-to-end tests we still very much care about our users. And When it comes to end-to-end tests, just to remind they are the highest cost,

lowest speed, but they give you highest confidence level And all these kind of three points have to be kept in mind throughout your writing tests. So to shift focus from developers towards your users is what we need to do when we are writing these end-to-end tests. So it might sound great to do this shift of the focus, but how do we actually do that? So end-to-end task testing is what we need is what we need to do. User-centric testing. It means that instead of interactions, we do tasks. We focus on tasks in hand. You do not care about, you know, for instance, we have a workflow of the user logging in. You don't want to describe your test as user uh types in , sends keys in for the username, for the password, a user presses the button.

uh to submit this form. You don't want that language to be used when you're writing your tests. Instead, user enters the username, user enters the password, user presses OK button. You do not go deep into like send keys if you talk selenium. You don't focus on individual interactions, you focus on the task at hand. Instead of if you're talking about particularly writing code, writing test code, instead of page object model, we're using something called screen play model. screenplay pattern where really it's how you write, how you speak, how you name your functions what matters. So but back to our apps. End-to-end testing The common way to do what's the common way to do end-to-end? Obviously it's Selenium Web Driver. It's at this point is a de facto

end-to-end tool. Great tool. Test it's basically a testing standard at this point, and it's generic, meaning it can be fit for most use cases, I would say. And more importantly, it uh shows you It has quite a high integration with plenty of other open source projects, plugins, uh weights, objects, etc. So it has a great community behind. It's an open source project. So saying a web driver, what it actually looks like behind the scene doesn't really matter to us as much, but ultimately what we care about, it has different bindings. for different programming language so what you'd like to use you can use but again I want to go away from matching your back end with your front-end testing

and also it has uh drivers for you know uh internet explorers firefox and whatever you'd like So since I've been talking about JavaScript in particular, the front-end testing for the frontend, I'd like to bring up a WebDriver. js And by the way, it is important how we kept how I capitalize GS. Because web driver GS with two capital J and S is a different framework altogether. G in the s the large capital G and uh J and small S is the uh official uh web driver um bindings for JavaScript So Sling of WebDriver. js, it's a still de facto end-to-end tool, still a testing standard, still generic, and has a great integration with the open source community.

And in terms of how it actually looks behind the scene, just the bindings are Node. js. Exactly what we want here. And that's what I'll showcase here. So we have a suite initialized, we name create to do We initialize the web driver, doesn't matter what exactly it does, just believe that it does initialize it. Then you create an actual test case, you're using the valid data to create it to do You make sure you wait for the page to load, then you send the name of the web driver of the um to do of the new to do and you basically uh what is in this case you make sure it actually appears and that's about all you made sure that it appeared on the web you edit a new to-do

and that's all Nothing special there. That's how you would do it with any other web driver bindings, Java, C sharp, whatever else. Unfortunately they're not just a good part to WebDriver. js. Uh with any end-to-end test complexities, there's always an issue of how you select, how you find your elements on the web, uh, how you locate them. What's the test flow like and what kind of single page application, uh what framework it can work for What about uh WebDriver. js though? WebDriver. js is great. I mean selector-wise, it has everything standard ID and CSS and XPath bindings for locators. It has buy allocators like buy dot for test flows it used to have something called promise manager

because node js by default is a sync you wouldn't want to have uh your tester or your developers worry about managing this asynchrony and so the promet manager allowed you to write this kind of sequential code before but we since removed it and now it's a wait and a sync. By we I mean open source community did. And it's still very generic, so it can work with anything It but still it looks limited though. I mean the buy selectors are great. ID, CSS uh selectors are nice But is there anything better? And for special case number two, let's say we have Angular as a front end. Angular is quite popular. And you know, when it comes to Angular, if you've never seen it before, which is fine.

For the way how it's written, you don't have just regular classes. You have special attributes like entry submit, model, change, etc. Basically, it helps you quite heavily to write a front-end. And in terms of actually testing Angular, we still use the same pyramid for unit testing, for integration, but for end-to-end testing we have something special It's not just WebDriver. js. We don't have to use just WebDriver. js for that. We have Protractor. Protractor is the end-to-end test framework for Angular apps. It's created and maintained by the Angular JS folks So ProTractor is basically everything the WebDrive. js has. But then it has additional things.

has still servers uh selenium server it has uh api for angular in particular but also it has connection to the angular js app it knows exactly when the app is loaded and ready to be tested, which is a great addition to any test infrastructure. For the protractor, selector wise, it has element bindings It has bi repeaters. In terms of the test flow, it knows as I said exactly when the app is ready. You don't have to wait for a particular element to appear. in order to test, which is a great help. And it's built uh intentionally built for Angular. In terms of tests Similar test create to do, validate the data, you load the browser for a particular URL, you find element, buy model, you send keys, and you make sure it appears.

You can see that how much less code I had to write, but I did ultimately the same thing. And I didn't have to wait for the app to load. All of that is handled for me. But Angular is great, but there are so many other UI frameworks. Obviously there is React, there is Vue, there is Amber, Backbone, Svelte, and many others. I think there are more UI frameworks that I can ever count But let's say we were to use Prochacta with React. You technically can. You can remove this ignore synchronization property it's called. Basically it doesn't wait for Angular app to load, because it won't ever load with React, right? But still it's it's kind of silly. You're trying to fit one thing into another that it wasn't built for it. Uh really

you You know, i I would say non-Celenium UI test frameworks is the next thing to try other than uh you know general uh web driver for JavaScript. And in here I'd like to showcase two things, test Cafe and Cypress. io. So what have we learned so far though? Patterns that patterns matter the most when it comes to testing. The common patterns are how you're handling weights, how you execute in parallel your tests, rapid test development, how quickly can I get my feedback from the test as a developer, especially in hybrid development teams. And can I record and do do I have an IDE for my test automation framework? In terms of test cafe? We

it handles tweights pretty well. It does allow you to parallel execute your tests. It has a great feedback loop, but gives you feedback fairly quickly. And it has a recorder and IDE for you to try To give you an example, if you were to write code itself for Test Cafe, you can you create a fixture what you're trying to test, you load the page, you can see this stream kind of API style of writing code, you initialize your test. You type a name, you click the button, you submit it, and you're done. I'm not gonna dive into particular code that Test Cafe use, but really the recorder that it has is quite nice it allows you to kind of uh if you have a manual team uh manual tester team at your company you can use them to basically record a bunch of these tests and then convert them to code to

code It's kind of a great first step to teach them how to write automation and it's also quite a nice way to debug your test if you wanted to. So if anything, the idea for test cafe is quite nice But Cypress IO is actually more, I think, getting more traction online lately. And the same thing, it handles weights very well. I can attest to that. It has a great parallel execution support. It gives you feedback extremely quickly and it does allow you to record your tests. It has a special plugin for that So if we were to write test code for Cypress, you know, for let's say a login page, something different from the to-do You basically create the test case for that.

You specify username and password. You can even uh tweak cookies right here If you wanted to. You can say you can get cookies, validate those, um, you can do basically anything you want. And Cypress has evolved largely from you know when I I first look uh took this um and made this uh snapshot. for the code for this presentation. And the Cypress again it allows you to record things fairly well. You can see in the browser on the site how it's executed. and how you can repeat that too. You can also pause it if you want to debug your tests. And uh yes, Cypress community has been growing rapidly, has lots of resources about it. So , important thing to remember is to choose the right tool for the right problem. Don't try to feed a thing into

a use case that doesn't work. There are so many test frameworks that I haven't even touched. WebDriver IO, Nightwatch 3S, WD, Nemo, so many. Avoid tool mix-ups if anything. Do not have different tools, test tools in your um test infrastructure. It increases the complexity When you're trying to test a complex thing with a complex test infrastructure, you're making it twice as hard, at least twice as hard. There is no ubiquitous language, there is no domain-driven design anymore. And there is no common language. So how do you choose what to use for your infrastructure? You can look at the open source projects at the GitHub pages, GitHub stars. how many npm load downloads it has, external integrations, and also give it a try and do the proof of concept first before you transition everything you have to use this framework.

But more importantly, choose flexibility and your use case. Build it for your use case. In terms of flexibility, what I mean by that is look at cost of transition. Let's say today you're using Angular and you use Protractor. But what if you know that your team, your company is transitioned to React in a couple months, in a year, in two? Or maybe you haven't actually thought of that, but it will happen. So maybe investing in Prochactor is not the best option So think of that. Think about return of an on investment. Does it even worth try switching to Cyprus if you're fully on Selenium? It might. Count hours, count time you spent on maintaining those Selenium tests. and then see if it valid, you know, justifies you spending time learning Cypress, moving to Cypress, etc.

Can you actually customize the framework you're using for use case, for your use case if it changes And can you replace it easily? In terms of again, thinking of your particular application, what's your team expertise? If you're all um you know non-JavaScript developers. Maybe again everything I said in this presentation of testing with JavaScript is nonsense. for your team. It doesn't worth the trouble to then write JavaScript tests. But if you are like a full stack, as they would say sometimes, uh team, then it might make more sense. Think of what the application uses. Again, that's what why it's important. If you're using Angular, if you're using React, it will play a major role of what test infrastructure you have.

Because test infrastructure is what matters. Can you actually support? Do you care about parallel execution at all? Does your you know um service do your service can handle that? And I always like to end my talks with call for action. Evaluate your current test infrastructure, test architecture Does it worth changing it? Have the main boundaries, know exactly where you're working. Are you going all the way to end-to-end tests, integration tests, or you can simply stay at unit tests and then don't bother of my changes here? Unify your test strategy. Think about it at the long run and again don't have the shifts between we have end-to-end tests. It's one framework you are uh unit test with another and integration we use some you know uh

old fashioned language for that so think about that as well have it as few hoops you have to jump through when you testing and debugging those tests as possible. Think of on onboarding someone new for your test infrastructure and gives you kind of a good perspective. But experiment first. You know, don't just uh convince your team to switch before you do the poor proof concept. Give it a try first and then see if it works. So thank you so much. Unfortunately we can't do Q<unk>A over the internet here, but if you want to contact me, please do on my on my Twitter, my blog, website, LinkedIn, or directly email me. Thank you for your time and I'd like to see you in the next Junga Con. Thank you.