At Symflower we started off writing our UI system tests with Jasmine, but switched later on to Cypress. This blog post summarizes our experiences with Jasmine and why it pays off to switch to a framework that better fit your requirements.
Generally when bugs appear in a UI, a bug report may look like this:
- Go to page X.
- Click element Y.
- See that Z does not work.
- Optionally, a GIF is provided to showcase the behavior.
Reproducing these steps in Jasmine on pages which have a more complicated build-up can be difficult to get right on the first attempt. Such tests require that you construct the components from scratch, declare all interacting components, set the correct states, include mock objects for some components, take care that asynchronous calls finish and make sure that two-way-bindings actually. Doing such a bottom-up approach is tedious for larger scale tests.
The following code snippet uses Jasmine to test that an error message is displayed if the wrong password is entered:
it('should show invalid message when password is invalid in field with type password', () => {
fixture.detectChanges();
const passwordField = fixture.debugElement.query(By.css('input[type="password"]')).nativeElement;
passwordField.dispatchEvent(new Event('focus'));
component.loginForm.controls.password.setValue('');
fixture.detectChanges();
passwordField.dispatchEvent(new Event('blur'));
fixture.detectChanges();
const invalidMessage = fixture.debugElement.query(By.css('.text-danger'));
fixture.detectChanges();
expect(invalidMessage).toBeTruthy();
});
There is a lot of boilerplate in there which is framework specific: We need to trigger the binding mechanism, query the element not just by CSS but also with obscure calls and dispatch events. We found ourselves in the situation where we need to manage the application state within the test to behave as it would in the real application. This results in significant overhead to get a reproducing failing test for a bug report.
Why did we choose Cypress over Jasmine?
The ideal framework would behave as closely as possible as someone who writes a bug report. We found that Cypress offers exactly this functionality. Cypress takes a top to bottom approach since it executes its commands in a running browser on the actually deployed application. There is no need to be aware of the used frameworks because Cypress simply interacts with the DOM: the HTML and CSS. Thus, circumventing the problems we were facing with Jasmine.
The following code snippet is practically everything that is needed to test a similar scenario:
it('Unsuccessful with wrong user account', () => {
cy.section('Try to log in with a nonexisting account', () => {
cy.login(wrongEmail, Cypress.env('symflowerUserDefaultPassword'));
cy.url().should('not.eq', Cypress.config().baseUrl + '/#/projects');
cy.get('.notification').should('be.visible');
cy.get('.notification .message').should('have.text', ' the specified user email or password is incorrect ');
cy.get('.notification-text-for-danger-and-warning').should('have.text', ' Send an email to the support and request help. ');
});
});
There is no manual setup of components, no mocking since we make real requests, and no handling of framework specific event loops. On top of that it is quite readable from the code alone without having to be an expert in a certain framework.
In addition to easy to write and read system tests, Cypress offers a range of useful functionality that we introduce in the following sections.
The extensibility of Cypress
In case you find yourself writing the same snippet over and over, Cypress is easily extensible by writing custom commands.
A login command may look like this:
Cypress.Commands.add('login', (email?, password?) => {
email = email ?? Cypress.env('symflowerUserDefaultEmail');
password = password ?? Cypress.env('symflowerUserDefaultPassword');
return cy.visit('/').get('.input-email').type(email).get('.input-password').type(password).get('.button-submit').click();
});
Since commands extend Cypress globally, we can use login command in any test: cy.login() for using user and password from the environment or cy.login('email', 'password') to overwrite the defaults with specific arguments.
The Cypress dashboard
Cypress comes with a dashboard for managing tests locally during development. Separate tests can be selected and upon writing new code these tests will be executed. Within the dashboard the tests are run in selectable browsers giving visual feedback about what is happening in the browser.
Cypress is easy to debug
Each step a test takes in its execution is recorded with a snapshot of the current state. These snapshots are then selectable via the dashboard and show the state of the browser during the selected snapshot. Visually showing the state of the browser during a certain event makes it easier to debug each intermediate step.
The Cypress Recorder plugin
The Cypress Recorder plugin allows to record each interaction with a browser to construct a working test without coding.
The following screenshot shows an example of a recording:
In this recording we click on a project of the Symflower web frontend and navigate to a certain directory. Such recorded test cases are a rough outline of a complete test case, e.g. they might lack the complete validation, but work nontheless for the testing context of every developer. This is a great feature since bug reports can be made by providing test cases even from non-technical people.
Visual feedback through screenshots and videos
Videos of failed tests can be recorded in order to easily identify how the application looked during the execution of each step. In our case the recorded videos are directly uploaded as assets within our CI. Hence, every failure can not just be debugged with textual logs of the test case execution, but also visually, providing us with quick feedback as to what happened.
Summary
We started off with writing E2E tests within Jasmine, which turned out to be too tedious and time consuming. The pain of writing web frontend and E2E tests came up so often during retrospectives that we searched for a better framework. Cypress gave us the opportunity to save time writing E2E tests and have tests available that are easily readable and maintainable. Jasmine is still used at Symflower for writing unit tests. Our learning from applying both testing frameworks is to always aim to use the right tools for the right jobs to increase productivity.
We hope you are as thrilled as we are to learn about all these testing topics. Please drop us your thoughts and suggestions at hello@symflower.com . We are especially eager to hear about the topics you would like to see in this series. 🤔
Since you read this far, you are definitely eager for more content. Register now to our newsletter to not miss out on any upcoming posts!