Structuring e2e Tests with Playwright

Playwright is a browser automation and testing library that is quickly gaining popularity. It is being developed at Microsoft by the same team that created Puppeteer, which for a long time was the go-to tool for browser automation, and is completely open-source. There are libraries for other languages as well (Python, Java, .NET), but this post will focus on the JavaScript version.

Essentially, Playwright consists of two parts: a library that can be used to automate browsers, and a test runner that can be used to run tests. The test runner is built on top of the library, and it is what we will be using in this post. Just to be clear: you can use Playwright without the test runner, too, if you want to use it for Puppeteer-like browser automation. In fact, it is what a big part of the functionality of my product Voulez is based on.

For the web application that we are developing at our company, I have been setting up a new e2e test suite using Playwright (it was previously based on Cypress). In this post, I will share some of the lessons I have learned while doing so.

Structuring unit tests

While Playwright is a great tool to write unit tests for web applications, we have already built an extensive unit test suite for our application using Jest. This works well and there is currently no need to change anything about it.

Here's how I would structure unit tests with Playwright, though:

• Have the tests in the same directory as the code they are testing, and have the test files named *.spec.ts.
• Use the test.describe and test functions to structure the tests.
• Use the expect function to make assertions.
• Apply the AAA pattern throughout the test cases.

When using test.describe, every test case (test) is run independently from the others, which is great for unit tests. Per default, Playwright runs tests inside a single test file in sequence, but you can also define to run them parallel using: test.describe.configure({ mode: 'parallel' }). This is useful if you have a lot of tests in a single file.

Structuring e2e Tests

In a typical e2e test, you want to test the application as a whole and not just a single component. This means that you need to set up the application in a certain state before running the test, and then clean up after the test has run.

In our case, the application is being built and run in a docker container, with a certain state being set up by running a script. That's why we have a special subdirectory for Playwright, using its own package.json. This way, in our CI pipeline, we can just mount this directory into the docker container and run the tests.

The directory looks like this:

- playwright
 - package.json
 - tests/
  - settings/
    - change-user-name.spec.ts
    - change-user-password.spec.ts
 - ...

We are grouping the test cases, which are named after the use case they are testing, in subdirectories for specific areas of the application. This way, we can easily run all tests for a specific area of the application, or all tests for a specific use case.

Inside a test file we are always using just a single test function. This is important because the different actions inside a test file are interdependent. Instead we are using test.step to structure the single test cases. This way, if a test case fails, we can see exactly which step failed.

test('Change the user name', async ({ page}) => {
  await test.step('Login to the application', async () => {
  await util.login(page, 'admin', 'admin');
 });

 await test.step('Open the settings page', async () => {
  await page.goto('/settings');
 });

 await test.step('Change the user name', async () => {
  await page.fill('input[name="name"]', 'New Name');
  await page.click('button:has-text("Save")');
 });

 await test.step('Check that the user name has been changed', async () => {
  await expect(page).toHaveText('New Name');
 });
});

This pattern is very descriptive and makes it easy to see what the test is doing. It also makes it easy to add new test cases, because you can just add a new test.step and don't have to worry about the order of the steps.

Different test files are run in parallel, which is great for performance.