Playwright is a powerful web testing tool supporting Chromium, Firefox and WebKit engines. The project has excellent documentation and many examples.

I use Playwright for E2E tests on the DEV environment. Today I'm going to show how you can integrate Playwright on GitHub Actions more efficiently.

The documentation suggests using the following snippet to run Playwright inside GitHub Actions:

steps:   - uses: actions/checkout@v3   - uses: actions/setup-node@v3     with:       node-version: '18'   - name: Install dependencies     run: npm ci   - name: Install Playwright     run: npx playwright install --with-deps   - name: Run your tests     run: npx playwright test   - name: Upload test results     if: always()     uses: actions/upload-artifact@v3     with:       name: playwright-report       path: playwright-report 

What is notable here?

• It's supposed to run all tests on the same type of GitHub Runner. ubuntu-latest by default.
• All supported browser engines (chromium, firefox, WebKit) will be installed on the Linux runner.
• Playwright will execute tests for all browser engines from the same runner.
• The last step will prepare an archive with an HTML report.

After several months of experiments I come up with the following configuration:

# ...  jobs:   test:     name: 🧪 ${{ matrix.project }} E2E Tests     runs-on: ${{ matrix.os }}     timeout-minutes: 20     strategy:       fail-fast: false       matrix:         include:           - project: chromium             os: ubuntu-latest             cache_dir: ~/.cache/ms-playwright           - project: firefox             os: ubuntu-latest             cache_dir: ~/.cache/ms-playwright           - project: webkit             os: macos-12             cache_dir: ~/Library/Caches/ms-playwright     steps:      - uses: actions/checkout@v3      - uses: actions/setup-node@v3       with:         node-version-file: '.nvmrc'         cache: 'npm'      - name: ⚡️ Cache NPM dependencies       uses: actions/cache@v3       id: cache-primes       with:         path: node_modules         key: ${{ runner.os }}-npm-${{ hashFiles('package-lock.json') }}      - name: ⚡️ Cache playwright binaries       uses: actions/cache@v3       id: playwright-cache       with:         path: ${{ matrix.cache_dir }}         key: ${{ runner.os }}-${{ matrix.project }}-pw-${{ hashFiles('**/.playwright-version') }}      - name: 📥 Install deps       if: steps.cache-primes.outputs.cache-hit != 'true'       run: make install      - name: 📥 Install ${{ matrix.project }} with Playwright       if: steps.playwright-cache.outputs.cache-hit != 'true'       run: npx playwright install --with-deps ${{ matrix.project }}      - name: 🎭 Playwright tests       run: npx playwright test --project=${{ matrix.project }}       env:         DEBUG: pw:api,pw:browser*      - name: 📊 Upload test results       if: failure()       uses: actions/upload-artifact@v3       with:         name: playwright-report-${{ matrix.project }}         path: playwright-report 

What I've changed in GitHub Workflow for Playwright tests?

• I use a matrix with three browser engines to run the jobs simultaneously. Instead of ubuntu-latest, I run WebKit tests on macos-12 runner. I had to make the change to avoid fragile tests. I noticed that the page.goto() API works differently on Linux and macOS for the WebKit engine. WebKit didn't wait for load event, which causes failures from time to time. At the same time, I've never caught such a problem locally on my MacBook. If you're interested, the ticket is still open in the Playwright bug tracker.
• The fail-fast parameter is set to false to wait for all testing jobs to complete, even if one of them fails for some reason.
• I believe it is essential to specify timeout-minutes for E2E tests so that the tests will crash if the infrastructure hangs or behaves incorrectly.
• I use the .nvmrc file inside the repository to specify the NodeJS version. DRY all over the place!
• Configured aggressive cache for npm dependencies and binary files with browsers. I added a special target to Makefile to calculate the Playwright browsers hash:

bump-playwright:   npm install --save-dev @playwright/test   npx playwright --version > .github/.playwright-version 

After executing make bump-playwright command, the latest PW version will be installed and the value of the installed release will be written to the .github/.playwright-version file. GitHub Actions calculate the hash for the cache from the file content. Example file:

$ cat .github/.playwright-version Version 1.29.2 

• Each runner installs only one browser and it's pretty handy cause the HTML report will have the results for that browser — easy to navigate and spot the issues. This distinction also allows you to have an idea of the test execution speed in each browser and to notice flickering tests specific to a particular engine.
• Playwright tests run with enabled DEBUG, so that in case of a crash, you can quickly get additional information for diagnostics.
• In contrast to the recommended configuration from the documentation, I prefer to upload artifacts with the report only in case of test failures. I don't see the point in storing those artifacts on the cloud, given how often the tests are run.

I shared my findings for configuring E2E playwright tests in GitHub Actions. Feel free to ask questions about the example GitHub workflow.

What Playwright tricks do you use in your daily work? Would love to hear about your hidden gems 💎.

Cover photo by Denny Müller.