Playwright Tests using GitHub Actions + Vercel

March 5, 2024

For a recent project, I wrote some Playwright end to end (e2e) tests which require a running web app connected to a database that contains seed data. This works fine locally, but I wanted to include it as part of the GitHub pull request checks. This meant we would need the app running somewhere externally with an accessible database. Once that exists, we need a GitHub Action to run the e2e tests against that application.

Vercel: Free App Hosting and Database

Vercel offers a free "Hobby" plan which gives you automatic deployments via GitHub along with a database. My app happened to be built using Next.js, so the integration was smooth, but they support other frameworks as well. The network and storage limits are not great, but should be adequate for a starter project, and you can move to higher levels if needed.

If you don't already have an account, create one and then sync your GitHub project. Once you do so, Vercel will generate deployments automatically, both preview and production, creating those deployment environments within your GitHub project (more info on that here). In my case, this instant deployment was a bit too quick, as I had to reconfigure some things on both the Vercel side and in the source code of the app to get it to work properly.

Prisma / Vercel Integration Issues

The default build command for my Next.js project generates a build, but Vercel caches that which can lead to issues with your Prisma client. There is a help article from Prisma which explains the solution. Essentially you need to add prisma generate to the Vercel build command prior to running the (app) build.

Create Database

Within Vercel, navigate to the Storage tab to create a new database. I chose a Postgres Database, and connected the database to my newly connected project. Within the advanced options of the connection panel, I set the "Environment Variables Prefix" to "DATABASE" as it then populated the DATABASE_URL environment variable with the value needed by my Prisma configuration. I also only chose to use this database instance with "Preview" deployments, not using the same database for "Production" deployments.

Database Migrations and Seeds

To run migrations and seed the database, a script was added to the project's package.json called ci:db. This script resets the database prior to migration, so we have a clean database on each deploy. A previous post was written about seeding databases. Using bun to run this script after the next build produced the complete build command for the Vercel project:

prisma generate && next build && bun run ci:db

With all this configuration in place, the next Vercel deployment had a functioning app connected to a clean database with seed data.

GitHub Action: Run e2e Tests

Playwright has documentation on how to run tests as GitHub actions, but we need to point Playwright to our Vercel deployment. To do so, I chose to use the wait-for-vercel-preview GitHub action. This action will wait for the Vercel deploy to complete, and provide as output the URL to the running application. An example workflow is then:

name: E2E Tests
on:
  pull_request:
    branches: [main]

jobs:
  e2e_setup:
    name: e2e test setup
    runs-on: ubuntu-latest
    outputs:
      preview_url: ${{ steps.waitFor200FromVercelPreview.outputs.url }}
    steps:
      - name: Waiting for 200 from the Vercel Preview
        uses: patrickedqvist/wait-for-vercel-preview@v1.3.1
        id: waitFor200FromVercelPreview
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          max_timeout: 300
  e2e_tests:
    needs: e2e_setup
    name: e2e tests
    timeout-minutes: 5
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repo
        uses: actions/checkout@v4
      - name: Install bun
        uses: oven-sh/setup-bun@v1
      - name: Use Node.js 20.x
        uses: actions/setup-node@v4
        with:
          node-version: '20.x'
      - name: Install dependencies
        run: bun install --frozen-lockfile
      - name: Install Playwright Browsers
        run: npx playwright install --with-deps
      - name: Run e2e tests
        run: bun run test:e2e
        env:
          PLAYWRIGHT_BASE_URL: ${{ needs.e2e_setup.outputs.preview_url }}

This will set the environment variable PLAYWRIGHT_BASE_URL to the Vercel URL. Update the playwright.config.ts file with an override for that environment variable:

  ...
  use: {
    ...

    baseURL: process.env.PLAYWRIGHT_BASE_URL || 'http://localhost:3000',

    ...
  },
  ...

Vercel Authentication

The wait-for-vercel-preview GitHub action currently doesn't support the Vercel authentication requirements (see https://github.com/patrickedqvist/wait-for-vercel-preview/issues/62). As a workaround, disable the authentication requirements:

• Navigate to: Vercel -> <project> -> Settings
• Click on the "Deployment Protection" tab
• Disable "Vercel Authentication"

Summary

New pull requests will automatically deploy the app to Vercel, reset, migrate, and seed the database, and run the e2e tests. PRs will take a bit longer to run, something you will need to take into account, but the result is only merging code that does not break e2e tests.