Github + Vercel + Playwright e2e + using Automation Bypass Secret

[thomasfalcon-cracker (Thomasfalcon Crackerbarre)]

Struggling here to get my playwright E2E tests to run during GitHub Actions with Vercel’s Protection Bypass Automation.

1: GitHub is Mono Repo with 4 deployments (devSite1, devSite2, prodSite1, prodSite2
2: Vercel deployments are automatic default config, on every push, pr, merge, etc. and I have not disabled those to use GitHub actions for builds and deploys.
3: I don’t want preview urls (or any non-prod urls) exposed to the public, so Vercel Deployment Protections is enabled.

Now here is where I am stuck, Vercel Docs say 2 things:

1. Use protection-bypass-automation for e2e bypass, (and the example config is even for Playwright. →
   Protection Bypass for Automation
2. Use GitHub Actions for running e2e (e.g. Playwright → How can I run end-to-end tests after my Vercel Preview Deployment?

But Vercel System Environment Variables are not available to GitHub actions, so I can’t reference the VERCEL_AUTOMATION_BYPASS_SECRET in my GitHub actions YAML or my playwright.config.ts file.

So here’s my ask:
What am I doing wrong to implement the VERCEL_AUTOMATION_BYPASS_SECRET as described, or are the docs incomplete for usage with GitHub Actions?

[pawlean (Pauline P. Narvas)]

Hi, @thomasfalcon-cracker! Welcome to the Vercel Community.

Thank you for your patience and the detailed context! Happy to help out here.

You’re correct that Vercel system environment variables are not automatically available to GitHub Actions. However, there’s a way to make this work!

1. Add the VERCEL_AUTOMATION_BYPASS_SECRET to your GitHub repository secrets
2. Create a GitHub Actions workflow file (e.g., .github/workflows/e2e-tests.yml)

Example

name: E2E Tests

on:
  deployment_status:

jobs:
  e2e-tests:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: '14'
      - name: Install dependencies
        run: npm ci
      - name: Install Vercel CLI
        run: npm i -g vercel@latest
      - name: Pull Vercel Environment Information
        run: vercel pull --yes --environment=preview --token=${{ secrets.VERCEL_TOKEN }}
      - name: Get Deployment URL
        id: deployment
        run: echo "::set-output name=url::$(vercel ls --token=${{ secrets.VERCEL_TOKEN }} -m ${{ github.sha }} | grep -v 'Fetched' | awk '{print $2}')"
      - name: Run Playwright tests
        env:
          VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}
          DEPLOYMENT_URL: ${{ steps.deployment.outputs.url }}
        run: npx playwright test

4. Update your playwright.config.ts

Example

import { PlaywrightTestConfig } from '@playwright/test';

const config: PlaywrightTestConfig = {
  use: {
    baseURL: process.env.DEPLOYMENT_URL,
    extraHTTPHeaders: {
      'x-vercel-protection-bypass': process.env.VERCEL_AUTOMATION_BYPASS_SECRET
    },
  },
  // Add other configuration options as needed
};

export default config;

This setup allows you to:

• Run tests on every successful Vercel deployment (including previews)
• Use the Protection Bypass Automation secret in your tests
• Keep your preview URLs protected while allowing automated tests

Important notes:

1. The workflow is triggered on deployment_status events, ensuring tests run after each Vercel deployment.
2. The VERCEL_TOKEN is required for using the Vercel CLI in the workflow. Add this to your GitHub secrets, obtaining it from your Vercel account settings.
3. For monorepo setups with multiple deployments, you may need additional logic in your workflow to determine which tests to run based on the deployed project.
4. The VERCEL_AUTOMATION_BYPASS_SECRET is project-specific. If you have multiple Vercel projects, you’ll need to manage multiple secrets.
5. Using the Vercel CLI to get the latest deployment URL is more reliable than relying on GitHub event data, especially for complex deployment scenarios.

Let us know how you get on!