Github + Vercel + Playwright e2e + using Automation Bypass Secret

Struggling here to get my playwright E2E tests to run during GitHub Actions with Vercel's Protection Bypass Automation.

1: GitHub is Mono Repo with 4 deployments (devSite1, devSite2, prodSite1, prodSite2 2: Vercel deployments are automatic default config, on every push, pr, merge, etc. and I have not disabled those to use GitHub actions for builds and deploys. 3: I don't want preview urls (or any non-prod urls) exposed to the public, so Vercel Deployment Protections is enabled.

Now here is where I am stuck, Vercel Docs say 2 things:

Use protection-bypass-automation for e2e bypass, (and the example config is even for Playwright. -> https://vercel.com/docs/security/deployment-protection/methods-to-bypass-deployment-protection/protection-bypass-automation
Use GitHub Actions for running e2e (e.g. Playwright -> https://vercel.com/guides/how-can-i-run-end-to-end-tests-after-my-vercel-preview-deployment

But Vercel System Environment Variables are not available to GitHub actions, so I can't reference the VERCEL_AUTOMATION_BYPASS_SECRET in my GitHub actions YAML or my playwright.config.ts file.

So here's my ask: What am I doing wrong to implement the VERCEL_AUTOMATION_BYPASS_SECRET as described, or are the docs incomplete for usage with GitHub Actions?

Hi, ! Welcome to the Vercel Community. 😄

Thank you for your patience and the detailed context! Happy to help out here.

You're correct that Vercel system environment variables are not automatically available to GitHub Actions. However, there's a way to make this work!

Add the VERCEL_AUTOMATION_BYPASS_SECRET to your GitHub repository secrets
Create a GitHub Actions workflow file (e.g., .github/workflows/e2e-tests.yml)

1name: E2E Tests23on:4  deployment_status:56jobs:7  e2e-tests:8    if: github.event.deployment_status.state == 'success'9    runs-on: ubuntu-latest10    steps:11      - uses: actions/checkout@v212      - uses: actions/setup-node@v213        with:14          node-version: '14'15      - name: Install dependencies16        run: npm ci17      - name: Install Vercel CLI18        run: npm i -g vercel@latest19      - name: Pull Vercel Environment Information20        run: vercel pull --yes --environment=preview --token=${{ secrets.VERCEL_TOKEN }}21      - name: Get Deployment URL22        id: deployment23        run: echo "::set-output name=url::$(vercel ls --token=${{ secrets.VERCEL_TOKEN }} -m ${{ github.sha }} | grep -v 'Fetched' | awk '{print $2}')"24      - name: Run Playwright tests25        env:26          VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}27          DEPLOYMENT_URL: ${{ steps.deployment.outputs.url }}28        run: npx playwright test

Update your playwright.config.ts

1import { PlaywrightTestConfig } from '@playwright/test';23const config: PlaywrightTestConfig = {4  use: {5    baseURL: process.env.DEPLOYMENT_URL,6    extraHTTPHeaders: {7      'x-vercel-protection-bypass': process.env.VERCEL_AUTOMATION_BYPASS_SECRET8    },9  },10  // Add other configuration options as needed11};1213export default config;

This setup allows you to:

Run tests on every successful Vercel deployment (including previews)
Use the Protection Bypass Automation secret in your tests
Keep your preview URLs protected while allowing automated tests

Important notes:

1. The workflow is triggered on deployment_status events, ensuring tests run after each Vercel deployment. 2. The VERCEL_TOKEN is required for using the Vercel CLI in the workflow. Add this to your GitHub secrets, obtaining it from your Vercel account settings. 3. For monorepo setups with multiple deployments, you may need additional logic in your workflow to determine which tests to run based on the deployed project. 4. The VERCEL_AUTOMATION_BYPASS_SECRET is project-specific. If you have multiple Vercel projects, you'll need to manage multiple secrets. 5. Using the Vercel CLI to get the latest deployment URL is more reliable than relying on GitHub event data, especially for complex deployment scenarios.

Let us know how you get on!