Enabling Fluid Compute Broke Playwright Scraping

I have been using playwright on my node.js app and set it up with https://github.com/sparticuz/chromium to get the scraping working with Vercel. Sometimes the scraping takes a long time and I wanted to try fluid computing to increase speed and give the function a larger max time to run. However when I enable it the scraping breaks.

The Error in the logs

browserType.launch: Target page, context or browser has been closed

[pid=21][err] /tmp/chromium: error while loading shared libraries: libnspr4.so: cannot open shared object file: No such file or directory

I found a user who seemed to have a similar issue but adding the env variable did not help me. https://community.vercel.com/t/puppeteer-errors-when-fluid-compute-is-enabled/6037

Here is my code for setting up the playwright browser

1import chromium from '@sparticuz/chromium'2import { addExtra } from 'playwright-extra'3import { chromium as pw, Browser, Page } from 'playwright'4import StealthPlugin from 'puppeteer-extra-plugin-stealth'5//import { chromium as playwrightChromium } from 'playwright'6//import { chromium as playwrightChromium } from 'playwright-extra'7import 'puppeteer-extra-plugin-stealth/evasions/chrome.app/index.js'8import 'puppeteer-extra-plugin-stealth/evasions/chrome.csi/index.js'9import 'puppeteer-extra-plugin-stealth/evasions/chrome.loadTimes/index.js'10import 'puppeteer-extra-plugin-stealth/evasions/chrome.runtime/index.js'11import 'puppeteer-extra-plugin-stealth/evasions/iframe.contentWindow/index.js'12import 'puppeteer-extra-plugin-stealth/evasions/media.codecs/index.js'13import 'puppeteer-extra-plugin-stealth/evasions/navigator.hardwareConcurrency/index.js'14import 'puppeteer-extra-plugin-stealth/evasions/navigator.languages/index.js'15import 'puppeteer-extra-plugin-stealth/evasions/navigator.permissions/index.js'16import 'puppeteer-extra-plugin-stealth/evasions/navigator.plugins/index.js'17import 'puppeteer-extra-plugin-stealth/evasions/navigator.vendor/index.js'18import 'puppeteer-extra-plugin-stealth/evasions/navigator.webdriver/index.js'19import 'puppeteer-extra-plugin-stealth/evasions/sourceurl/index.js'20import 'puppeteer-extra-plugin-stealth/evasions/user-agent-override/index.js'21import 'puppeteer-extra-plugin-stealth/evasions/webgl.vendor/index.js'22import 'puppeteer-extra-plugin-stealth/evasions/window.outerdimensions/index.js'23import 'puppeteer-extra-plugin-stealth/evasions/defaultArgs/index.js'24import 'puppeteer-extra-plugin-user-preferences/index.js'25import 'puppeteer-extra-plugin-user-data-dir/index.js'2627export const defaultHeaders = {28    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',29    Accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',30    'Accept-Language': 'en-US,en;q=0.9',31    'Accept-Encoding': 'gzip, deflate, br',32    Connection: 'keep-alive',33    'Upgrade-Insecure-Requests': '1',34    'Sec-Fetch-Dest': 'document',35    'Sec-Fetch-Mode': 'navigate',36    'Sec-Fetch-Site': 'none',37    'Sec-Fetch-User': '?1',38}3940export async function setupBrowser(): Promise<{ browser: Browser; page: Page }> {41    const playwrightChromium = addExtra(pw)42    playwrightChromium.use(43        StealthPlugin({44            enabledEvasions: new Set([45                'chrome.runtime',46                'defaultArgs',47                'iframe.contentWindow',48                'media.codecs',49                'navigator.languages',50                'navigator.permissions',51                'navigator.plugins',52                'navigator.vendor',53                'navigator.webdriver',54                'sourceurl',55                'user-agent-override',56                'webgl.vendor',57                'window.outerdimensions',58            ]),59        })60    )6162    const browser = await playwrightChromium.launch({63        headless: true,64        executablePath: process.env.AWS_EXECUTION_ENV ? await chromium.executablePath() : undefined,65        args: [66            ...chromium.args,67            '--no-sandbox',68            '--disable-gpu',69            '--disable-setuid-sandbox',70            '--disable-web-security',71            '--disable-features=IsolateOrigins,site-per-process',72            '--disable-site-isolation-trials',73        ],74    })7576    const page = await browser.newPage()77    await page.setExtraHTTPHeaders(defaultHeaders)7879    return { browser, page }80}

I just attempted to use Node22.14.0 instead and got the same result even with fluid compute off. With node20 and fluid compute off the scraping works fine.

Can you share a minimal repository so we can take a look?

So after playing around with it I was actually able to get the initial scraping part to work. However I noticed that the specific site I am scraping is giving me a 403 error only when fluid compute is on. It seems to be getting caught by cloud flare where it was not before. I am not sure how Fluid would effect the browser settings or stealth plugin I am using to setup the Playwright browser.

I created a much lighter version of my current repo but it contains the same playwright setup https://github.com/jedwards4044/playwright-test

If you send a GET request to this URL it runs the Vercel version https://playwright-test-psi.vercel.app/

Here is the site I am trying to scrape https://www.kennedyfloral.com/

I confirmed that the code worked until I turned on fluid computing.

Actually I fear that more things are now broken. My playwright success rate is at about 50% after it was close to 100% before. I even tried old deployment from a month ago and getting errors that I have never seen. Is it possible that flipping fluid compute on and then off could effect these old deployments?

So I am thinking this error was probably caused by Fluid Compute in some way since it started right when I enabled it. After a lot of generic playwright errors I finally found one talking about "insufficient resources". It seems folders in tmp/ that playwright uses were filling up and then the scraper was struggling to run in Vercel. I was able to fix it with help from this https://github.com/Sparticuz/chromium/issues/231.

Thanks for details. using /tmp is very hacky workaround and not very reliable. It also has very limited access and resource. I am glad you were able to fix the issue.