Posting on Vercel Support’s recommendation — they closed our cases (#01kk8SOy6UKegLV1, #01vFrUSzTe6tgJgI, #01tyRQIB8ATKONvt) noting Container Images is beta and outside per-case support scope, and pointed here for the product/engineering teams tracking beta feedback.
Setup: FastAPI (Python) app deployed as a Dockerfile container on Fluid compute (vercel.json services → runtime: container). Pro plan, region lhr1. /health is a trivial handler returning {"status":"OK"} with no I/O. The uvicorn/FastAPI server handles high concurrency in a single process.
Problem: Per-instance concurrency for containers appears to be ~1. Concurrent requests each spawn a NEW cold instance (~18–29s boot) instead of being dispatched to an already-warm container.
Reproduction (production, single client):
Sequential — one GET /health at a time, 3s apart: all warm (8/8, 23–325 ms).
Concurrent burst — 12 parallel GET /health fired at once (mimics a page load firing many API calls):
| result | count | latency |
|---|---|---|
| warm | 2 / 12 | ~55 ms |
| cold boot | 10 / 12 | 18.7 – 29.1 s |
~83% cold under concurrency.
Why it matters: real traffic is concurrent — every page load fires multiple parallel API calls — so a container backend ends up cold on the majority of requests. Sequential access looks fine, which masks the problem.
Also observed:
- Intermittent
500 INTERNAL_FUNCTION_INVOCATION_FAILED— container provisioning occasionally exceeds the ~15s port-bind window (“could not connect to $PORT”). - Observability reports Cold Start ~21.6%.
Questions for the team:
- What is the per-instance concurrency for container deployments — is it effectively 1?
- Are there plans to bring Fluid in-function concurrency (currently Node.js/Python runtimes) to containers, so one warm container absorbs a burst instead of cold-booting N instances?
- Is there any current way (config or flag) to raise per-instance concurrency for containers?
Happy to share a screen recording of the burst test.