UTF-8 headers are causing errors

Hello, I am getting the following error in Vercel logs: Vercel Runtime Malformed Response Header Error: Header 'cf-ipcity: LuleÃ¥' contains non-ASCII characters..

I can't stop this header from being passed into vercel request headers, so looking for solutions.

I don't think UTF-8 in header values is something too uncommon. Even Vercel docs mention that UTF-8 is fine: https://vercel.com/docs/errors/MALFORMED_REQUEST_HEADER

Validate UTF-8 encoding: Confirm that all request headers, especially cookie values, are valid UTF-8 strings. Non-UTF-8 characters in headers, particularly in the cookie header, often cause this error

Currently my Next.js does 2 things:

it serves the frontend
it rewrites some requests to get results from another service (which needs these headers).

Given UTF-8 headers are common, I think it's safe to assume it's a bug on Vercel side, but until it's fixed, how do I go around and patch it on my side?

Thank you.

Vercel Staff

It does look like we're inconsistent in rejecting non-ASCII while documenting UTF-8 is ok, I've passed this along to the team

In the meantime, you may be able to intercept this with middleware, which should run before the Vercel Runtime error fires. Can you try looking for that header and either deleting it or encoding it into ASCII?

https://nextjs.org/docs/app/building-your-application/routing/middleware#setting-headers

Vercel Staff

It looks like this header is actually malformed, in UTF-8 it should be Luleå and the problematic LuleÃ¥ is ISO-8859-1. You should be able to rewrite it using middleware

I am a bit confused, would that error not happen before it reaches middleware? The header that causes the bug is a part of request to Vercel, not a response.

Vercel Staff

Middleware runs on the CDN edge before requests hit the Vercel datacenters, so this should work. If not, let me know and I'll escalate internally for a different fix

okay, in the meantime I actually came up with a different workaround so now this header comes in a different format before it reaches Vercel, but let's assume this is going to work for whoever stumbles on this issue. thank you, Jacob

Vercel Staff

Awesome, glad that's working for you!

If you don't mind, our Firewall product team is interested in any use-cases you have where you need CF in front of Vercel. Anything we can't do within Vercel infra is a product gap we need to close, so if you could share a little more about what you're doing I can pass that along and we'll get it on the roadmap

a few things, I guess first of all - price, I find price on Vercel for network traffic and images high and unpredictable, so prefer to put a CDN like Cloudflare in front of all the assets, it's solvable by using a separate CDN domain (though images are buggy and still use the same domain even if assetPrefix is specified, had to write a custom imageLoader).

Vercel Staff

Regarding the original issue, we’re aware of this long-standing Vercel platform constraint and are currently rolling out a change to eliminate it entirely. Once this is fully rolled out (ETA: by the end of this week if everything goes well), you can include non-ASCII characters in response headers. And if Vercel Functions runtime finds non-ASCII characters, it will encode them accordingly (percent-encode, RFC 3986) instead of throwing 500

Vercel Staff

This has been rolled out to all Hobby/Pro teams on Vercel. You can redeploy and verify the change. Thanks

Hello, I am getting the following error in Vercel logs: Vercel Runtime Malformed Response Header Error: Header 'cf-ipcity: LuleÃ¥' contains non-ASCII characters..

I can't stop this header from being passed into vercel request headers, so looking for solutions.

I don't think UTF-8 in header values is something too uncommon. Even Vercel docs mention that UTF-8 is fine: https://vercel.com/docs/errors/MALFORMED_REQUEST_HEADER

Validate UTF-8 encoding: Confirm that all request headers, especially cookie values, are valid UTF-8 strings. Non-UTF-8 characters in headers, particularly in the cookie header, often cause this error

Currently my Next.js does 2 things:

it serves the frontend
it rewrites some requests to get results from another service (which needs these headers).

Given UTF-8 headers are common, I think it's safe to assume it's a bug on Vercel side, but until it's fixed, how do I go around and patch it on my side?

Thank you.