AI Gateway doesn’t always use BYOK

I‘ve got AI gateway integrated with AI SDK and I’ve also added an OpenAI API key added to my org. I keep getting errors on my generations with an error message about rate limited free credits. But my understanding is that if I provide my own key, that should be used instead of the Vercel credits. Is there something else I should be doing to force my key to be used instead of the credits? I know my key is used sometimes because I can see the usage in the OpenAI dashboard. I’ve got plenty of credit there, so I shouldn’t be getting rate limited or using free credits at all. But it keeps happening every handful of calls.

Thanks for any help!

Some extra info: if I take AI_GATEWAY_API_KEY out of .env and use the OpenAI provider directly with AI SDK the periodic errors go away. My local is using the same OpenAI API key as what I put into the AI Gateway integrations. Testing the key there works. From my POV, this seems like an AI Gateway issue.

Vercel Staff

OpenAI API has its own rate limits, so that could be part of what you're hitting.

https://platform.openai.com/docs/guides/rate-limits

With BYOK, you should add the API key in AI Gateway integrations settings. You'll see the following message when adding your key:

Your API key will handle all OpenAI requests. If rate limits or errors occur, requests automatically fall back to system credentials.

This topic is closed. It no longer accepts new replies.