Hi, thanks for the quick response.
We haven’t seen any logs or app-level errors, so it does sound like a firewall-level issue. I just checked the Firewall tab and will monitor for challenged or blocked requests going forward.
Regarding the agent:
- The request is coming from OpenAI’s ChatGPT, using their Browsing tool.
- I don’t control the exact IP or headers, but based on curl tests, it likely uses a User-Agent similar to:
Mozilla/5.0 (compatible; ChatGPT; +https://openai.com/bot)
- The request doesn’t include cookies, and may not include typical headers like Accept-Language or Referer.
For my platform, it would be more than ideal if AI agents could access and parse public data properly, not just for SEO, but also for integration into AI-driven search and discovery tools. Ensuring this access is reliable is a big part of my long-term strategy.
Is there a way to allow traffic from AI agents like ChatGPT or whitelist requests with known safe headers/IPs?