Hello I tried out the new botId feature. implemented checkBotId on some of the API endpoints.
There was quickly a outcry of issues from the users on production and had to revert. Users on corporate networks with vpn proxy setup, users on firefox are getting flagged.
Rather than IP whitelisting, is there a way to use the botId feature propoerly?
Hi @ghoshan-ai, thanks for highlighting this feedback. I’ve reached out to the team and they’ll take a look and reply here.
Hi @ghoshan-ai, this is what our team suggested:
The traffic inbound from those corporate networks looks bot-like because firewalls on these networks are likely acting as forward proxies that are intercepting, analyzing, and forwarding that traffic. In scenarios like this we see abusive scraper traffic intermingled with authentic customers but the presence of the firewall makes distinguishing between the two unreliable.
Regarding your indication of firefox users, it looks like you had two checks from firefox that failed Bot ID and both were behind the aforementioned forward proxy.
If you have a very high confidence that the botID detections are false positives then you can add WAF bypasses for the JA4 signatures of the challenged traffic. At present this kind of decision is best implemented in your WAF rather than as a global configuration within Bot ID or Bot Protection.
Hello @anshumanb
Thanks for the analysis. However, I cannot add the JA4 signatures of all our end users.
For now i will rely on pure rate-limiting as a mitigation for bot abuse on the platform.