Millions of edge requests on my site thanks to Uptimerobot

[alex-r89 (Alex)]

Hi,

My site gets around 3-4k viewers a month but I noticed that someone has been pinging it with uptimerobot (noticed this due to the user agent in the logs).

They are pinging a non existent route in my app, /api/ping, which is resulting in millions of edge requests per month and millions of requests for 404 pages being served in my logs.

I have blocked requests to this route, as well as the uptimerobot user agent, using the Vercel Firewall. This has removed the pings from the Vercel logs entirely, however ,the requests to /api/ping are still being counted in my usage for edge requests - I can see 22,000 today to /api/ping (for context my / route only has 400 edge requests in this same time period), even though I blocked the request in the firewall yesterday. I have read online that denying a request path in the Vercel Firewall doesnt actually stop the edge requests or lower the edge request usage.

Is there no way to block these requests at all and limit my edge request usage? Or am I just at the mercy of this person who is pinging my site thousands of times a day?

[system]

There’s another community post with 404 debugging tips that might be helpful. Please give these solutions a try and let us know how it goes.

Debugging 404 Errors

Sometimes things don’t go as expected when deploying a website. If you see a 404 on your site, that means the page or resource couldn’t be found. There are many possible causes, and the absence of something can be tricky debug if you don’t know where to look. Here are some things you can do to find the cause and fix it. Debugging Tips Check the error code If you see a mostly white screen with 404: NOT_FOUND along with a Code and and ID then you can click the blue info box below the error deta…

A human should be around soon to offer more advice. But you can also get helpful information quickly by asking v0.

[amyegan]

Hey @alex-r89,

UptimeRobot is a website monitoring service. Is that something that you, or one of the other services you’re using, might have set up to track your site?

Keep in mind that some usage may be incurred from each request depending on the rule type and whether or not persistent actions are configured. More details on that here: Usage & Pricing for Vercel WAF.

Something else that might help is the new bot filter we recently launched in beta. If you give that a try, please let us know what you think in the Bot Protection feedback thread

Feedback Thread for Bot Filter Public Beta

Hi everyone, I’m Liz Hurder, Sr. Product Marketing Manager at Vercel. Today we are releasing our newest security feature into public beta, Bot Filter. [OG Card sm dark] Our Bot Filter managed ruleset prevents bot attacks in a single click. It is available for free on all plans. You can read more about it in our blog. During this public beta period, we want to hear from you. Post here if you have any feedback, feature requests, or experiences with Bot Filter. Thank you!

[alex-r89 (Alex)]

Hi Amy,

No its not something I, or anyone I know, has set up - I don’t know why the unknown person has done this.

I have not set up persistent actions as im not on the Pro or enterprise plan.

So it seems like there is no real way to stop the edge requests being used as, even though the Vercel WAF is blocking the request, its still counted as a use as its a custom deny rule?

This seems quite counter productive/intuitive. I dont understand why blocking a request from being processed would be a paid feature - if the request isnt being processed doesnt vercel’s usage drop too?

[amyegan]

When you use a custom rule to deny a request, there is no charge for WAF free feature use. But some incur Edge Requests (ER) and incoming Fast Data Transfer (FDT) usage may be incurred if those resources are used for routing the request.

You may want to add a robots.txt to your site. That lets you signal to web crawlers and bots that they should not access that page. You can even disallow the whole site for that bot if you like. This page has step-by-step instructions for how to set it up:

https://datadome.co/bots/uptimerobot/

Note that not all bots will respect that. But you can try out the new bot filter to add an extra layer of protection against malicious bots.