Crawl requests from Google to /_vercel/insights/view

Help
1

Hi all,

In Google Search Console, in the crawl stats, I’ve noticed that thousands of crawl requests returned a JSON response, although I do not directly serve any JSON responses.

image
image926×746 24.7 KB

When I look at the detailed report of those requests, I see they are all being made to /_vercel/insights/view !

image
image1878×3766 419 KB

Now why would Google do that? I checked my code to rule out that my site accidentally includes this path in the sitemaps or as an internal link. This is not the case.

So what might be the cause of this?

4

Google will crawl any URL it finds, and the insights link is present in the page HTML

You can add it to your robots.txt

User-agent: *
Disallow: /_vercel/insights/
5

@kantarse-3906, I wanted to check in regarding the crawl requests to /_vercel/insights/view. Have you found a solution, or do you still need help with this issue? Let me know if that helps!

6

Hi @jacobparis thanks for the follow up. Yes, adding disallow to robots.txt solved the problem. I still don’t understand why Google so consistently crawled such an endpoint though.