There's another community post with 404 debugging tips that might be helpful. Please give these solutions a try and let us know how it goes.

https://community.vercel.com/t/debugging-404-errors/437

A human should be around soon to offer more advice. But you can also get helpful information quickly by asking v0.

Regarding the challenges you're experiencing when inspecting articles, this is likely due to the inspection tool faking a mobile user agent on desktop and is unrelated to the soft 404 issue.

I want to let you know that we've opened an internal incident to pull resources from different teams to thoroughly investigate this matter. We're taking this seriously and are working on a solution. In fact, we're planning to release a tentative fix today specifically targeted at verified crawlers. This fix will increase the duration of our skew protection feature.

One theory we're exploring is that Googlebot's new caching implementation may be requesting resources that are up to 10 days old, which would have been removed from our edge network if new deployments have been created since then.

Hi Swarnava,

I am also experiencing these issues. Pages are constantly disappearing from google search results and being flagged as soft 404 in the Google Search Console. Requesting a re-index returns the pages after a couple of days but this is not feasible with thousands of pages. I had a few pages disappear again today.

This is a major problem as SEO is critical to our business. If there is any information you need to help with addressing this issue please let me know, I am happy to cooperate to get this fixed as soon as possible.

Thank you for your continued patience. Could you please ensure that the pages returning a soft 404 have an error boundary like this one? https://github.com/uncurated-tests/graceful-degradation-hydration-error/blob/main/app/gracefully-degrading-error-boundary.tsx

After adding the error boundary, please try recrawling the same pages again and let us know if it resolves the soft 404. We're suggesting this based on a soft 404 result another customer shared from their Google index results. Their screenshot showed us a client-side exception error, so we believe this adjustment could help. We appreciate your collaboration and look forward to identifying the issue together!

If soft 404s persist after adding the error boundary and re-crawling, could you please share a screenshot of the Google index crawler's URL Inspection page results? This should include what’s displayed on the page when the soft 404 occurs.

Hello, just a suggestion here that we needed to do. We did implement the ErrorBoundry as well as a safety net. However we saw that since the Skew protection only were handling up to 7 days of cache control for these chunks. Before the release of the hardcoded 60 days skew protection we saw additionally to this that Google were caching/using chunks that was part of a deploy that was made up to two month ago. So what we had to do here to help Google find these chunks, we needed to use the Vercel API and download all the old js/css chunks and release it with the deploy.

Really appreciate you sharing this, Sebastian. What you brought up about the short caching window + how Skew protection was interacting with that really is important. Your workaround; re-releasing the old chunks via the Vercel API, makes sense and highlights an issue that clearly more people seem to be (are) running into. Although; challenging if you are not skilled enough to pull that manouver off technically.

I really hope this lands with the Vercel team. Your example makes a strong case for making Skew protection available more broadly, not just for Enterprise plans. That support could help more teams avoid these headaches (read: nightmares).

Thanks again for digging into this and sharing! ⭐

Hello, is there a final solution suggestion for this issue from Vercel team? I am also having similar issues on my new application, built with next.js and published on Vercel.

When I test a URL on Google Search Console, I get plenty of such errors from Googlebot:

Interestingly, when I try the testing tool for another page that is rendered exactly the same way, I don't get any such error:

I am concerned about the indexability of my site, which is of course super important.

Some of chunks of my site encounter fetching error too, causing the html rendering failed

https://search.google.com/test/rich-results/result?id=NeTr9OVs1Uvs-QYPXOi-AQ

Can you make a new thread with more details?