(edited)
Hey everyone, I'm facing a security challenge in my Next.js app:
-
Current Setup:
- Using
@edge-csrf/nextjsfor CSRF protection - CSRF token generated in middleware
- Token passed to client via headers and cookies
- Using
-
The Problem:
- Python scraper still gets 200 responses or copies requests from the network tab and tests it
- Need to prevent successful scraping attempts
-
What I've Tried:
- Implementing dynamic CSRF token generation
- Adding custom headers to detect scraping
-
What I Need:
- Robust solution to block scraping attempts
- Ensure CSRF protection works against automated tools
- Next.js-specific best practices for this scenario
Any ideas on how to strengthen our defenses? Thanks in advance! 🙏
middleware.ts
const csrfProtect = createCsrfProtect({ cookie: { secure: process.env.NODE_ENV === "production", sameSite: "strict", },});
// Next.js middleware functionexport const middleware = async (request: NextRequest) => { const response = NextResponse.next();
try { await csrfProtect(request, response); } catch (err) { if (err instanceof CsrfError) return new NextResponse("invalid csrf token", { status: 403 }); throw err; }
return response;};search/page.tsx
export default function SearcPage() { const csrfToken = headers().get("X-CSRF-Token") || "missing"; return <SearchClientPage csrfToken={csrfToken} />;}SearchClientPage .tsx
const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => { event.preventDefault(); let formData = new FormData(event.currentTarget); formData.set("csrf_token", csrfToken); formData.set("from", "search"); formAction(formData); };Response from copy request from network tab and test it with PowerShell or py
1:{"success":true,"message":"","data":{.....data}}}