413 Payload Too Large

HI there,

I'm on the hobby plan and have a serverless function to resize images using Sharp.

I know that there is a 4.5 MB limit, and have read here that this can be bypassed, using a streaming function.

I have also read here that on the Hobby Plan, using NextJS page router the following is true:

By default, Hobby users that deploy their Next.js Page Router function on Vercel will have streaming enabled for all functions.

The problem is that when trying to send a large file (e.g. > 5 MB) I still get the 413 error regardless of my implementation.

Is this just not possible or am I doing something wrong?

Thanks.

Hi,

This is unfortunately a limit from our upstream provider and can't be increased. You can learn more about how to bypass them here: https://vercel.com/guides/how-to-bypass-vercel-body-size-limit-serverless-functions

Hi . Thanks for the reply.

I did reference the document that you mentioned in my original post and in that doc it specifically states:

Often, it can be hard or impossible for you to reduce the size of it depending on your needs. In this case, we recommend using streaming functions, which don't have this limit. See streaming functions for more information.

Which is how I got to the point of reading the second document I cited in my original post and eventually posting here because I can't seem to get around the issue even when using a "streaming function."

It could very well be my implementation of the streaming function. I'm using Formidable and some streaming technique, but it seems like the function stops immediately when seeing a large payload.

If this is a "no can do," understand, I'm just confused because the docs seem to say that it's a "can do."

I'm not sure if it will help, but here's my function.

It works in dev with a 6MB file, but not in production due to the aforementioned limit.

import { IncomingForm, File as FormidableFile } from "formidable";
import { NextApiRequest, NextApiResponse } from "next";
import { storage } from "../../lib/db";
import chalk from "chalk";
import fs from "fs";
import sharp from "sharp";

const tempDir = "/tmp";
if (!fs.existsSync(tempDir)) {
    fs.mkdirSync(tempDir, { recursive: true });
}

function deleteTempDir(dirPath: string) {
    fs.rm(dirPath, { recursive: true, force: true }, (err) => {
        if (err) {
            console.error(
                chalk.yellow("Error deleting temporary image directory:"),
                err
            );
        } else {
            console.log(
                chalk.green("Temporary image directory deleted successfully.")
            );
        }
    });
}

export const config = {
    api: {
        bodyParser: false,
        supportsResponseStreaming: true,
    },
};

const handler = async (req: NextApiRequest, res: NextApiResponse) => {
    if (req.method === "POST") {
        const form = new IncomingForm({
            keepExtensions: true,
            uploadDir: tempDir,
            maxFileSize: 10 * 1024 * 1024,
        });

        form.parse(req, async (err, fields, files) => {
            if (err) {
                console.error("Formidable error:", err);
                return res
                    .status(500)
                    .json({ message: "Error parsing the file" });
            }

            try {
                const { userId, slug } = fields;
                const imageFile = Array.isArray(files.imageFile)
                    ? (files.imageFile[0] as unknown as FormidableFile)
                    : (files.imageFile as unknown as FormidableFile);

                if (!imageFile) {
                    return res
                        .status(422)
                        .json({ message: "Missing image file!" });
                }
                if (!userId) {
                    return res.status(422).json({ message: "Missing userId!" });
                }
                if (!slug) {
                    return res.status(422).json({ message: "Missing slug!" });
                }

                const singleSlug = Array.isArray(slug) ? slug[0] : slug;
                const imagePath = imageFile.filepath;

                if (!fs.existsSync(imagePath)) {
                    return res
                        .status(500)
                        .json({ message: "Uploaded file not found" });
                }

                const metadata = await sharp(imageFile.filepath).metadata();

                if (!metadata.width || !metadata.height || !metadata.format) {
                    return res
                        .status(422)
                        .json({ message: "No image metadata found!" });
                }

                console.log(chalk.cyan("Image height:"), metadata.height);
                console.log(chalk.cyan("Image width:"), metadata.width);
                console.log(
                    chalk.cyan("Original aspect ratio:"),
                    metadata.width / metadata.height
                );
                let useThisContentType = `image/${metadata.format}`;
                let transformStream = sharp();
                if (metadata.width > 992) {
                    console.log(
                        chalk.cyan("Resizing with original aspect ratio")
                    );
                    useThisContentType = "image/webp";
                    const originalAspectRatio =
                        metadata.width / metadata.height;
                    transformStream = transformStream.resize({
                        width: 992,
                        height: Math.round(992 / originalAspectRatio),
                        fit: sharp.fit.contain,
                    });
                } else if (
                    metadata.width === metadata.height ||
                    metadata.height > metadata.width
                ) {
                    useThisContentType = "image/webp";
                    console.log(chalk.cyan("Resizing with new aspect ratio"));
                    const targetAspectRatio = 4 / 3;
                    transformStream = transformStream.resize({
                        width: 992,
                        height: Math.round(992 / targetAspectRatio),
                        fit: sharp.fit.cover,
                        position: sharp.strategy.entropy,
                    });
                }

                transformStream.on("error", (err) => {
                    console.error("Sharp error:", err);
                    res.status(500).json({
                        message: "Error processing the image",
                    });
                });

                const targetPath = //redacted
                const bucketName = //redacted
                const uploadStream = storage
                    .bucket(bucketName)
                    .file(targetPath)
                    .createWriteStream({
                        contentType: useThisContentType,
                    });

                const readStream = fs.createReadStream(imageFile.filepath);
                readStream.pipe(transformStream).pipe(uploadStream);

                uploadStream.on("finish", () => {
                    console.log(
                        chalk.green("Processing and upload successful")
                    );
                    res.status(200).json({ imagePath: targetPath });
                    deleteTempDir(tempDir);
                });

                uploadStream.on("error", (err) => {
                    console.error("Upload error:", err);
                    deleteTempDir(tempDir);
                    res.status(500).json({
                        message: "Error uploading the image",
                    });
                });
            } catch (error: any) {
                console.error("error", error);
                let message = error.message;
                if (message.includes("File too large")) {
                    message = "Max size for image file is 10 MB";
                }
                res.status(409).json({ message });
            }
        });
    } else {
        res.setHeader("Allow", "POST");
        res.status(405).end("Method Not Allowed");
    }
};

export default handler;

I think I must be misinterpreting the docs to think that there is a way around a > 4.5MB payload by using a streaming function.

If someone can confirm or set me straight, that would be appeciated.

In the meanwhile, I'll leave my workaround here in case it might be useful for others facing the same issue.

Upload the image file to temp storage (e.g. Firebase Storage)
Call an api with the image path and get the file from storage in the serverless function
Process the image with Sharp
Delete the object in storage from step 1

This is an extra step and it takes a little longer, but it does work in production as there is no large payload in the api call.

Hey . The request body is still limited to 4.5 MB with streaming functions. It looks like that's the problem you were running into.

I'm glad you found a workaround! Thanks for sharing it