Enabling Fluid Compute broke my server

(edited)

Enabling Fluid Compute broke my server Worse, turning it off and redeploying did not fix the issue . I am using webpack to copy binary files used for tokenizing llm queries from a dependency (node_module) to the server folder. Everything was working on the deployment earlier in the day and previous deployments.

If I had to guess, the binary is being launched from a different directory than previously... or the file structures have been changed. My expectation is that the cloud environment should not produce a different result than the local server; Especially after reverting the feature. Please advise on how to handle binary files, and restore previous functionality for non fluid-compute execution.

Error: ENOENT: no such file or directory, open '/var/task/.next/server/data/bpe/v3/vocab.bin

webpack: (config, { isServer }) => {
        if (isServer) {
            config.plugins.push(
            new CopyWebpackPlugin({
                patterns: [
                  {
                    from: path.resolve(__dirname, 'node_modules/mistral-tokenizer-ts/data/'),
                    to: path.resolve(__dirname, '.next/server/data/'), 
                  }
                ],
              })
            );
        }
      return config;
    },

Vercel Staff

Hey . It sounds like we have some debugging work to do, so I moved us to a new thread. I think this deserves to be its own topic.

Do the deployments fail to complete, or is it an error you see when running the app? Other than enabling fluid compute, did any code or configuration change between the previous successful deployments and the broken ones?

I would expect things to go back to the way they were after turning off Fluid Compute. If you disable Fluid Compute in project settings and redeploy the last good deployment, do you still see the same error?

Running locally works fine with pnpm run dev
Local builds succeed with pnpm build
Deployments succeed
All endpoints work* ** Except the endpoint that reads from the filesystem. This causes a runtime error.

No code has changed around the endpoint that reads from the filesystem, nothing has changed in the webpack config. This has been redeployed and still doesn't work. If I check my local build, the files are being copied to the correct folder.

Steps that Happened: Working Config:

Fluid Compute - Off
US East

Broken Config:

Fluid Compute: On
US West

Still Broken Config:

Fluid Compute: Off
US West

Still Broken Config Mirrors OG config:

Fluid Compute: Off
US East

Either the build pipeline is no longer copying the files correctly, or the execution environment has changed and files are no longer in the same place.

Additional Context:

Using App router ** Experimented with outputFileTracingIncludes instead of webpack copy, but it didn't seem to work

    outputFileTracingIncludes: {
      '/app/actions-llm': ['./node_modules/mistral-tokenizer-ts/data/**'],
    },

The mistral-tokenizer-ts is being imported from a react server function /app/actions-llm. Within mistral-tokenizer-ts, fs is used to load the correct binary data depending on input arguements.

Vercel Staff

Thanks for sharing all of that detail. Is your repo public or do you have a minimal reproducible example I can use to dig into this more?

It is not public, this is our company's product, and we've had to disable features to keep this from affecting our beta users.

To reproduce > Create a client component in a next app that invokes the following server function. Deploy in a new vercel project, if issue doesn't reproduce, then enable fluid compute. The following should work locally

app/llm-actions.ts

'use server';

import { getTokenizerForModel } from 'mistral-tokenizer-ts'

export async function tokenize(message: string): Promise<{tokens: readonly number[], length: number}> {
    const tokenizer = getTokenizerForModel('open-mistral-7b')
    const tokens = tokenizer.encode(message);

    return {
        tokens,
        length: tokens.length
    }
}

next.conf.ts

import type { NextConfig } from 'next'

import CopyWebpackPlugin from 'copy-webpack-plugin';
import path from 'path';

const nextConfig: NextConfig = {
    webpack: (config, { isServer }) => {
        if (isServer) {
            config.plugins.push(
            new CopyWebpackPlugin({
                patterns: [
                  {
                    from: path.resolve(process.cwd(), './node_modules/mistral-tokenizer-ts/data/'),
                    to: path.resolve(process.cwd(), '.next/server/data/'), 
                  }
                ],
              })
            );
        }
      return config;
    },
}

export default nextConfig;

package.json

 "dependencies": {
    "mistral-tokenizer-ts": "^2.2.0",
    "copy-webpack-plugin": "^12.0.2",
}

reply to #5

Any further follow up? This broke a pretty major pipeline for us?

Vercel Staff

Sorry for the delay on this. I still haven’t found any other cases where a project that wasn't working with Fluid compute was still broken after turning it back off.

The compute team is helping me investigate. I also reached out to you in a private message so you can share the actual team and project name with me to help with the investigation

Vercel Staff

Based on the info you shared so far, a build cache issue seems to be a likely cause.

You can use vercel --force with Vercel CLI or add an Environment Variable VERCEL_FORCE_NO_BUILD_CACHE with a value of 1 on your project to deploy without the build cache.

What happens when you try that?

Vercel Staff

Hey . Were you able to get this working without the build cache?

Vercel Staff

I haven't heard back from you for a while. Hopefully that's because your project is working again

I'm going to go ahead and mark the build cache suggestion as the solution. But please let me know if that's not the right answer, and I'll dig deeper into the issue.

This topic is closed. It no longer accepts new replies.

webpack: (config, { isServer }) => { if (isServer) { config.plugins.push( new CopyWebpackPlugin({ patterns: [ { from: path.resolve(__dirname, 'node_modules/mistral-tokenizer-ts/data/'), to: path.resolve(__dirname, '.next/server/data/'), } ], }) ); } return config; },

Running locally works fine with pnpm run dev
Local builds succeed with pnpm build
Deployments succeed
All endpoints work* ** Except the endpoint that reads from the filesystem. This causes a runtime error.

Steps that Happened: Working Config:

Fluid Compute - Off
US East

Broken Config:

Fluid Compute: On
US West

Still Broken Config:

Fluid Compute: Off
US West

Still Broken Config Mirrors OG config:

Fluid Compute: Off
US East

Either the build pipeline is no longer copying the files correctly, or the execution environment has changed and files are no longer in the same place.

'use server'; import { getTokenizerForModel } from 'mistral-tokenizer-ts' export async function tokenize(message: string): Promise<{tokens: readonly number[], length: number}> { const tokenizer = getTokenizerForModel('open-mistral-7b') const tokens = tokenizer.encode(message); return { tokens, length: tokens.length } }

import type { NextConfig } from 'next' import CopyWebpackPlugin from 'copy-webpack-plugin'; import path from 'path'; const nextConfig: NextConfig = { webpack: (config, { isServer }) => { if (isServer) { config.plugins.push( new CopyWebpackPlugin({ patterns: [ { from: path.resolve(process.cwd(), './node_modules/mistral-tokenizer-ts/data/'), to: path.resolve(process.cwd(), '.next/server/data/'), } ], }) ); } return config; }, } export default nextConfig;