Testing Vercel Workflows?

(edited)

I’m trying to find out how to test workflows but can’t find anything (or really, Codex can’t figure it out). As I understand, there’s no guidance, docs, or examples (surprisingly 🤯). Has anyone figured anything out that works well?

I tried createLocalWorld which looks like the right direction generally but I’m running into problems, specifically:

WorkflowRunNotFoundError: Workflow run "wrun_01KFFTQVMXDA021KBEBGMR95ST" not found

what’s wrong with the emoji display and texteditor btw 🤔

Vercel Staff

The local world is the right choice for testing, it's what Next will use during development too. Do you see any of your test runs appear when you run the observability tools?

# View runs with CLI
npx workflow inspect runs
# View runs with Web UI
npx workflow inspect runs --web

Might be a misunderstanding – by testing I mean Vitest:

it("…", async () => {
    …

    const world = createLocalWorld({
      dataDir: `.workflow-test-data/${crypto.randomUUID()}`,
    });
    const run = await start(
      handleJobIngestion,
      ["https://jobs.example/1"],
      { world }
    );
    const result = await run.returnValue;

    expect(result).toBe("done");
    …
  });

Output

 FAIL  src/__tests__/job-ingestion.test.ts > handleJobIngestion > …
WorkflowRunNotFoundError: Workflow run "wrun_01KFGG0A08FDRTV1WHVRR25DYF" not found
 ❯ Object.get node_modules/.pnpm/@workflow+world-local@4.0.1-beta.26_@opentelemetry+api@1.9.0/node_modules/@workflow/world-local/dist/storage.js:207:27
 ❯ Run.pollReturnValue node_modules/.pnpm/@workflow+core@4.0.1-beta.39_@aws-sdk+client-sts@3.971.0_@opentelemetry+api@1.9.0/node_modules/@workflow/core/dist/runtime.js:113:29
 ❯ src/__tests__/job-ingestion.test.ts:66:20
     64|       { world }
     65|     );
     66|     const result = await run.returnValue;
       |                    ^
     67| 
     68|     expect(result).toBe("done");

Vercel Staff

I checked with the workflow team and we have an official vitest plugin on the roadmap, but in the meantime Mux got workflows working in Vitest using the Vite adapter

https://github.com/muxinc/ai

GitHub

GitHub - muxinc/ai: A typescript library for connecting videos in your Mux account to multi-modal LLMs.

A typescript library for connecting videos in your Mux account to multi-modal LLMs. - muxinc/ai

awesome, that’s what I was looking for – thanks!

I just (finally, after way too much digging 🤦) realized I can’t mock the network in my tests since the workflows execute in a separate process which limits the approach pretty substantially for real-world projects? Any news on official testing harness? I have to say I find it a bit surprising this has been out for so long without a clear and supported way for testing workflows.

I guess the easiest way to fix this for now is to just not enable the workflow plugin for tests and test workflows as regular functions?

import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    include: ["**/*.test.ts"],
  },
});

it("works", async () => {
  const result = await myWorkflow();

  expect(result).toEqual({ success: true });
})

it("…", async () => { … const world = createLocalWorld({ dataDir: `.workflow-test-data/${crypto.randomUUID()}`, }); const run = await start( handleJobIngestion, ["https://jobs.example/1"], { world } ); const result = await run.returnValue; expect(result).toBe("done"); … });

FAIL src/__tests__/job-ingestion.test.ts > handleJobIngestion > … WorkflowRunNotFoundError: Workflow run "wrun_01KFGG0A08FDRTV1WHVRR25DYF" not found ❯ Object.get node_modules/.pnpm/@workflow+world-local@4.0.1-beta.26_@opentelemetry+api@1.9.0/node_modules/@workflow/world-local/dist/storage.js:207:27 ❯ Run.pollReturnValue node_modules/.pnpm/@workflow+core@4.0.1-beta.39_@aws-sdk+client-sts@3.971.0_@opentelemetry+api@1.9.0/node_modules/@workflow/core/dist/runtime.js:113:29 ❯ src/__tests__/job-ingestion.test.ts:66:20 64| { world } 65| ); 66| const result = await run.returnValue; | ^ 67| 68| expect(result).toBe("done");