Showcase: SinapsAI – Zero-Dependency In-Process Circuit Breaker & Failover Adapter for Vercel AI SDK - AI SDK

Hi everyone! 👋

I've been building LLM-powered applications using Next.js 15 and the Vercel AI SDK (ai), and love the clean developer experience of streamText and React hooks.

However, in production we constantly ran into two recurring bottlenecks:

Provider Outages & 429 Rate Limits: When OpenAI or Anthropic suffers a spike or outage, the entire Route Handler fails unless you hand-roll complex fallback logic.
Proxy Overhead: Existing AI gateways (like LiteLLM or Portkey) require deploying a separate Python/Docker container and Redis, which introduces an extra network hop (+80ms to +250ms TTFT) and defeats the simplicity of Vercel Serverless.

To solve this, I built SinapsAI (sinapsai/ai-sdk) — a 100% in-process AI Gateway and Control Plane for TypeScript that plugs directly into the Vercel AI SDK as a custom provider.

⚡ What it provides:

Zero External Dependencies: Pure Web Standards (fetch, Web Streams). Zero Docker, zero Redis required for single-node / edge execution.
In-Memory Circuit Breaker: Automatically trips OPEN on consecutive provider failures, instantly switching to your fallback model before user requests hang.
Multi-Provider Failover & Lowest-Latency Routing: Dynamically route between Groq, OpenAI, Anthropic, Gemini, and DeepSeek based on real-time response times.
Deep Reasoning Streaming: Seamlessly pipes reasoning_content deltas (CoT) into Vercel AI SDK streams.
Universal Edge Support: Runs smoothly on Node.js Serverless and Vercel Edge Runtime (edge-light).

💻 5-Line Next.js 15 Route Handler Example:

// app/api/chat/route.ts
import { createSinapsAI } from 'sinapsai/ai-sdk';
import { GroqProvider, OpenAiProvider, AnthropicProvider } from 'sinapsai';
import { streamText } from 'ai';

// 1. Initialize the resilient provider mesh
const sinaps = createSinapsAI({
  strategy: 'lowest-latency',
  providers: [
    new GroqProvider({ apiKey: process.env.GROQ_API_KEY! }),
    new OpenAiProvider({ apiKey: process.env.OPENAI_API_KEY! }),
    new AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY! }),
  ],
});

export async function POST(req: Request) {
  const { messages } = await req.json();

  // 2. Drop right into streamText with automatic multi-provider fallback
  const result = streamText({
    model: sinaps('llama-3.3-70b-versatile', {
      fallbackModels: ['gpt-4o-mini', 'claude-3-7-sonnet-20250219'],
    }),
    messages,
  });

  return result.toDataStreamResponse();
}

🔗 Open Source & Links
GitHub: [https://github.com/roedyrustam/sinapsai](https://github.com/roedyrustam/sinapsai)

GitHub

GitHub - roedyrustam/sinapsai: ⚡ Blazing-fast in-process AI Gateway & Control Plane with Circuit Breaker, Automatic Failover, and Unified LLM API for Node.js & TypeScript

⚡ Blazing-fast in-process AI Gateway & Control Plane with Circuit Breaker, Automatic Failover, and Unified LLM API for Node.js & TypeScript - roedyrustam/sinapsai

Documentation: https://github.com/roedyrustam/sinapsai#readme Vercel AI SDK Adapter Code: https://github.com/roedyrustam/sinapsai/tree/main/src/ai-sdk I'd love to hear feedback from the Vercel community! What routing strategies or edge cache patterns are you currently using with Vercel AI SDK?

// app/api/chat/route.ts import { createSinapsAI } from 'sinapsai/ai-sdk'; import { GroqProvider, OpenAiProvider, AnthropicProvider } from 'sinapsai'; import { streamText } from 'ai'; // 1. Initialize the resilient provider mesh const sinaps = createSinapsAI({ strategy: 'lowest-latency', providers: [ new GroqProvider({ apiKey: process.env.GROQ_API_KEY! }), new OpenAiProvider({ apiKey: process.env.OPENAI_API_KEY! }), new AnthropicProvider({ apiKey: process.env.ANTHROPIC_API_KEY! }), ], }); export async function POST(req: Request) { const { messages } = await req.json(); // 2. Drop right into streamText with automatic multi-provider fallback const result = streamText({ model: sinaps('llama-3.3-70b-versatile', { fallbackModels: ['gpt-4o-mini', 'claude-3-7-sonnet-20250219'], }), messages, }); return result.toDataStreamResponse(); } 🔗 Open Source & Links GitHub: [https://github.com/roedyrustam/sinapsai](https://github.com/roedyrustam/sinapsai)