How do I manually save and load chat context properly with reasoning traces in Vercel AI SDK

AI SDK
,
1

I’m trying to implement AI to my application with persistent chat history. For the basics, I had to manually construct the user prompt and assistant response, including its reasoning details, and store it to an array of ModelMessage so the context persists.

Implementation Overview

Here is the code snippet for an overview, which is quite basic:

const context: ModelMessage[] = await loadContext(user_id);

// Construct a prompt
const constructedContent: Array<ImagePart | TextPart> = [];

// Append the user's text prompt
constructedContent.push({
    type: 'text',
    text: prompt,
});

const constructedPrompt: UserModelMessage = {
    role: 'user',
    content: constructedContent,
};

// Append the latest prompt to the context
context.push(constructedPrompt);

const outputs = await generateText({
    model: openrouter.chat("google/gemini-3-flash-preview", {
        reasoning: {
            enabled: true,
            effort: "medium",
        },
    }),
    messages: context,
    system: SYSTEM_PROMPT,
    temperature: 1,
});

// Append the assistant's response to the context
context.push(...outputs.response.messages);

// save the updated context
await saveContext(user_id, context);

For reference:

generateContent.ts:
This file has been truncated. show original
contextMemory.ts:
This file has been truncated. show original

Normally for models like GPT-4o and Gemini, I had to store the reasoning details as per recommendations from the official docs. I lazily pushed the response objects to the array of context, not just manually constructing the assistant response.

Saving works, as you can see right here:

Image
Image1534×839 53.1 KB

Image
Image1533×866 49.2 KB

Loading Chat History

Now when loading the chat history, I get this error:

Image
Image1605×932 115 KB

It took a while to figure out, but it is an Invalid prompt error. Using convertToModelMessages() doesn’t work as it is intended for UIMessage objects.

Trying to do so resulted in this:

Image
Image1604×933 103 KB

Workaround

The workaround is to remove the providerOptions field, which works, but it would also mean trimming the reasoning traces with it.

Would be nice if there’s practical guidance on how to overcome this; all I want is to make context persistence work including its reasoning traces. There’s not even proper documentation on how to do so.

Environment

  • @ai-sdk/openai: ^3.0.48
  • @openrouter/ai-sdk-provider: ^2.3.3
  • ai: ^6.0.142
5

Hi Wyatt,

I would split this into two stored representations instead of trying to make one MongoDB document serve both jobs.

For example:

1. UI/history record
   - what you show in the chat UI
   - can include reasoning parts, metadata, timestamps, model name, token usage, etc.

2. Model context record
   - only the clean ModelMessage[] you plan to send back into generateText/streamText
   - no UI-only fields
   - no provider response metadata
   - no fields copied blindly from the provider response

outputs.response.messages is useful for continuing a conversation, but I would avoid persisting the full object and replaying it forever without normalizing it first. If a provider adds metadata or reasoning-specific fields that are valid as output but not valid as future input for that provider, you can get exactly this kind of “invalid prompt” problem after loading.

A safer save step is to whitelist the fields you actually want to send back:

function cleanModelMessages(messages: ModelMessage[]): ModelMessage[] {
  return messages.map((message) => ({
    role: message.role,
    content: message.content,
  }))
}

Then store reasoning separately for display/debugging, not necessarily as part of the next model prompt. In many apps, the previous assistant’s final answer is what you resend as context, while reasoning traces are stored for audit/UI and not replayed to the model on every turn.

So the flow would be:

const context = await loadCleanModelContext(userId)

context.push({
  role: "user",
ModelContext(userId)

context.push({
  role: "user",
  content: [{ type: "text", text: prompt }],
})

const result = await generateText({
  model: openrouter.chat("google/gemini-3-flash-preview", {
    reasoning: {
      enabled: true,
      effort: "medium",
    },
  }),
  system: SYSTEM_PROMPT,
  messages: context,
})

// store clean context for the next call
await saveCleanModelContext(userId, [
  ...context,
  ...cleanModelMessages(result.response.messages),
])

// store richer UI/debug data separately
await saveUiTranscript(userId, {
  text: result.text,
  responseMessages: result.response.messages,
  providerMetadata: result.providerMetadata,
})

Also, you’re right that convertToModelMessages() is mainly for converting UIMessage[] from useChat into model messages. I would not use it on data that is already stored as ModelMessage[].

The AI SDK docs mention response.messages for conversation history here:

If removing providerOptions makes the prompt valid, that is a good signal that the loaded object contains provider-specific data that should not be replayed as-is. I’d keep that data in your UI/audit collection, but only end a normalized ModelMessage[] back to the model.