Using useChat, when streaming the tool response output we find that the text deltas coming in are not marked as output from a given tool, only that the tools output is ‘available’.
Is there a way to flag incoming text deltas as being output from a given tool?
Approximate implementation.
// backend (configs omitted for brevity)const stream = streamText({ /* ... */ tools: { generateText: tool({ execute: ({prompt}) => { /* ... */ return streamText({ prompt, }); } }) }})return stream.toUIMessageStreamResponse()// front endconst useChatRefs = useChat({ onToolCall: (toolCall) => { // this does fire once, when the tool is called and before the response // starts streaming in console.debug({toolCall}); }, transport: new DefaultChatTransport({api: `${OUR_API_URL}/chat`}), onError: handleError,});We get messages with part with type tool-textGeneration however, even when the state is output-available, the output property is an object rather than a built-up text string (as in this example https://ai-sdk.dev/docs/ai-sdk-ui/chatbot-tool-usage#client-side-page)
{ "type": "tool-textGeneration", "toolCallId": "tooluse_nI1eu2t1R3-JaKZY3FfcmA", "state": "output-available", "input": { "prompt": "Write a two paragraph story about an ant" }, "output": {"_totalUsage": {"status": {"type": "pending"}},"_finishReason": {"status": {"type": "pending"}},"_steps": {"status": {"type": "pending"}},"includeRawChunks": false,"baseStream": {}}}Instead, the subsequent part (type: text) contains the building-up string for that tools streamed output.
See data chunks:
// notably, the tool _input_ deltas are sent to the front end...data: {"type":"tool-input-delta","toolCallId":"tooluse_nI1eu2t1R3-JaKZY3FfcmA","inputTextDelta":"an ant\"}"}data: {"type":"tool-input-available","toolCallId":"tooluse_nI1eu2t1R3-JaKZY3FfcmA","toolName":"textGeneration","input":{"prompt":"Write a two paragraph story about an ant"}}data: {"type":"tool-output-available","toolCallId":"tooluse_nI1eu2t1R3-JaKZY3FfcmA","output":{"_totalUsage":{"status":{"type":"pending"}},"_finishReason":{"status":{"type":"pending"}},"_steps":{"status":{"type":"pending"}},"includeRawChunks":false,"baseStream":{}}}data: {"type":"finish-step"}data: {"type":"start-step"}data: {"type":"text-start","id":"0"}// our generated story, as un-marked text-deltasdata: {"type":"text-delta","id":"0","delta":"The smallest"}data: {"type":"text-delta","id":"0","delta":" member of the colony,"}data: {"type":"text-delta","id":"0","delta":" Pip the ant, ha"}TL;DR:
How do we know that tool output is from a given tool on the front-end.