It would be great if the ai-sdk had an audit option where it could log information into a predefined schema so I can capture chat turns, tool invocation, cost, token usage, system prompt used, chat history.
Hi @redsquare, welcome to the Vercel Community!
This is a nice use case. Let me share it with our team.
Hey @anshumanb Thanks, I only ask because I am currently implementing it then thought how many other people are having to replicate this effort!
Thanks for sharing! We will prioritize observability after we ship v6. If you like, create an issue and ping me in it (@gr2m), I’ll make sure we track it for the observability milestone to make sure we incorporate your feedback
There are a large number of LLM observability tools out there that capture this for you and present it in a nice UI: