Is it possible to expose attachements to tools?

Discussions
,
1

I’m trying to write a simple tool, that uses attachements uploaded by the user using the ai package.

But I can’t get it to work?


    const result = streamText({
        model: openai('gpt-4o'),
        messages,
        tools: {
            addUpdate: tool({
                description: 'A simple tool, that requires a user attachement',
                parameters: z.object({
                    name: z.string().min(1).max(200),
                    attachmentUrl: z.string().describe('Url of the attachment to add'),
                    // attachment: z
                    //     .object({
                    //         name: z.string().describe('The filename of the attachment'),
                    //         url: z.string().describe('The URL or data URL of the attachment'),
                    //         contentType: z.string().describe('The MIME type of the attachment'),
                    //     })
                }),

But it does not seem like the LLM can access user uploaded attachements? Is that true?

I noticed that if I access the messages

[
  {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": <some user message>
      },
      {
        "type": "image",
        "image": <image url>
      }
    ]
  },
1 Like