Slack is a really good interface for AI agents
- it has good web/desktop/mobile apps
- you can upload images, videos, audio, and other files
- context can be organized into threads
- you can either work directly with APIs or with any existing Slack integration
I’ve started building my own open source AI Slackbot named Evil Jacob and based on Vercel’s official Slackbot guide
You can check these guides for the Slack setup, this article will be about the the bot itself
The bot needs to be able to consume all webhooks coming from Slack in order to receive messages, then perform actions either immediately or in the future
What makes it an agent is “agency” — the ability to decide on its own what to do. That means we pass the event that occurred to an LLM and allow it to call sequences of tools to construct its response.
Choosing models
The Slackbot template I started with came with OpenAI as the main LLM and Exa for web search.
I don’t have an OpenAI account so I swapped it for Groq/Llama, but I had a really hard time getting the tool calls to work reliably. About half the time it would work, and the other half it would output text like <function=searchWeb which probably means I’m doing something wrong.
I switched to Anthropic and Claude as my main LLM and then everything worked well
Durability
There are two rules when building webhook based background job systems
- webhooks are fraught with race conditions, assume you’ll be dealing with duplicates
- background jobs can fail and need to be able to retry
The general structure is to take all message webhooks, create a unique ID out of channel:thread_ts, and add each to a redis backed queue.
Then we process the queue transactionally
- if it’s a mention, add this thread ID to KV
- if it’s a thread, check if the thread is in KV
- process the job by passing to the LLM, and remove the job from the queue when it’s don
Vercel has an easy integration with Upstash that comes with a lot of this out of the box
- Upstash Redis is simple KV
- Upstash QStash is a simple queue
- they also have a webhook endpoint that I might try for ingesting the slack events
Handling Responses
The agent needs to be able to reply either to its current conversation right now, or somewhere else later. We can add a tool call to write delayed or scheduled jobs back into the queue, which essentially lets the agent talk to itself.
A prompt like this
Reply here, send me a DM, and tag me from a new post in #general
Should trigger three tool calls for scheduling new responses
Every morning at 8am, look up a coffee shop in Toronto and DM me the address
Should trigger one tool call for a scheduled cron job in Upstash, that delivers the same request every day
Future Features
From here on out it’s just adding features as I want them
- Pass images/files from messages to the bot
- Read and write from my Google Calendar
- Read my emails
It’s a bit of a pain switching back and forth from production endpoints in the Slack app to my local tunnel during development. I think I’ll end up making two Slack Apps, one for prod and one for dev. The prod one will check for the existence of the dev app, and refuse to accept jobs when it’s online.
What would your personal agent do?
You’re too good with this.