When a message becomes “poisoned” (fails repeatedly and can’t be processed), there’s no way to manually remove it. Attempting to query or lease the message via API returns a 404 Message Not Found error, leaving developers with no option but to wait for the message TTL to expire (up to 7 days, or 24 hours by default).
In my case, the callback code belonged to an old deployment and the issue couldn’t be fixed retroactively. Switching from push to poll didn’t help either - poisoned messages are simply invisible to the consumer. I also tried removing the old deployment, and while the logs and alerts stopped, bill kept increasing (Queue Notifications).
This creates a poor developer experience during incident recovery scenarios where you need to clear problematic messages quickly.
The preventative solution is to set a smaller TTL and limit retries both on send and in the callback ahead of time-but it still feels limiting to not have a backup method to recover when things go wrong.
My suggestion is to add API endpoints or dashboard controls to:
-
View/inspect poisoned messages
-
Manually delete or purge specific messages
-
Clear all messages in a topic or consumer group
Thank you,
Wiktor
