The Vercel Agent is currently both expensive and close to useless. Over the last 4 days, it used $52 USD and came back with ~4 code reviews that are mostly inaccurate, implementing them will break my code.
without knowing the full context. I created the test regex without the ending $ because I want /[locale]/landing-page/* to all get captured by the test.
Comparing to other offerings like BugBot and CodeRabbit, both cost under $60 USD for “unlimited” usage per month and returns ~5 to 10 reviews per PR, Vercel Agent’s suggestions are very expensive and cannot really be trusted.
Thanks for trying it while the feature is in beta. Sharing your feedback with the team so we can investigate why it didn’t work well for your team and make adjustments before the general availability release.
@wiiiimm Thanks William, this is very helpful feedback and apologies about the inconvenience. We’re going to be adding debouncing soon, which should help a bunch of reviews being triggered close to each other. We’re also looking further into the context issue you highlighted.
Let me know if you want me to test it again later… but i will need some more test credits. Burnt through all $100 USD now and I only had like under 10 reviews lol
@wiiiimm we just improved some workflows to help reduce the number of reviews which in turn reduces cost.
I’d recommend this flow:
configure Vercel Agent to not review drafts in Agent settings
open PRs as draft
push N commits to the PR
publish PR as ready
get a review
continue with PR workflow
Vercel Agent reviews every commit. If you are pushing many commits to a single PR, it may result in more reviews than you’d like. If a new commit does not have any issues, Vercel Agent will review but not leave any comments. The above workflow enables you to push many commits in draft state without getting reviews.
Email me casey.gowrie@vercel.com with your Vercel team ID and we can help look further.
configure Vercel Agent to not review drafts in Agent settings
This is how I had Vercel Agent configured from day one. I was working a lot last week so I don’t have a problem with Vercel Agent working overtime and charges for doing reviews. My concern was the lack of feedback (I got under 5 feedbacks and none of them was on point). In comparison, other code review products returned 10 to 20 suggestions (suggestions + logic flaws etc) on the same PR. I will drop you an email with my Vercel team ID later today.
Hi, in the UI, the only setting for which repositories to review is “All repositories”. What does that mean? Does it mean every repo from which a project in my org is deploying to Vercel?
The thing is, I’d like to use my own account to test-drive Agent at work, but there is no clear path for me to configure which repos and in which GitHub orgs this should run. Every other AI reviewer I’ve tried presents you with a list of repositories in which PRs are reviewed. How would I go about adding Agent to my work org?
Also, I find the credits model weird. I’d find it much more intuitive to simply have Agent usage billed at the end of the month, together with all other platform usage.
Enable Vercel Agent with $100 free credit for automated AI reviews” how do I stop these ads from showing up on our PRs?