Yeah, I think this should really be treated just as advisory. I just wanted to give you guys headup before any surprise. I will share more details later after testing this out for a week,
On Fri, 2 Oct 2026 at 08:15, Jungtaek Lim <[email protected]> wrote: > Maybe good to make clear that this is an auxiliary process and committer+ > must review on their own? The scenario I can think of is the author sees > the AI review and diligently addresses them, and the committer chimes in > later and says it's incorrect. The review content from automatic AI review > is no owner in terms of responsibility. > > I understand many of us are leveraging AI assisted review (and I'm almost > pushed to this way due to amount of PRs to review) but I expect the final > content (whether it is a code, review, written doc, whatever) would be > owned by the one who posts it, and have a responsibility with it - it's not > that LLM would have responsibility with it. This is great for double > checking but I hope this isn't going to replace the primary review. > > On Fri, Oct 2, 2026 at 6:59 AM Hyukjin Kwon <[email protected]> wrote: > >> Hi all, >> >> I've been running a trial of automated AI code review on new apache/spark >> PRs, and I'd like to share how it works and get your feedback. >> >> For now the reviews are posted from my own GitHub account (@HyukjinKwon) >> while I check that it behaves well. Once it's proven, I plan to move it to >> the https://github.com/spark-test account, so the reviews are clearly >> separate from my personal ones. I plan to run this for the next week. >> >> *What it does* >> >> Each review is a full, multi-pass review of the PR diff, run by Claude >> (via Claude Code). Findings are checked against the code before they're >> posted. They're posted as a normal GitHub review: a short summary plus >> inline comments, each labeled with a severity, e.g. "Non-blocking (P2)" >> or "Nit (P3)". Example: >> https://github.com/apache/spark/pull/59184#pullrequestreview-5379759787 >> >> *No LGTM, no approvals* >> >> - It never approves a PR and never posts "LGTM". Every review is a plain >> "Comment" review, so it doesn't count as an approval or a change >> request, and it has no effect on merging. >> - If a review finds nothing new, it posts nothing at all: no "no >> findings" or "all comments addressed" message. >> - It doesn't push commits or edit anyone's PR. >> - Human review and committer judgement are unchanged. Please treat its >> comments like any other reviewer's: feel free to push back or ignore. >> >> *When a review is triggered* >> >> - A new PR is reviewed shortly after it's opened. Draft PRs are reviewed >> once they're marked ready for review. >> - After a review posts findings, the bot waits. It reviews again only >> when the author pushes new commits. >> - It also waits while a PR has open review comments from anyone that are >> newer than the latest commit, so it doesn't pile on while the author is >> still responding to reviewers. >> - Replying in a thread without pushing doesn't trigger a re-review. >> >> *When it stops* >> >> - When a review finds nothing new. The PR gets a fresh review if the >> author pushes again later. >> - When the PR is merged or closed. >> >> *Early numbers* >> >> Over the first ~12 hours (Oct 1), it covered 10 PRs, including 5 >> re-reviews after the author pushed. It posted 7 reviews with 14 findings >> in total, mostly P3 nits. Four PRs came back clean and got no comment. >> Several authors pushed fixes, and the follow-up reviews found nothing >> new. >> >> *Feedback* >> >> Please let me know if the comments are useful, noisy, or wrong, either >> here or on the PR. If you'd rather it not review your PRs, tell me and >> I'll exclude them. I'm also happy to adjust when it triggers based on >> what the community prefers. >> >> Thanks, >> Hyukjin >> >
