Skip to main content

Agents comment on pull requests but never give a verdict

With agents able to work on pull requests that already exist (ADR 0016), people want them to help review their team's: read the change and leave comments where they belong. We decided that a role holding the new pull_requests.comment permission gets review_pull_request, which posts one review on a pull request on its task: a summary, comments on lines, or both. The review only ever comments. The tool has no way to approve or request changes, so an agent's review never counts towards merging and never blocks it; what merges is still decided by people.

Considered Options​

  • Letting agents approve or request changes: rejected. An approval is what lets a pull request merge, including through merge_pull_request; an agent that could approve could approve its own work. Requesting changes would block a person's pull request on an agent's say.
  • Comments on any pull request, by number, without checking it out: rejected; the agent would comment on code it has not read, and nothing on the task would show what it reviewed.
  • One comment per call: rejected; each would notify people separately, where a review arrives as one.
  • Replies in existing threads: left out for now; Forgejo's API cannot post them.

Consequences​

  • A review is posted under the forge token's account, often a person's. Its summary therefore ends with a line naming the role and linking the task, so it is not read as that person's own; a review of line comments only gets a summary for that line.
  • A review is pinned to the commit the agent has checked out for that pull request, its superproject's or a submodule's. It is refused when the pull request's head on the forge is another commit, or when the agent has not checked its branch out, so a comment on a line is always on the code the agent read.
  • To review a pull request it did not open, an agent first checks it out, which records it on the task (ADR 0016). A conversation's agent reviews through a task, like any other change.
  • Owners hold pull_requests.comment like every permission; it is given to a role, as pull_requests.merge is.