Skip to content

Update pytorchbot comment to have a PR status section - #8664

Merged
janeyx99 merged 7 commits into
mainfrom
pytorchbot-comment
Sep 23, 2026
Merged

janeyx99 merged 7 commits into
mainfrom
pytorchbot-comment

Conversation

@janeyx99

@janeyx99 janeyx99 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a contributor-facing PR Status note at the top of the existing Dr.CI comment on pytorch/pytorch.

  • derive the PR stage from the live triaged, in progress, and ready for review labels
  • if the PR is approved already by an authorized reviewer, the status is just "Approved" right off the bat
  • reuse the merge command approval rules so the status note and merge behavior agree
  • refresh the marked section on label, review-request, and review webhooks
  • rebuild it during the existing 15m Dr.CI sweep so missed webhooks converge
  • preserve the existing section when GitHub review state is unavailable due to an outage, etc.

This PR only reports the workflow state. It does NOT apply or transition lifecycle labels.

Contributor-facing text

See https://github.com/pytorch/test-infra/pull/8664/changes#diff-dd76dc7f6bcf9ce8104683b4598c5a5e03e019aec397acfa8f5199a328dd52e4R109

Precedence is approval, ready for review, in progress, then triaged, so transient overlap reports the furthest stage.

Design and commit structure

We implemented this in stages so each commit does one thing:

  1. extract the merge bot approval policy into a shared leaf module
  2. harden that policy before adding a second caller
  3. define the pure state-to-message renderer and marked-section helpers
  4. fetch the live GitHub inputs used by the renderer
  5. integrate the section with the periodic full Dr.CI render "sweep"
  6. update only the marked section immediately from relevant webhooks
  7. defer the rollout for "triaged" until we have pre-review set up

Rollout note

As of 2026-09-22, pytorch/pytorch has roughly 1,708 open PRs carrying triaged and none carrying either later-stage label. We therefore do not add the automation for 'triaged' yet but we do land the rest so that we can slowly test. We can phase in functionality for "triaged" later by reverting the seventh commit.

Validation

  • full Jest suite
  • Prettier check
  • Next.js lint (pre-existing warnings only)

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 27, 2026
@vercel

vercel Bot commented Aug 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
torchci Ignored Ignored Sep 23, 2026 5:36pm UTC

Request Review

@janeyx99
janeyx99 force-pushed the pytorchbot-comment branch from e7ce42a to 3285df6 Compare August 27, 2026 22:46

@izaitsevfb izaitsevfb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the approach is ok with me, but please take a look at the nits below

Comment thread torchci/lib/prStatus.ts
Comment thread torchci/lib/prStatus.ts
Comment thread torchci/lib/prStatus.ts Outdated
Comment thread torchci/lib/reviewApproval.ts
@janeyx99
janeyx99 force-pushed the pytorchbot-comment branch 13 times, most recently from cce3bf9 to 61f01a8 Compare September 22, 2026 21:31

@izaitsevfb izaitsevfb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please check the nits below

const ASSIGNABLE_REVIEWER_ASSOCIATIONS = ["COLLABORATOR", "MEMBER", "OWNER"];

/**
* The PR's approval verdict from its reviews: "" (nobody has decided),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

export function getReviewerLogins(
reviews: PullRequestReview[],
excludeLogins: string[] = []
): string[] {
const excluded = new Set(excludeLogins);
return Array.from(
new Set(
reviews
.filter((review) =>
ASSIGNABLE_REVIEWER_ASSOCIATIONS.includes(review.author_association)
)
.map((review) => review.user?.login)
.filter(
(login): login is string => Boolean(login) && !excluded.has(login!)
)
)
);

const assignedReviewers =
needsReviewers && author
? buildAssignedReviewers(
owner,
pull?.data.requested_reviewers ?? [],
pull?.data.requested_teams ?? [],
getReviewerLogins(reviews as any, [author])
)
: [];

not blocking, but something to be aware of.


🟡 Any org member who leaves one review comment is listed as an assigned reviewer who must agree, and cannot be taken off the list. (ai-generated section)

To recover reviewers GitHub drops from requested_reviewers once they review, fetchPrStatusState adds everyone returned by getReviewerLogins, which keeps any COLLABORATOR, MEMBER or OWNER review of any state. On pytorch/pytorch MEMBER covers every org member, so a drive-by inline comment makes that person one of the "assigned reviewers" the pre-review note says must agree. Review history is never removed, so once someone has reviewed, removing their review request does not take them off the list, although the handler subscribes to review_request_removed to keep the list current. Recovering reviewers from the timeline's review-request events would track actual assignment; failing that, the note could say "requested reviewers" and list only requested_reviewers.

Reviewed by codex gpt-6-astra + claude-fable-5 at high effort, against 2df0b6f.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is okay--if an org member takes the time to review a pre-review PR, that person likely is appropriate to react or prevent the PR from going forward

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for this is because GitHub has requested_reviewers and requested_teams which exclude people who've already reviewed. So to approximate the reviewer box we union both (and subtract the PR author and anyone who is not COLLABORATOR, MEMBER, or OWNER)

return;
}
} else if (
payload.action === "review_requested" ||

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flagging this race. not sure how frequent that could happen in real world, but a re-check with re-try seems cheap enough?


🟡 A section built from labels read earlier can overwrite a newer one, and nothing may correct it for a long time. (ai-generated section)

The webhook path renders from the label list in the event payload (prStatusBot.ts line 41) and writes the whole comment body it read before its GitHub calls; the sweep reads labels live, then writes only after classifying that PR's CI. Nothing re-checks before writing. So if a bot swaps triaged for in progress and the unlabeled delivery finishes after the labeled one, or a label changes while the sweep is mid-PR, the older stage is what lands; the webhook write can likewise put back CI results a sweep just replaced. The description relies on the sweep for convergence, but the sweep only revisits PRs with workflow activity in the last 30 minutes or with pending jobs, so a PR waiting quietly for review keeps the wrong stage until its next label or review event. Re-reading the labels, and the comment, immediately before updateComment would narrow this a lot.

Reviewed by codex gpt-6-astra + claude-fable-5 at high effort, against 2df0b6f.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a few things to narrow the race:

  • during the webhook triggered update, make sure to get the latest comment at the latest time possible--right before write (which is the same thing as recheck essentially)
  • fetch live labels in the sweep after finishing CI classification
    This doesn't eliminate the race but make it less likely.

Comment thread torchci/lib/prStatus.ts
`before the PR will be marked "in progress".`
);
case "none":
return "";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

valid nit.


⚪ A triaged PR with no reviewer renders "All assigned reviewers () must agree". (ai-generated section)

formatReviewers returns an empty string for an empty list and the pre-review message interpolates it unchanged, which a test pins. At rollout this note goes onto the open pytorch/pytorch PRs that already carry triaged, and any of them without a requested reviewer shows the empty parentheses and an instruction nobody can act on. A separate sentence for the empty case, such as saying reviewers have not been assigned yet, would fix it.

Reviewed by codex gpt-6-astra + claude-fable-5 at high effort, against 2df0b6f.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did notice this, and this shouldn't really happen, and if it does happen it should be fixed elsewhere, so i think we're fine for this pr

@janeyx99
janeyx99 force-pushed the pytorchbot-comment branch 3 times, most recently from 9fe850b to 08cfdc4 Compare September 23, 2026 14:32
Move the existing approval rules into a shared module and add unit coverage. Keep PytorchBotHandler's behavior unchanged.
Sort a copy of the review list instead of mutating caller input, and ignore reviews whose GitHub user is null.
@janeyx99
janeyx99 force-pushed the pytorchbot-comment branch 3 times, most recently from 6d48724 to c21e17f Compare September 23, 2026 15:22
@janeyx99
janeyx99 marked this pull request as ready for review September 23, 2026 15:34
Comment thread torchci/lib/prStatus.ts
Comment thread torchci/lib/prStatus.ts Outdated

@albanD albanD left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

@albanD albanD left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The message look good.
The logic sound fair, but I'm not really a TS expert.

Map workflow labels and approval state to contributor-facing guidance. Add helpers to extract and replace the section without changing the rest of the Dr.CI comment.
Use reviews to determine approval and recover reviewers who have already responded. For triaged PRs, combine them with current reviewer assignments. Return null if reviews cannot be fetched so callers preserve the existing section.
For pytorch/pytorch, render the section from live GitHub labels so lagging mirrored labels cannot overwrite newer state. Preserve the existing section when GitHub inputs cannot be fetched.
Update existing Dr.CI comments on label, review-request, and review events. Preserve the section during push updates and read the comment last to narrow races with the sweep.
@janeyx99
janeyx99 force-pushed the pytorchbot-comment branch 2 times, most recently from b1ed2ee to a0f5311 Compare September 23, 2026 17:20
@janeyx99
janeyx99 merged commit b5e16eb into main Sep 23, 2026
8 checks passed
@janeyx99
janeyx99 deleted the pytorchbot-comment branch September 23, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants