fix(risk-scorer): deferral is not a control - #406
Open
tompahoward wants to merge 1 commit into
Open
Conversation
The pipeline agent scores the action in front of it, so deferring work moves the hazard outside the scored window and any hold-this-back recommendation scores lower on the current hop regardless of whether total risk fell. There is no way to price un-integrated work. Adds the rule beside 'Monitoring is not a control', which is the same shape: both are things that look like controls but do not exercise the failure before it ships. Monitoring shortens time-to-notice; deferral relocates the hazard. Scoped so it does not over-apply. Separating an unrelated change out of a risky commit genuinely narrows blast radius, so the test is whether the end state still requires the held-back action. If it does, that is deferral; if not, that is separation. Refs #405
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #405.
Adds a
Deferral is not a controlrule to the pipeline agent, placed directly besideMonitoring is not a controlbecause they are the same shape: both are things that look like controls but do not exercise the failure before it ships. Monitoring shortens time-to-notice. Deferral relocates the hazard.The rule
Scoping
Deliberately bounded so it does not forbid legitimate splits. Separating an unrelated change out of a risky commit genuinely narrows blast radius without deferring anything. The test written into the rule is whether the end state still requires the held-back action: if it does, that is deferral; if not, that is separation.
The rule also names the direction of the bias it corrects, because that is what makes it actionable rather than abstract: uncorrected, per-action scoring favours smaller batches per action, which is the opposite of what teams practising trunk-based development want.
What this does not include
No paired promptfoo eval. The existing harness at
packages/risk-scorer/agents/eval/is scoped to the external-comms agent's credibility axis, and there is no pipeline-agent eval to extend, so the R009 prose-surface floor is not discharged by this PR. Building one is a larger piece of work and I did not want to bundle it. Say if you would rather this waited for that.Prose is em-dash free per the house convention.