fix(knowledge): close the learning feedback loop#1722
Conversation
Three gaps that left the knowledge feedback loop open are now closed: - G1: phase-complete now records the REAL phase outcome (success/failure) via updateRetrievalOutcome, not a hardcoded `true`. Previously the call ran before `success` was determined, so failed phases could never emit 'failure' outcome events and the negative half of the outcome signal was dead. - G2: confidence feedback no longer dead-ends. A just-bumped entry clamped to the floor (0.1) with a net-negative outcome signal is now demoted (confidence_floor_demoted flag, strips retrieval statusBoost) or quarantined per config. Conservative defaults: requires >=3 outcome events and a net-negative signal. - G3: contradiction signals unified. Curator flag_contradiction now emits a `contradicted` event post-transaction (in addition to the existing tag), and maybeQuarantineOnContradiction auto-quarantines entries whose in-window contradicted count crosses the threshold (default 3 in 30d). The genuinely-closed sub-loop was previously only the violation-count -> escalator -> enforce-block path. Confidence/contradiction/phase-failure halves were wired to record signal but nothing consumed it. Reviewed: independent GLM-5.2 reviewer (NEEDS_REVISION -> all 5 findings addressed) + independent GLM-5.2 critic (APPROVE). Validation: typecheck clean, lint clean, 276 tests pass / 0 fail across 11 affected files (each run in its own process per repo convention; the combined-run failures are pre-existing on main, tracked in #1712).
Drift check report✅ No drift detected across skills, tools, commands, and agents. |
🤖 Multi-Stage PR ReviewPipeline: Friday-35B (Qwen3.6-35B-A3B) (context pack) → Mellum2-12B (explore → candidates) → GLM-5-turbo (challenge + blind-spot) 🔍 PR Intent
📦 Implementation SummaryThree independent feedback-loop gaps are closed. G1: The ✅ /
|
| Obligation | Status | Evidence (file:line) |
|---|---|---|
| O-001 | SUPPORTED |
src/tools/phase-complete.ts:1383 — passes success variable; old literal-true call removed |
| O-002 | SUPPORTED |
src/hooks/knowledge-store.ts:906–1089 — applyConfidenceFloorAction with signal/evidence gates, demote/quarantine/none paths |
| O-003 | SUPPORTED |
src/hooks/curator.ts:1718–1745 — emits contradicted events with agent:'curator' post-transaction |
| O-004 | SUPPORTED |
src/hooks/knowledge-escalator.ts:280–352 — maybeQuarantineOnContradiction with raw-event counting, idempotency guard |
| O-005 | SUPPORTED |
src/config/schema.ts:1102–1131 — six new fields with correct Zod types and defaults matching docs |
| O-006 | SUPPORTED |
src/hooks/search-knowledge.ts:487–491 — !entry.confidence_floor_demoted gate on statusBoost |
| O-007 | SUPPORTED |
src/hooks/knowledge-store.ts:943–944,1014 — recovered array clears stale flag via entry.confidence_floor_demoted = false |
| O-008 | SUPPORTED |
undefined is falsy → not-demoted (legacy); floorAction:'none' early-returns; tag_only skips quarantine call |
🚨 Confirmed Findings
None found.
🔬 Unverified but Plausible Risks
None found.
🧪 Test / Coverage Gaps
None found. All three gaps (G1 source guard + behavioral companion, G2 demote/quarantine/recovery/idempotency/none, G3 event emission/tag-backcompat/threshold/below-threshold/idempotency/window/end-to-end/tag_only) are covered. The retrieval-ranking test confirms the demoted entry actually sinks.
📋 Shipped-vs-Claimed Gaps
None found. 290 tests pass (all in isolated processes per repo convention). New config defaults match the schema, the release fragment, and the PR description exactly.
🔁 Validation provenance
- Confirmed findings kept: 0 (the reviewer-confirmed list was empty).
- Confirmed findings dropped: N/A.
- Blind-spot findings added: 0. I traced every
+line across all seven changed source files, verified cross-file type contracts (ConfidenceFloorOptionswiring), checked lock ordering (sequential, no deadlock), validated thetouched-array propagation fromapplyConfidenceDeltas→applyConfidenceFloorAction, confirmed hive-tier quarantine is intentionally omitted (documented caveat), and verified the G1 guard condition is a semantic tightening consistent with the function's purpose. No defects found.
📝 Merge Recommendation
APPROVE
Clean, well-structured closure of three documented feedback-loop gaps with conservative defaults, proper fail-open semantics, correct cross-file wiring, and thorough test coverage.
| Check | Result |
|---|---|
| No CRITICAL findings | ✅ |
| No unresolved STEALTH_CHANGE | ✅ |
| No UNSUPPORTED obligations | ✅ |
| Test coverage adequate | ✅ |
| No hardcoded secrets | ✅ |
| All async errors handled | ✅ |
| Input validation present | ✅ |
| No broken agent role boundaries | ✅ |
| Prompt format contracts intact | ✅ |
| Lockfile consistent | ✅ |
🔒 Reviewed by a multi-stage local-first funnel (architect context pack → explorer candidates → critic challenge/author) for high recall with low false-positive noise. Findings are advisory — verify before acting.
4238102
Closes #1715
Summary
The knowledge feedback loop was an open circuit — outcome signals were recorded but nothing acted on them. Three gaps fixed (G1/G2/G3 from the gap catalog):
phase-completepreviously calledupdateRetrievalOutcome(dir, name, true)with a hardcodedtrueat a point in the flow beforesuccesswas determined. The call now runs after the real outcome is finalized and passes that value, so a failed phase (policy=enforce+ missing required agents) emits'failure'outcome events andfailed_after_shown_countcan finally increment.applySkillUsageFeedback,applyKnowledgeVerdictFeedback) bumpedentry.confidencebut nothing acted on low confidence — the bumps dead-ended at the floor. NowbumpKnowledgeConfidenceBatchruns a post-bump sweep that demotes (confidence_floor_demotedflag → strips retrievalstatusBoost) or quarantines per config entries that are floor-clamped AND have a net-negative outcome signal over ≥3 outcome events. Reversible by default.contradicted_count(incremented only viaknowledge_receipt) and the curator'sflag_contradiction(tag-only) were disconnected. The curator now emitscontradictedevents post-transaction (withagent: 'curator'for audit), and a newmaybeQuarantineOnContradictionauto-quarantines entries whose in-window contradicted count crosses the threshold (default 3 in 30d). Counts come from the RAW event log, not the mtime-keyed rollup cache, to avoid stale-count risk.The genuinely-closed sub-loop was previously only the violation-count → escalator → enforce-block path. This change closes the confidence/contradiction/phase-failure halves.
Invariant audit
contradictedevents,confidence_floor_demotedflag) write only to existing.swarm/knowledge + events files; no new locations.bumpKnowledgeConfidenceBatchand the new floor-action are best-effort, never throw).statusBoostgate is read-only on retrieval scoring.docs/releases/pending/1715-close-knowledge-feedback-loop.md; no version files edited.Test plan
Validation commands run + results:
Pre-existing-failure proof (combined run is NOT a regression): the combined single-process run produces 24 failures — but I verified this is identical on clean
origin/main(worktree at HEAD c5eb59c, before any of these edits): same 24 failures, same pass count. This is the documented cross-file mock/state contamination tracked in #1712, not caused by this change.Independent review: GLM-5.2 reviewer (fresh context) returned NEEDS_REVISION with 5 findings (2 HIGH broken config fixtures + missing tests; 1 MED hive quarantine no-op; 2 LOW). All addressed. GLM-5.2 critic (separate fresh context) returned APPROVE with 2 optional LOW improvements, both applied (floor-action
console.warn, behavioral retrieval-ranking test).New test coverage:
none=legacy, idempotency, retrieval down-ranking.contradictedevent, tag preserved (back-compat), threshold quarantine, below-threshold skip, idempotency on already-quarantined, window respected, curator→quarantine end-to-end,tag_only=legacy.true, passessuccessvar, old false-premise comment gone). Companion behavioral test (false→'failure'event) already exists atknowledge-reader.test.tsTest 12.How to use
New config knobs (all optional, conservative defaults):
knowledge.confidence_floor_action:'none' | 'demote' | 'quarantine'(default'demote').knowledge.confidence_floor_min_outcomes: default3.knowledge.confidence_floor_signal_threshold: default0(net-negative).knowledge.contradiction_threshold_action:'tag_only' | 'quarantine'(default'quarantine').knowledge.contradiction_quarantine_threshold: default3.knowledge.contradiction_quarantine_window_days: default30.For the legacy dead-end behavior:
confidence_floor_action: 'none'+contradiction_threshold_action: 'tag_only'.Migration
No migration required. New config fields default to conservative behavior. Existing entries without
confidence_floor_demotedread asundefined→ not-demoted (legacy). Hive floor-entries get the flag (honored by retrieval) but not quarantine (the quarantine mechanism reads only the swarm file — documented in the release fragment).