feat(cli): grade index_repository outcomes with distinct exit codes - #2092
Open
Blysspeak wants to merge 2 commits into
Open
feat(cli): grade index_repository outcomes with distinct exit codes#2092Blysspeak wants to merge 2 commits into
Blysspeak wants to merge 2 commits into
Conversation
|
Thanks for opening this — it has been seen, and it is queued. This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence. Current review status: working through a backlog. What that means for this PR, concretely:
Things that will genuinely speed it up whenever review does happen:
If this fixes a bug, a reproduction we can run is worth more than a description of the symptom. Thanks for contributing, and sorry in advance for the wait. |
A freshness gate cannot read a tool's account of itself — it reads the
process status. index_repository returned 0 for a partial index and the
same 1 for both a missing repository and a pipeline that fell over inside
an existing one, so no caller could tell those apart.
The CLI now grades its own outcome:
0 indexed, quality at or above the thresholds
1 hard failure inside a reachable repository (unchanged meaning)
2 indexed but below a quality threshold
3 repo_path absent, unreadable, or not a directory
Grading happens in the CLI only. MCP responses are untouched, so agents
and the soak test, which judge by JSON-RPC and tool-level isError, see
exactly what they saw before.
Thresholds are separate on purpose and overridable:
CBM_GATE_MAX_UNUSABLE default 0 a file that did not parse at all
CBM_GATE_MAX_PARTIAL_PCT default 10 share of partially parsed files
Partial parsing marks constructs a grammar does not cover — a property of
language support, not of index quality; this repository itself sits at
6.2%. A gate its own project cannot pass gets switched off.
Supporting fields, all additive:
files_indexed, parse_partial_pct the denominator the ratio needs and
the share stated outright, so a gate can quote it in a refusal
reason on failures: target_unavailable vs pipeline_failed
graph_generation in index_status advances on every graph mutation,
unlike indexed_at which only says when indexing ran
seed_symbols_list in detect_changes the changed symbols themselves,
so a consumer can expire records per symbol instead of per file
absolute_path in get_code_snippet file_path there used to flip between
absolute and relative depending on source_mode; it is now always
the graph's relative path, with the absolute form beside it
.cbmignore excludes the vendored tree-sitter parsers: 39.4M lines of
generated code drop to 1.0M, and the graph stops filling with them.
Signed-off-by: Vladislav Rahmanov <rahmanov.official@yandex.ru>
Blysspeak
force-pushed
the
feat/graph-gate-contract
branch
from
September 7, 2026 16:34
d1abca2 to
1f08f82
Compare
The contract landed measured but unpinned: six outcomes were checked by
hand against the built binary and written into the pull request, and
nothing in the suite would notice if a code moved. A verification that
does not run again is a claim, not a test.
Twelve cases over cbm_cli_index_exit_status, which grades a result
envelope into a process exit code:
clean run stays 0; an unusable file and a partial share above the
ceiling both grade 2; "degraded" grades 2 on the status alone; an
absent target grades 3 while a pipeline failure keeps 1 — the pair
that shared code 1 before and gave the whole contract its reason.
Three of them exist to fail if the grader stops working rather than to
pass when it does:
- the ceiling itself is exercised from both sides, 10 of 100 passing
and 11 failing, because a test that only checks obvious excess stays
green with the check switched off entirely;
- the threshold overrides carry a positive control at the defaults, so
"passed after raising the ceiling" cannot be confused with "no check
ran";
- eight unreadable threshold values are pinned to fall back on the
documented default, since a typo that silently disables a gate is the
failure the strict parse was written to prevent.
Silence is pinned too: a null, empty, unreadable, or content-less
envelope preserves the base status in both directions. Inventing a
quality verdict out of an unparsable payload would fail runs for the
crime of an unexpected response shape.
Envelopes are built with yyjson rather than hand-escaped, so the
payload's quoting is the transport's and not the test's own.
Signed-off-by: Vladislav Rahmanov <rahmanov.official@yandex.ru>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Breaking change:
cli index_repositoryexit codesA freshness gate cannot read a tool's account of itself — it reads the process
status. Today
index_repositoryreturns0for an index that covered half thetree, and the same
1whether the repository is missing or a pipeline fell overinside one that exists. Callers cannot tell those apart.
If you consume this exit code, read the table. Grading happens in the CLI
only: MCP responses are untouched, so agents and
scripts/soak-test.sh— whichjudge by JSON-RPC and tool-level
isError— see exactly what they saw before.status: degraded)repo_pathabsent, unreadable, or not a directoryNew environment variables, both optional, negative disables the check:
CBM_GATE_MAX_UNUSABLE— default0, absolute count of unparsable filesCBM_GATE_MAX_PARTIAL_PCT— default10, share of partially parsed filesThe two defaults differ deliberately. A file that did not parse at all is a
defect and gets no tolerance. Partial parsing marks constructs a grammar does
not cover — a property of language support, not of index quality: this
repository itself sits at 6.2% with a fully working 26k-node graph. A gate its
own project cannot pass gets switched off, which protects nothing.
Supporting fields, all additive
files_indexedandparse_partial_pcton the index response — the ratio hadno denominator to be computed from, and a gate that can only pass or fail
cannot say how partial a parse was. "Fresh, but 6.2% of files parsed
partially" is a usable hint where a bare verdict is not.
reasonon failures —target_unavailableorpipeline_failed. The hinttext read identically for both.
graph_generationinindex_status— advances on every graph mutation,unlike
indexed_at, which only records when indexing ran. Not to beconfused with the coverage block's
meta.generation, a copy ofindexed_at.seed_symbols_listindetect_changes— the changed symbols themselves, notjust their count, so a consumer can expire records per symbol instead of
invalidating every record touching the file. Capped at 256;
seed_symbolsremains the exact total.
absolute_pathinget_code_snippet—file_paththere used to flip betweenabsolute and relative depending on whether the source was read from disk, so
the same symbol answered differently by
source_mode. It is now always thegraph's relative path, as in every other tool, with the absolute form beside
it.
.cbmignoreExcludes
internal/cbm/vendored/andvendored/: 39.4M lines of generatedtree-sitter parsers drop to 1.0M (2120 tracked files to 1037), so the graph
stops filling with generated code that no one navigates.
Verification
Full suite green: 7954 passed, 0 failed, 7 skipped (141 suites).
All six outcomes measured against the built binary, exit status read without a
pipeline so
$?is the command's own: