Skip to content

feat(cli): grade index_repository outcomes with distinct exit codes - #2092

Open
Blysspeak wants to merge 2 commits into
DeusData:mainfrom
Blysspeak:feat/graph-gate-contract
Open

feat(cli): grade index_repository outcomes with distinct exit codes#2092
Blysspeak wants to merge 2 commits into
DeusData:mainfrom
Blysspeak:feat/graph-gate-contract

Conversation

@Blysspeak

Copy link
Copy Markdown

Breaking change: cli index_repository exit codes

A freshness gate cannot read a tool's account of itself — it reads the process
status. Today index_repository returns 0 for an index that covered half the
tree, and the same 1 whether the repository is missing or a pipeline fell over
inside one that exists. Callers cannot tell those apart.

If you consume this exit code, read the table. Grading happens in the CLI
only: MCP responses are untouched, so agents and scripts/soak-test.sh — which
judge by JSON-RPC and tool-level isError — see exactly what they saw before.

outcome before after
indexed, clean 0 0
indexed, files parsed partially above threshold 0 2
indexed, a file did not parse at all 0 2
indexed, node count far below expectation (status: degraded) 0 2
pipeline failed inside a reachable repository 1 1
repo_path absent, unreadable, or not a directory 1 3

New environment variables, both optional, negative disables the check:

  • CBM_GATE_MAX_UNUSABLE — default 0, absolute count of unparsable files
  • CBM_GATE_MAX_PARTIAL_PCT — default 10, share of partially parsed files

The two defaults differ deliberately. A file that did not parse at all is a
defect and gets no tolerance. Partial parsing marks constructs a grammar does
not cover — a property of language support, not of index quality: this
repository itself sits at 6.2% with a fully working 26k-node graph. A gate its
own project cannot pass gets switched off, which protects nothing.

Supporting fields, all additive

  • files_indexed and parse_partial_pct on the index response — the ratio had
    no denominator to be computed from, and a gate that can only pass or fail
    cannot say how partial a parse was. "Fresh, but 6.2% of files parsed
    partially" is a usable hint where a bare verdict is not.
  • reason on failures — target_unavailable or pipeline_failed. The hint
    text read identically for both.
  • graph_generation in index_status — advances on every graph mutation,
    unlike indexed_at, which only records when indexing ran. Not to be
    confused with the coverage block's meta.generation, a copy of indexed_at.
  • seed_symbols_list in detect_changes — the changed symbols themselves, not
    just their count, so a consumer can expire records per symbol instead of
    invalidating every record touching the file. Capped at 256; seed_symbols
    remains the exact total.
  • absolute_path in get_code_snippetfile_path there used to flip between
    absolute and relative depending on whether the source was read from disk, so
    the same symbol answered differently by source_mode. It is now always the
    graph's relative path, as in every other tool, with the absolute form beside
    it.

.cbmignore

Excludes internal/cbm/vendored/ and vendored/: 39.4M lines of generated
tree-sitter parsers drop to 1.0M (2120 tracked files to 1037), so the graph
stops filling with generated code that no one navigates.

Verification

Full suite green: 7954 passed, 0 failed, 7 skipped (141 suites).

All six outcomes measured against the built binary, exit status read without a
pipeline so $? is the command's own:

valid source          exit 0  files_indexed 4
broken syntax         exit 2  parse_partial_count 1 / 5 files
4KB of urandom as .py exit 2  parse_unusable_count 1
no source files       exit 0  files_indexed 3
unreadable subdir     exit 1  reason pipeline_failed
missing path          exit 3  reason target_unavailable
this repository       exit 0  parse_partial_pct 6.2, files_indexed 999

@Blysspeak
Blysspeak requested a review from DeusData as a code owner September 7, 2026 16:05
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

  • It will not be closed for inactivity. No stale bot touches pull requests here.
  • It may still sit a while before a human reads it. That is on us, not on you.
  • Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

  • Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
  • Get CI green, or say which failures you believe are pre-existing.
  • Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
  • Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

A freshness gate cannot read a tool's account of itself — it reads the
process status. index_repository returned 0 for a partial index and the
same 1 for both a missing repository and a pipeline that fell over inside
an existing one, so no caller could tell those apart.

The CLI now grades its own outcome:

  0  indexed, quality at or above the thresholds
  1  hard failure inside a reachable repository (unchanged meaning)
  2  indexed but below a quality threshold
  3  repo_path absent, unreadable, or not a directory

Grading happens in the CLI only. MCP responses are untouched, so agents
and the soak test, which judge by JSON-RPC and tool-level isError, see
exactly what they saw before.

Thresholds are separate on purpose and overridable:
  CBM_GATE_MAX_UNUSABLE     default 0   a file that did not parse at all
  CBM_GATE_MAX_PARTIAL_PCT  default 10  share of partially parsed files

Partial parsing marks constructs a grammar does not cover — a property of
language support, not of index quality; this repository itself sits at
6.2%. A gate its own project cannot pass gets switched off.

Supporting fields, all additive:
  files_indexed, parse_partial_pct  the denominator the ratio needs and
      the share stated outright, so a gate can quote it in a refusal
  reason  on failures: target_unavailable vs pipeline_failed
  graph_generation in index_status  advances on every graph mutation,
      unlike indexed_at which only says when indexing ran
  seed_symbols_list in detect_changes  the changed symbols themselves,
      so a consumer can expire records per symbol instead of per file
  absolute_path in get_code_snippet  file_path there used to flip between
      absolute and relative depending on source_mode; it is now always
      the graph's relative path, with the absolute form beside it

.cbmignore excludes the vendored tree-sitter parsers: 39.4M lines of
generated code drop to 1.0M, and the graph stops filling with them.

Signed-off-by: Vladislav Rahmanov <rahmanov.official@yandex.ru>
@Blysspeak
Blysspeak force-pushed the feat/graph-gate-contract branch from d1abca2 to 1f08f82 Compare September 7, 2026 16:34
The contract landed measured but unpinned: six outcomes were checked by
hand against the built binary and written into the pull request, and
nothing in the suite would notice if a code moved. A verification that
does not run again is a claim, not a test.

Twelve cases over cbm_cli_index_exit_status, which grades a result
envelope into a process exit code:

  clean run stays 0; an unusable file and a partial share above the
  ceiling both grade 2; "degraded" grades 2 on the status alone; an
  absent target grades 3 while a pipeline failure keeps 1 — the pair
  that shared code 1 before and gave the whole contract its reason.

Three of them exist to fail if the grader stops working rather than to
pass when it does:

  - the ceiling itself is exercised from both sides, 10 of 100 passing
    and 11 failing, because a test that only checks obvious excess stays
    green with the check switched off entirely;
  - the threshold overrides carry a positive control at the defaults, so
    "passed after raising the ceiling" cannot be confused with "no check
    ran";
  - eight unreadable threshold values are pinned to fall back on the
    documented default, since a typo that silently disables a gate is the
    failure the strict parse was written to prevent.

Silence is pinned too: a null, empty, unreadable, or content-less
envelope preserves the base status in both directions. Inventing a
quality verdict out of an unparsable payload would fail runs for the
crime of an unexpected response shape.

Envelopes are built with yyjson rather than hand-escaped, so the
payload's quoting is the transport's and not the test's own.

Signed-off-by: Vladislav Rahmanov <rahmanov.official@yandex.ru>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant