Skip to content

fix(index): correct TypeScript callers and report partial enrichment - #84

Open
redxzeta wants to merge 3 commits into
intuit:mainfrom
redxzeta:fix/typescript-callers-partial-index-reporting
Open

redxzeta wants to merge 3 commits into
intuit:mainfrom
redxzeta:fix/typescript-callers-partial-index-reporting

Conversation

@redxzeta

@redxzeta redxzeta commented Oct 4, 2026 •

Copy link
Copy Markdown

Summary

TypeScript arrow helpers could be stored as variables, and calls inside nested generator callbacks could lose their named owner. SCIP enrichment could add different incorrect owners, while failed enrichment was swallowed and later queries presented an incomplete graph without a warning.

This change classifies variable-bound arrows, function expressions, and generator expressions as callables in TypeScript and TSX. Inferred calls use the innermost extracted Function, Method, or Test and its actual symbol ID; anonymous callbacks fall back to an enclosing callable. Callable definitions take precedence over overlapping variable matches, with Test precedence and merged metadata retained. Other languages keep their ownership behavior. Extractor schema version 2 causes unchanged files to be repaired during incremental indexing.

Local indexing now atomically saves call-resolution and per-SCIP outcomes in .infigraph/index-status.json. Applicable foreground enrichment failures exit 1 while retaining the syntax graph. Background stages start pending and persist child outcomes. Run identity and indexed-file fingerprints guard imports and updates from obsolete workers. Unchanged source retains failures and retries failed/interrupted enrichment, while successful stages are reused. Optional project prerequisites are checked before tool provisioning; skipped optional stages do not fail indexing.

SCIP artifacts and import scratch files belong to the current attempt. Existing user artifacts are preserved, and missing, empty, symlink, or invalid artifacts are rejected. CLI callers/callees/impact and the corresponding MCP tools share coverage notices for populated and empty results. Queries read status without repairing it; legacy, invalid, mismatched, and remote status is explicitly unknown. MCP arguments and text response shape remain unchanged. No graph migration or dependency change is required.

Test plan

  • Unmodified cargo test --all passes: local validation uses --no-default-features for this CPU; the full attempt exposed two reproduced baseline failures described below. Default-feature behavior awaits upstream CI.
  • Unmodified strict Clippy passes: blocked locally by reproduced baseline lints described below.
  • Formatting and git diff --check pass.
  • Focused extraction, imported calls, nested callbacks, test filtering, ambiguity, transitive impact, incremental repair, SCIP ownership, report persistence, and CLI/MCP notice tests pass.
  • Controlled subprocess coverage exercises unavailable tools, process/signal failures, missing/invalid artifacts, recovery, no-op retries/reuse, retained artifacts, obsolete workers, and independent projects.
  • Fresh Forkara comparison with real SCIP and a separate deterministic failure/interruption/recovery trial.

The final-head workspace run passes 1,286 tests (14 ignored), excluding only the two independently reproduced baseline failures below. Focused CLI (35), core caller (5), SCIP (12), core report (2), and MCP report (1) tests also pass.

Commands: cargo test --locked --all --no-default-features -j1 -- --test-threads=1 --skip combined_docs_merge_search_link_and_rebuild --skip test_ui_server_binds_loopback_not_wildcard_by_default; cargo fmt --all -- --check; cargo clippy --locked --all-targets --all-features -j1 -- -D warnings -A clippy::chunks_exact_to_as_chunks -A clippy::needless_borrow -A clippy::useless_vec.

Notes

Forkara trials use revision f7ad9270fd6eb2e16a42216f1bc79b8ab8d99ade, baseline Infigraph cf82f5d7440d9be158e219df1ef256d6359ed1f6, and proposed head 4a775e7648414cbe3cf9684066fcc5856ea87856. Both binaries use the dev profile and --no-default-features, fresh --no-embed indexes, isolated homes/registries, and trial-local @sourcegraph/scip-typescript 0.4.0 with Node 24.18.1. Real SCIP succeeds in both; heap exhaustion was not reproduced.

The final proposed graph classifies all three checked helpers as Functions. refreshStatusUpstreamIfStale has status and readStatusDetails callers; fetchUpstreamRefForStatus and makeStatusUpstreamRefreshCacheTimeToLive have makeGitCore as caller. These agree with source under the documented anonymous-callback policy. Checked fetch/policy impact paths include makeGitCore and the extracted test helper makeIsolatedGitCore, removing baseline variable owners. The real trial exposed the additional SCIP ownership defect and duplicate AST/SCIP caller output, which have regression coverage here.

Baseline/final index statistics: 6,967 files and 217,239 symbols each; CALLS edges change from 124,292 to 93,066. Measured wall times are 220.542s and 368.996s respectively; the final trial overlapped workspace tests, so these are indexing costs, not a controlled performance comparison.

A fresh controlled Forkara index runs a fake SCIP process that exits 7: indexing exits 1, the syntax graph remains queryable, and a no-op retries the failure. Restarted CLI/MCP queries warn for populated and empty results without changing graph/status bytes. An actual hidden child interrupted with SIGTERM leaves a visible running/interrupted notice; controlled valid-artifact recovery exits 0 and marks the stage succeeded. An obsolete child exits 1 without launching the indexer or changing the newer receipt. This controlled reporting recovery is separate from the successful real SCIP trial. Retained user artifacts are preserved. After the real indexer-generated tsconfig.json is indexed, a stabilized no-op reuses successful enrichment (6.359s) without launching SCIP. Repeated real imports emitted existing duplicate-symbol PK warnings; exhaustive import/freshness behavior remains outside this change.

Baseline validation limits, reproduced independently on clean cf82f5d:

  • combined_docs_merge_search_link_and_rebuild aborts after a roughly 92 GB allocation request.
  • web::tests::test_ui_server_binds_loopback_not_wildcard_by_default fails its loopback/wildcard assertion on this environment; its poisoned mutex causes subsequent test failures when left enabled.
  • Rust 1.98 strict Clippy flags unchanged embedding/doc tests with chunks_exact_to_as_chunks, needless_borrow, and useless_vec. All-target/all-feature Clippy passes with only those three baseline lint categories allowed. No source suppressions or unrelated fixes are included.

SCIP line-only ownership leaves boundary-line or indistinguishable owners unresolved. Arbitrary factories, anonymous tests without an extracted enclosing callable, module-level sources, and ambiguous targets remain limitations. Stage success records completion, not exhaustive relationship coverage or working-tree freshness. Semantic-search quality is untested with --no-embed. Existing checkout/index/tool snapshots are preserved; trial artifacts are excluded from this PR.

Exact-head upstream CI currently reports action_required with zero jobs started; default-feature runtime behavior remains unverified. Review is required and no review has been submitted.

Related: #48. This does not address its broader freshness requirements.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant