Skip to content

feat(foundation): add fuzzymatch for did-you-mean connector suggestions - #2769

Merged
devarismeroxa merged 1 commit into
mainfrom
feat/ws1-fuzzymatch
Aug 5, 2026
Merged

feat(foundation): add fuzzymatch for did-you-mean connector suggestions#2769
devarismeroxa merged 1 commit into
mainfrom
feat/ws1-fuzzymatch

Conversation

@devarismeroxa

Copy link
Copy Markdown
Contributor

Why

Two commands have been blocked on this utility not existing.

  • repair left the "connector plugin not found" class unrepairable — "the correct plugin name is not mechanically knowable... Deferred until a did-you-mean index exists" (20260712-repair-command.md, §6).
  • conduit generate then made it a hard prerequisite rather than polish: its acceptance bar requires an unknown connector to produce a closest match and an install suggestion, never a fabricated plugin name (20260722-conduit-generate.md, §7).

The generate use is the demanding one. When a model invents a connector, feeding `postgre` does not exist; did you mean `postgres`? back into the retry prompt converts a hallucination into a self-correction inside the retry budget, instead of a terminal failure.

Two named consumers — shared infrastructure, not speculative generality.

Invariants

Both enforced by test and by the fuzzer:

  1. Output is deterministic — ordered by edit distance, then lexicographically. An error message whose wording depends on map iteration order can't be asserted on, alerted on, or put in a golden file.
  2. A suggestion is never fabricated — every returned string is an element of candidates, and nothing is returned unless it clears the similarity floor. A confident wrong name is worse than silence when a model is going to act on it.

Design choices (all from §7, not invented here)

Choice Reason
Plain Levenshtein, not Damerau Real connector typos are substitutions/omissions/insertions (postgre, kafak), not adjacent transpositions. Simpler to audit, no coverage lost — TestSuggest_TranspositionIsCovered pins that a transposition still lands inside the floor at cost 2
Case-insensitive Names arrive from natural-language prompts with no casing convention
Floor = max(2 edits, 30% of length) Neither bound works alone: a flat 2 edits on a 26-char name is stricter than anyone typing by hand manages; 30% of a 3-char name is zero
Two-row matrix Runs once per candidate over the whole catalog inside a retry loop; the discarded rows can't be read again

Adversarial self-review

  1. A test that proved nothing. My first long-name case for the relative bound used a candidate at edit distance 2 — exactly the absolute bound. It passed under an absolute-only rule too, so the mutation survived (see below). My code comment claiming "4 edits, well past the absolute bound" was simply wrong about its own fixture. Replaced with a genuine distance-4 case; the test now asserts the distance explicitly so a future edit can't silently reintroduce a non-discriminating one.
  2. nil vs empty slice. The doc comment promised nil on no match; the code returned []string{} via make([]string, 0). Caught by the first test run. Callers branch on == nil and len() == 0 interchangeably only if the two never diverge.
  3. Duplicate candidates. Callers assemble catalogs by concatenating sources (built-in + installed + registry), so duplicates are the normal case. Without a dedupe, three copies of postgres consume all three suggestion slots and crowd out the real alternative.
  4. Multi-byte input. Targets ASCII plugin names, but input arrives from NL prompts and model output. Rune-based throughout; TestLevenshtein_MultiByteRunes pins that distance counts characters, not bytes.

Tests — every mutation verified

Mutation Tests killed
Absolute bound only (drop relative) FloorIsLooserOfTwoBounds
Relative bound only (drop absolute) RealTypos, TranspositionIsCovered, FloorIsLooserOfTwoBounds, DeterministicAcrossCandidateOrder, RespectsMaxSuggestions
Drop lexicographic tie-break DeterministicAcrossCandidateOrder
Drop duplicate skip DuplicatesDoNotConsumeSlots
Widen floor 5x RealTypos, NeverFabricates, FloorIsLooserOfTwoBounds — suggests kafka for mysql
Drop case folding RealTypos

FuzzSuggest asserts both invariants plus the cap on arbitrary input: 9.7M executions, no failures.

Tests use the real built-in connector list (file, generator, kafka, log, postgres, s3) rather than invented fixtures, so the floor is tuned against distances that occur in practice. NeverFabricates checks the connectors people actually ask for and don't have — mysql, mongo, snowflake, redis, bigquery — and requires zero suggestions for each.

What was run

go build ./..., go vet, golangci-lint run (0 issues), go test -race, and the 45s fuzz run above.

Risk tier

3. New leaf package, no existing call sites, no data path, no serialized format. No new dependencies.

Roadmap

v0.20 WS1 (conduit generate) — prerequisite per design doc §7. Once merged, repair's v2 scope should revisit the connector-not-found row it deferred for exactly this reason.

🤖 Generated with Claude Code

https://claude.ai/code/session_015GQFzakPShAYj8CcwajYDD

Two commands have been blocked on this utility not existing.

The `repair` design doc (20260712-repair-command.md, §6) left "connector plugin
not found" unrepairable: "the correct plugin name is not mechanically
knowable... Deferred until a did-you-mean index exists". `conduit generate`
(20260722-conduit-generate.md, §7) then made it a hard prerequisite rather than
polish — its acceptance bar requires an unknown connector to produce a closest
match and an install suggestion, never a fabricated plugin name.

The generate use is the demanding one. When a model invents a connector,
feeding "`postgre` does not exist; did you mean `postgres`?" back into the retry
prompt converts a hallucination into a self-correction inside the retry budget
instead of a terminal failure. Two named consumers, so this is shared
infrastructure, not speculative generality.

Two invariants, both enforced by test and by the fuzzer:

1. Output is deterministic — ordered by edit distance, then lexicographically.
   An error message whose wording depends on map iteration order cannot be
   asserted on, alerted on, or put in a golden file.
2. A suggestion is never fabricated. Every returned string is an element of
   candidates, and nothing is returned unless it clears the similarity floor.
   A confident wrong name is worse than silence when a model will act on it.

Per the design doc: plain Levenshtein (real connector typos are substitutions,
omissions, and insertions, not adjacent transpositions), case-insensitive
(names arrive from natural-language prompts), and a similarity floor that is the
looser of an absolute 2-edit bound and a relative 30%-of-length bound. Neither
bound alone works: a flat 2 edits on a 26-character name is stricter than anyone
typing by hand manages, and 30% of a 3-character name is zero.

Levenshtein keeps two rows rather than the full matrix — it runs once per
candidate over the whole catalog inside a retry loop, and the discarded rows can
no longer be read.

Tests, each mutation-verified:

- absolute bound only -> FloorIsLooserOfTwoBounds fails
- relative bound only -> 5 tests fail
- lexicographic tie-break dropped -> DeterministicAcrossCandidateOrder fails
- duplicate-candidate skip dropped -> DuplicatesDoNotConsumeSlots fails
- floor widened 5x -> NeverFabricates fails (suggests `kafka` for `mysql`)
- case folding dropped -> RealTypos fails

The first pass of the long-name case did NOT kill the absolute-only mutation:
the candidate I picked sat exactly at distance 2, the absolute bound, so it
proved nothing. Replaced with a distance-4 case and the mutation now kills. The
test asserts the distance explicitly so the next edit can't silently reintroduce
a non-discriminating case.

FuzzSuggest asserts both invariants plus the cap on arbitrary input: 9.7M
executions, no failures.

Risk tier: 3. New leaf package, no existing call sites, no data path.

Roadmap: v0.20 WS1 (`conduit generate`), prerequisite per design doc §7.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015GQFzakPShAYj8CcwajYDD
@devarismeroxa
devarismeroxa merged commit 4e87748 into main Aug 5, 2026
10 checks passed
@devarismeroxa
devarismeroxa deleted the feat/ws1-fuzzymatch branch August 5, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant