fix(mcp): collapse duplicate results in memwal_recall

[harrymove-ctrl]

Problem

A limit=5 recall on a real dev account returned this:

1. [score=0.224] User prefers dark roast coffee
2. [score=0.224] User prefers dark roast coffee
3. [score=0.224] User prefers dark roast coffee
4. [score=0.224] User prefers dark roast coffee
5. [score=0.224] User prefers dark roast coffee

One preference consumed the entire retrieval budget. Every other memory the query should have surfaced was crowded out, and the model burned context on five copies of one sentence.

Why the fix is on the read side, not the write side

Storing the same fact repeatedly is legitimate. Each remember is a distinct event with its own blob and timestamp, and there is deliberately no content-level uniqueness constraint: the only unique index on the write path is

-- 013_remember_write_idempotency_index.sql
CREATE UNIQUE INDEX ... ON remember_jobs (owner, idempotency_key)
    WHERE idempotency_key IS NOT NULL;

which is request idempotency, guarding a retried request from being processed twice. It says nothing about content, and a repo-wide search finds no content dedup anywhere in the MCP tools, the SDK, or the relayer.

So this is not a regression, it is a gap. Deduping on write would silently discard a genuine re-statement of a fact, which is worse than the duplicates. Collapsing on read is safe and fixes the actual symptom.

Approach

collapseDuplicates folds results whose text matches after trimming, whitespace collapsing, and case folding, keeping the first occurrence. Rows arrive ranked best-first, so the survivor is the highest scoring copy.

Matching is exact-after-normalization, not fuzzy or embedding-based. Merging facts that merely resemble each other would hide real information; near-duplicate merging, if ever wanted, belongs behind an explicit opt-in.

The reply reports what was folded rather than quietly returning fewer rows than requested:

1. [score=0.224] User prefers dark roast coffee

(4 duplicate copies of the above collapsed; the same fact is stored more than once.)

That also surfaces the underlying repetition, which is usually worth knowing.

Tests

mcp/__tests__/recall-dedupe.test.ts, 7 cases, including the two that matter most for safety:

• leaves distinct facts untouched — an over-eager dedupe is worse than the duplicates it removes
• does not merely substring-match — "prefers dark roast coffee" and "prefers dark roast coffee in the morning only" must both survive, since losing either is silent data loss

Plus rank-order preservation, highest-scoring-survivor, whitespace/case normalization, and the empty set.

npm test in services/server/scripts: 202 pass, 0 fail.

Not included

Back-filling the retrieval budget after collapsing (over-fetching so limit=5 still returns 5 distinct facts) would need either an over-sample on every call, which multiplies SEAL decryption cost for the common no-duplicate case, or an adaptive second round trip. Worth doing, but it is a separate performance tradeoff and does not belong in this fix.

Found while verifying WALM-324. Unrelated to that fix and deliberately kept out of #681.

[harrymove-ctrl]

Folding this into #681 instead, at Harry's request: one PR is less review and merge overhead, and both changes are MCP fixes landing together.

The commit is carried over intact as 6117dabf on fix/mcp-proactive-instructions — same collapseDuplicates implementation and all 7 tests. Nothing is dropped.

Combined suite on that branch: 205 pass, 0 fail.