A Harper plugin that gives an application a shared, permissioned, live memory store for agents and
people. Any Harper app adds it to config.yaml; harper-eye, Dispatch and customer apps consume it
rather than each defining their own memory tables and endpoints.
Apps (own data sources + policy) harper-eye · Dispatch · customer apps
Plugins Reflex (corpus, lanes, answer) · Memory (this) ·
Summary Tree · Agent MCP surface
Harper core tables, subscribe, @embed, @fullText, HNSW, users/roles, mcp
| Concern | Owner |
|---|---|
| Memory table, lifecycle, scope semantics, supersession, live updates | this plugin |
| Mount path, which identities hold which grants, rollout policy | the host app (config + ScopeGrant rows) |
Embedding model behind the embeddingModel name |
the host app (Harper models config or models.registerBackend) |
| Authentication | Harper (context.user); interactive MCP clients via @harperfast/oauth mcp mode |
| Document corpus, chunking, retrieval lanes, answers | Reflex — memory joins it as a lane (Reflex #543), not as copied source rows |
Seeded from the Agent Memory Interface in harper-eye (#178): server-derived ownership, strict payload
allowlists, idempotent PUT create/revise with If-Match revision preconditions, withdraw-as-tombstone
(DELETE with If-Match), and identical 404 for "missing" and "not yours" on reads, DELETE, pins and
supersedes references. Those rules carry over unchanged unless a section below says otherwise.
| Area | v0 core |
|---|---|
| Lifecycle, scopes and grants, supersede and restore, hybrid search, bootstrap, import, sinks, principal resolver | implemented (src/) |
Write-time supersession candidates, the judge and auto mode (supersession.* options) |
implemented (src/supersession.ts) |
Provenance review flags (needsReview, confirm) |
implemented (src/provenance.ts) |
Live updates (Level 1 subscriptions, Level 2 MemoryWatch and /MemoryEvents) |
implemented (src/events.ts) |
MCP tools and memory:///{id} resource (mcp option) |
implemented (src/mcp.ts) |
Customer-content flag and quarantine (customerContentCheck option) |
implemented (src/quarantine.ts) |
| Supersession backfill: candidates for an existing memory, links between existing memories, created-order cursor | implemented (src/lifecycle.ts) |
| Package exports reaching the plugin from another app (Extension points, Across apps) | implemented (src/runtime.ts) |
scanMemories, getMemories, bootstrap sections and search annotators for derived plugins (Extension points) |
implemented (src/extensions.ts) |
| Delegation: one principal acting for another (Identity, Scopes and grants) | implemented (src/scopes.ts) |
Tables live in their own database, memory, so a host app's own Memory table cannot collide. The
plugin reads them through databases.memory, never through tables (which is the host's data
database).
Memory— one row per memory.contentis embedded on write (@embed) and full-text indexed (@fullText, which requires@table(audit: true)), so a memory is searchable at commit, with no indexing queue of the plugin's own. The vector covers only the head of the content: every embedding request (write, import, the table embedder, the search query) goes throughembedTexts(src/lifecycle.ts), cut at a code-point boundary tolimits.embeddingInputCharsUTF-16 code units, so a memory longer than the model's window still writes; full-text covers the whole. The bound is in characters (no tokenizer): the default (6,000) leaves headroom under a 2,048-token window rather than guaranteeing it. WithembeddingTruncateon, requests also carrytruncate: trueand the model cuts at its own window, keeping the start. Core (Harper 5.4) refuses that, before any request, for a backend that cannot truncate, and the cap may sit above the window, so on that refusal alone (refusesTruncate)embedTextsretries once without it, cut tomin(embeddingInputChars, embeddingFallbackChars); any other error propagates. The refusal is remembered per settings snapshot; after 5 minutes one caller asks again, so an entry that gainstruncationis picked up without a restart. Harper 5.3 ignorestruncate:probeCoreTruncatedetects that at configure time and the fallback bound applies. Off by default: a truncatingfallbackentry behind a primary that cannot would take every embed into another vector space, with no refusal to see. One write, import or search takes its model, caps and flag from one settings snapshot (EmbeddingSettings); the table embedder truncates only a row recorded under the configuredembeddingModel.ScopeGrant— read/write scope patterns held by a Harper role or user; a user's row may also name the principal it acts for (actsFor, Identity).MemoryPin— a principal's pinned memories (<principalId>:<memoryId>), loaded first at session bootstrap.MemoryWatch— a principal's time-bounded interest in specific memories (live updates).
See schema/memory.graphql. The table is not @exported: every read and write goes through the
Memory resource so the rules below have exactly one enforcement point. The exceptions are scanMemories and
getMemories (Extension points), unguarded reads for trusted in-process plugins that no route reaches.
Server-owned fields: ownerId, status, revision, author, lastEditor, supersededBy,
autoSupersedes, evidenceMemoryIds, needsReview, needsReviewSince, confirmed, contentFlag,
contentFlagged, scopeConfirmed, withdrawnBy, createdAt, embeddingModel, lastEventKind. createdAt is set by the lifecycle rather than @createdTime, because Harper's
@createdTime overwrites a supplied value on insert and import must preserve it. updatedAt stays
@updatedTime. needsReviewSince mirrors needsReview.since because Harper does not index object
values; lastEventKind records the lifecycle event kind (created, revised, superseded, withdrawn,
restored, flagged) on each write so real-time subscribers receive the exact transition.
contentFlagged is true exactly while contentFlag is set (the two are always written together), for the
same reason as needsReviewSince. None of needsReviewSince, contentFlagged and lastEventKind is ever in a
view. Rows written before a field existed read it as
absent, which every reader treats as null.
The principal is the Harper user on the request context (context.user.username). The plugin does no
authentication of its own and has no credential tables.
- Interactive MCP clients (Claude Code, desktop, claude.ai connectors): intended to authenticate with
OAuth 2.1 through
@harperfast/oauthmcpmode (withMCPAuth), which does not yet populate a Harper user object oncontext.userfor Harper core's/mcpendpoint (withMCPAuthonly attaches its middleware toserver.httpsorequest.userstaysundefinedon/mcp, and@harperfast/oauth's session-cookie hook setsrequest.userto a username string rather than{ username, role }, soresolvePrincipalrejects with401 principal_required; see HarperFast/oauth#279). Until that lands, MCP clients against/mcpauthenticate with Harper credentials or a host-app middleware / principal resolver. - Headless agents use Harper credentials bound to a user (operation tokens, or a host-app API key that resolves to a user).
- An app with its own principal registry (harper-eye's
AgentPrincipal) can install a resolver withsetPrincipalResolver((context, next) => …)that returns{ id, grants: { read, write }, superUser?, source? }, orundefinedfor unauthenticated.next()is the default Harper-user resolution, so a resolver can handle its own credentials and defer the rest. This exists for migration. - Other components on the same instance (harper-eye, Reflex) authorize with this identity and these grants
rather than their own:
resolvePrincipalandcanReadLabels(see Extension points) are the composition contract, and the plugin's own routes run on the same two functions, also when the component runs in another app (Across apps).
Principal ids must not contain whitespace, : or *, and are capped at 122 characters, so that
user:<id> fits a 128-character scope label.
Delegation. A person often reaches the plugin through several principals: their own user (owning history
imported for them) and one agent user per machine. actsFor on the delegate's user grant row
(ScopeGrant { id: "user:<delegate>", actsFor: <principal> }) lets the delegate act for the principal: edit the
principal's memories and share its private scope (Scopes and grants). It sits on the user row because that row is
already keyed by the delegate and read on every request; a delegate whose grants all come from its role gets a
user row carrying only actsFor. One field per delegate, so a principal acts for at most one other and its private
writes have one destination; only the caller's own row is read, so it is not transitive. The default resolution
reads it on every request, so clearing it (or deleting the row) takes effect on the next request; a custom
resolver returns actsFor itself. An actsFor naming the principal itself, or not a valid principal id, grants
nothing and is logged. So does one on a role: row: a role is shared by many people's agents, so it can never
name the one principal they act for.
Delegation never merges the two principals: author, lastEditor, confirmed.by and withdrawnBy name the
delegate, and the delegate keeps its own grants and user:<self>.
Memory ids are client-chosen (^[A-Za-z0-9_-]{16,64}$; clients normally generate a UUID) and share one
table-wide namespace across all owners rather than per-owner id namespaces. Because PUT /Memory/<id>
creates when the id is absent, a PUT to an existing id that the caller cannot read answers 409 id_unavailable rather than overwriting or returning 404; this reveals that the id is taken, which with
122-bit random UUIDs only matters to a caller who already knows the id.
A scope label is namespace:value (or the bare label global), at most 128 characters, never
containing * or whitespace. A memory carries 1–16 labels, stored deduplicated and sorted. Shared
vocabulary with Reflex visibility scopes (Reflex #573):
| Label | Meaning |
|---|---|
global |
everyone with global read |
user:<id> |
the owner and its delegates only (private) |
customer:<id> |
people and agents cleared for that customer's content |
team:<id> |
a team |
Each memory carries scopes: [String]. A caller sees a memory when any of its labels is in the
caller's readable set. To restrict content, use one specific label (customer:acme), not a list.
Grants are ScopeGrant rows keyed role:<roleName> or user:<username>, with read and write
pattern lists (a user row may also carry actsFor, Identity). A pattern is an exact label or a namespace wildcard (customer:*); a wildcard never
covers the bare global. Every principal implicitly reads and writes user:<self>, a delegate also
user:<actsFor>, and no grant reaches into the user namespace: user:<id> means the owner and its
delegates only, so a pattern such as user:* or user:bob is ignored rather than quietly opening private
memories (reading everything is what super_user is for). super_user reads and writes everything. Grant rows
are read on every request, so revoking one takes effect on the next request. An invalid pattern grants nothing
and is logged; at most 256 patterns per principal count (the implicit user: labels do not), which bounds the
condition every search carries.
Rules:
- Read:
scopes ∩ readable(caller) ≠ ∅, or the caller is the owner. A withdrawn memory is readable by its owner andsuper_useronly. A memory carrying a content flag is readable by its owner,super_user, and label readers whose read grants includecustomer:*only (see Customer content). A memory the caller cannot read is indistinguishable from one that does not exist (same 404 body onGET,DELETE,pinandsupersedesreferences, and{ pinned: false, memoryId }with nomemoryfield onunpin); onPUT /Memory/<id>, an existing unreadable id returns409 id_unavailable(see Identity). One deliberate exception: a review flag tells a former label reader of a withdrawn source that it was withdrawn (see Provenance). - Write scopes: creating or revising with scopes
Srequires every label ofSto be writable by the caller — the resulting scopes, so revising a memory that staysglobalneedsglobalwrite.visibility: "private" | "shared"is shorthand for[user:<owner>]/[global]; explicitscopeswins when both are given. Withdrawal is exempt. A delegate's private create goes to[user:<actsFor>], and a create whose scopes are exactly that is owned by the principal, so the principal and its other delegates can edit it; any other create is owned by its writer (createScopesAndOwner), which keeps an uncleared writer able to read, and so fix, its own content-flagged write. - Edit: only the owner, a principal acting for the owner, or
super_userrevises, withdraws, confirms, links or lists candidates, apart from the customer-content review (rule 4). A delegate must still read the memory through its labels (actsAsOwner): delegation never adds the owner's other labels or the owner's ownership-based reads (withdrawn, content-flagged), so a delegate that cannot read gets what a stranger gets. The owner keeps its own read of whatever labels a delegate gives its memory, as it does after a customer reviewer's rescope; a delegate can publish to the principal throughuser:<actsFor>anyway. Read access never grants edit; a reader that is not the owner gets 403 (not_memory_owner; on a revise it names the route left, a merged replacement withsupersedes: [id]), and a non-reader gets 404 onDELETEor 409id_unavailableonPUT. - Supersede, review flags and customer-content review: see below. They are the only writes that touch
another principal's memory: a supersede changes only status fields (
status,supersededBy,revision), a flag onlyneedsReview,needsReviewSinceandrevision, a customer reviewer's rescope onlyscopes(intocustomer:labels) and its consequences, a customer reviewer may withdraw a flagged memory, and a confirm-scope changes only the content flag,scopeConfirmedandrevision. - Search pushes the readable set into the query, applied before the limit in every lane, never
as a post-filter on a top-k. The set compiles to conditions (
src/scopes.tsreadableConditions): anorofownerId = self,scopes = <label>per exact grant (user:<actsFor>among them, neverownerId = <actsFor>), andscopes gelt [ns:, ns;)per wildcard — the exact prefix range, which Harper plans from the index wherestarts_withwould scan. Harper matches an array attribute when any element matches. For a caller not cleared for customer content, a second groupor(ownerId = self, contentFlagged not_equal true)adds the quarantine: kept apart from the label group so that group's estimate stays finite and its index can still drive a narrow caller's search, andnot_equalbecause it is true for rows written beforecontentFlaggedexisted, whereequals nullwould drop them (Harper does not index an absent attribute). Harper runsnot_equalas a plain filter; the negatednot_equalsswitches to a full index scan on its first miss, which measured 7× slower.
| Operation | Effect |
|---|---|
create (PUT) |
client chooses id (^[A-Za-z0-9_-]{16,64}$); absent id creates at revision: 1 and returns 201 { id, revision, candidates, contentFlag }; If-None-Match: * makes PUT create-only (412 if the id exists); an identical retry returns 200 { id, revision, candidates, contentFlag } |
revise (PUT) |
PUT /Memory/<id> on an owned memory whose editable fields (title, content, tags, scopes, evidence, validUntil) differ from the stored ones requires If-Match: "<revision>" (or *; missing → 428, stale → 412), bumps revision, clears needsReview, and returns 200 { id, revision, candidates, contentFlag } |
| confirm-scope | POST /Memory/<id>/confirm-scope (a reviewer holding customer:* read and write, or super_user; 403 customer_clearance_required otherwise, the missing-id 404 for a non-reader) clears the content flag and records scopeConfirmed: { by, at }; If-Match honoured when sent; 409 not_flagged, 409 on a withdrawn memory |
| supersede | create a memory via PUT /Memory/<id> with supersedes: [ids] (supersedes on a revise is 400 supersedes_not_allowed) |
| link | POST /Memory/<id>/supersedes { ids, revisions? } with If-Match: "<revision>" (missing → 428, stale → 412) links an existing memory as the replacement of existing older ones, under the supersede rules (see Supersession); returns { memory } |
withdraw (DELETE) |
DELETE /Memory/<id>?reason= requires If-Match: "<revision>" (or *; missing → 428, stale → 412); sets status = withdrawn and withdrawnBy (the caller), keeps the row as a tombstone, never deletes it |
| pin / unpin | records or deletes the caller's MemoryPin row (<principalId>:<memoryId>, up to 50 pins per principal; withdrawn pins are pruned on pin, 409 pin_limit_exceeded past 50); pin requires read access (same 404 when missing or unreadable), while unpin always deletes the caller's pin and includes memory only when readable |
| confirm | POST /Memory/<id>/confirm (owner, its delegate or super_user; 403 for another reader, the missing-id 404 for a non-reader) clears needsReview, records confirmed: { by, at } and bumps revision; If-Match is honoured when sent (412) but not required; 409 on a superseded or withdrawn memory |
status is active | superseded | withdrawn. Search returns active by default. Owners share one id
namespace: PUT /Memory/<id> to an existing id owned by another principal returns 403 not_memory_owner when readable by the caller and 409 id_unavailable when unreadable (with random
122-bit UUIDs, that only reveals existence to a caller who already knows the id).
Each evidence item is { kind, ref, note? } with kind one of memory | doc | pr | commit | url | text,
stored in caller-supplied order (order is significant for idempotent retry equality, whereas tags,
scopes, and supersedes are set-normalized). memory-kind items must name a valid memory id readable by
the writer on create and revise (missing, unreadable or malformed → 400 invalid_evidence, so citing
cannot probe existence), and their ids are stored deduplicated on evidenceMemoryIds.
Every lifecycle operation runs in a transaction of its own (not the request's, which would commit
whatever was staged before an error the resource turns into a response) and takes Harper's record
lock (Memory.lock) on every row it will change, in sorted id order, before reading it
(withLockPlan, src/lifecycle.ts). An operation plans its whole lock set before the transaction:
a PUT locks its id, its supersedes, the memories it newly cites, and the active citers of what it
supersedes; a link locks its id, the memories it newly supersedes, their active citers, and the
memories that supersede it (directly or through others; see Supersession); a withdrawal locks its id,
its supersedes (which only a link adds to, under the same lock) and its active citers. Under the locks, before its first write, the operation re-reads every row it will
change; one the plan missed (a citer that appeared, a supersedes populated after the pre-read)
aborts the attempt, and the next attempt locks the union in one sorted pass, so no operation waits
for a lock while holding one out of order; after three attempts it answers 503 unavailable. The
planning reads run only for a request that would pass the cheap checks (owner, a matching If-Match,
explicit targets and cited memories readable), so a refused request never locks a citation graph or a
memory it cites. An operation that supersedes or withdraws takes a 5-minute lease instead of Harper's
30 s default, since it flags every citer inside the transaction. No model call runs under a lock: a
PUT that writes content embeds it before the transaction and carries the vector in the write, and
the table's embedder (setEmbedAttribute, installed for every model) uses a carried vector, embedding
only a write that carries none. A PUT whose transaction finds it must write content it has
no vector for (another writer changed the row after the pre-read) releases its locks, embeds, and
retries. An embedding failure fails the write before it takes any lock. The plugin is the
only writer of Memory, so on one node a read-check-write is serialized: of N concurrent revisions at
the same If-Match revision exactly one succeeds. Any status change, including supersede and restore, bumps the
affected row's revision. The lockScope setting sets the scope of the Memory and MemoryPin locks (lockRows):
node(default) serializes within a node only. Across nodes, two revisions at the same revision can both succeed and replication keeps the later write — the limit harper-eye documented for optimistic concurrency. The same applies to derived status: a withdrawal's restore on one node and a supersede of the same memory on another replicate as competing writes, so the memory can end upactivebeside a live superseder until a later operation recomputes it.clustertakes Harper's cluster-wide record lock, which needs record locks on thememorydatabase (replication.recordLocksand an agreed home map). Without them every lifecycle write and pin answers 503unavailable: an explicitly cluster-scoped lock fails closed rather than falling back to a node lock. With every node up and every holder committing within its lease, exactly one of N concurrent revisions at one revision succeeds, on any nodes. Harper's lock guarantees exclusion, not that the next holder reads the previous holder's commit after a failure: once a holder crashed, became unreachable, or committed after its lease, Harper's recovery path can admit a holder that reads a stale revision, and two revisions can both commit. While any node of the home map is down, a write whose lock needs that node — a memory it homes, one it locked recently, or any first lock after a home restart, which waits on every node — answers 503 until it returns or an operator publishes a home map without it. The 5-minute lease does not bound this; Harper's delegation lease (6 minutes) and recovery barrier do.- Every node runs the same
lockScope. A node atnodenever asks for the cluster lock, so while a change rolls across the cluster two revisions can both commit; change it on every node while no lifecycle write is running. - Watches stay node-scoped at either setting (
putWatchKeepingLongerTtl), so reads never wait on, or fail because of, another node. Across nodes, two watches of one memory by one principal keep the later write, which can be the shorter TTL; Live updates already promise at-most-once delivery, and the next watch renews it.
Measured on the team's previous memory store: only 8% of updates used an explicit supersedes link,
and 32% of sessions read a stale memory without ever seeing its replacement. So supersession is a
first-class operation, and the write path proposes it.
- Explicit:
supersedes: [oldId](up to 10) on create (PUTto a new id), or later by a link (below). Requires that the caller can readold, thatoldis not withdrawn (409), and that the new memory's scopes are a superset ofold.scopes(403), so every label reader of the old memory can read its replacement. In the same transactionold.status = superseded,old.supersededBy = newId. An already superseded memory may be superseded again;supersededBynames the newest. - The superset holds after the supersede, not only at it: a superseded memory is frozen except
for withdrawal (revising via
PUT→ 409memory_superseded), and a scope change on a memory that supersedes others must still cover each non-withdrawn one. - Owner-only readers are not covered: an owner who reads the old memory only by ownership (none
of its labels is in their readable set) sees
supersededBybut may be unable to read the replacement. Covering them would publish the replacement to a principal its author did not choose. While the replacement is content-flagged, an uncleared reader seessupersededByas null (Customer content). - A memory is superseded exactly while a non-withdrawn memory lists it in
supersedes; the newest such memory (bycreatedAt, then id) is itssupersededBy. A superseder that is itself superseded (or expired viavalidUntil) still counts — its replacement carries the chain forward, and callers passincludeSuperseded=trueorincludeExpired=trueto inspect the chain. Withdrawing a superseder recomputes each memory it superseded under that rule, returning it toactivewhen no superseder is left; import derives status by the same rule. The lookup reads non-withdrawn superseders newest first through the indexedsupersedesandcreatedAt, so it is exact however many there are. - Link after the fact (
POST /Memory/<id>/supersedes { ids, revisions? },linkSupersedes): for memories that already exist, typically imported history whose source never recorded the link. It runs the create-time code (planSupersede/applySupersede, shared withPUT): the same refusals per target (404 missing or unreadable, 409 withdrawn, 403 narrowing), the same pointer rule, and the same flags. The caller owns the replacement, acts for its owner, or issuper_user(a reader that is not the owner gets 403, a non-reader the missing-id 404) and, as on a create, can still write every label of the replacement (403scope_not_writable).If-Matchnames the replacement's revision;revisions, when sent, maps every id to the target revision the caller judged, checked under the target's lock (412). The replacement gains the new ids insupersedes, arevisionbump andlastEventKind: revised; itsneedsReview,lastEditorandautoSupersedesare untouched (a link is neither the owner's review nor a judge's link; the customer-content check re-runs, over the replacement's whole text, only when a new id itself names a customer, as it does for the ids of a create; a flag or a review otherwise stays, and a link never asks the model check). Ids it already supersedes are skipped, so a repeated link writes nothing. A memory supersedes at most 100 (409supersedes_too_many, the import cap), bounding what a withdrawal re-derives. The cap and the target checks, revisions included, run on the planning read too, so a refused link locks only the replacement. - The replacement may itself be superseded (not withdrawn), so a chain found in history can be
linked newest first: after C supersedes B, "B supersedes A" still lands, and A reads
supersededBy: B. The freeze on superseded memories protects their scopes, which a link does not change. What it must not do is close a cycle, where every member is superseded and none active. Adding R → T closes one exactly when T already supersedes R directly or through others (non-withdrawn holders only; a withdrawn superseder's links no longer count). The link walks those ancestors of R through the indexedsupersedes, locks them with its plan, walks again under the locks, and answers 409supersede_cycle. Any link that would make T reach R adds an edge into one of those rows, so it locks one of them and serializes with this one. A create with targets runs the same check, because an import may list an id before it exists (a link may name a row in a later batch), so a new row can already have ancestors. Import itself preserves source history as given; its client drops cycles. A create of an id an import already lists is still storedactive(only import derives the new row's own status from the graph). - Candidates for an existing memory (
GET /Memory/<id>/candidates,memoryCandidates): the write-time candidates (below) of a stored memory, from its stored vector, restricted to memories created before it (createdAt, then id), as{ candidates: [{ id, title, similarity }] }. Owner, its delegate orsuper_user, with the same 403 / missing-id 404 split as a link, and 409 for a withdrawn memory. It neither judges nor applies, ignoressupersession.mode(which governs writes), and, unlike a write's advisory search, waits for index coverage (as a search does: about one index flush, 503 past 5 s) and fails with the search (503 / 500) instead of answering[], so a report run right after a bulk import misses nothing. The older-than rule is in the query (createdAt <= t, thencreatedAt < t or id < selffor the same instant), so same-millisecond rows with larger ids cannot fill the nearest-k before the cut. - Consequences of linking: a create retried with its original
supersedesafter a link no longer matches the stored row (400supersedes_not_allowed), like any retry after a later change; an import re-run reports a linked row asconflict. - Write-time candidates: every successful
PUT(create, revise, identical retry) returnscandidates: [{ id, title, similarity, verdict?, applied? }]: up to 5 of the nearest active, unexpired memories the writer can read that share at least one scope label with the written memory, other than itself and what it supersedes, with cosine similarity at leastsupersession.similarity(default 0.82). The search runs after the write commits, with the row's own vector and only among rows of the row'sembeddingModel, and does not wait for index coverage: candidates are advice, so a memory committed within the index lag can be missed, and a failed search yields[]rather than failing a committed write. It never holds a record lock.supersession.mode: offsearches nothing and returns[]. Measured on Harper 5.3.2 (one node, local, CPU capped at 2 GHz, 300 memories, judge stubbed): a create took 7.7 ms p50 / 10.2 ms p95 insuggestagainst 2.7 / 3.4 ms withoff, and an identical retry 6.1 / 7.6 ms against 1.9 / 2.4 ms. A real judge adds its model's latency, per candidate and in parallel. - Judge: with
supersession.judge(amodels.generatemodel name), each candidate is classifiedsupersedes | refines | unrelated. Only a complete one-word answer counts (at most 16 output tokens, 20 s per candidate); anything else, or a failed call, leaves the candidate without averdict. The judge runs outside every transaction, and only for aPUTthat wrote a new title or content (a retry or a tags-only revise gets its candidates unjudged). - Auto (
supersession.mode: auto, requiresjudge; defaultsuggest): on a create, the plugin searches with the write's vector and judges before the transaction, because aPUTcannot changesupersedesafter create. Eachsupersedesverdict joins the create'ssupersedesunder the explicit rules and only if the candidate'srevisionis still the one the judge compared;applied: true | falsesays whether it did. Applied links are also listed inautoSupersedes, so an identical retry compares only the links the author sent, and readers can tell the judge's links from the author's. Auto adds at most 5 links beyond the 10 a request may name. On a revise, verdicts are advice only. The judge can only cause a supersede the writer could have requested explicitly, so a prompt injected through memory content gains no access.
When a memory X becomes superseded (active → superseded) or withdrawn, every active memory D whose
evidenceMemoryIds contains X gets, in the same transaction, needsReview: { since, cause: superseded | withdrawn, sourceId: X, replacementId? } (replacementId is X's supersededBy at that
moment), a revision bump and a sink write (flagDependents, src/provenance.ts). A flag is
advice for a human or agent to re-check D, never an automatic invalidation: a conclusion often survives
one of its supports changing.
- One hop: flagging D never flags the memories citing D, and neither does revising or confirming D or X; only another supersede or withdrawal flags.
- Only transitions flag: superseding an already superseded memory, re-pointing
supersededBy, and restoring a memory toactiveflag nothing, and a restore does not clear flags it left behind. A memory that a withdrawal restores, and that cites the withdrawn memory, is flagged like any citer. - First cause stands: a memory already flagged keeps its first flag, unrewritten, until its owner
revises it or calls
confirm. A review should therefore re-check all of its memory evidence. confirmedis an event, not a state: it records the latest confirmation{ by, at }and survives later revisions; compare it withlastEditor.revisedAtto see which came last.- Not its replacement: a memory that lists X in
supersedes(on create, in the same import, or by a link) is never flagged for X; a link patches the replacement before it flags. - Citing serializes with the transition: a create or revise locks the memories it newly cites, so a citer either commits first and is flagged, or reads X's new status. A citation the row already has is in the index, where the transition finds the citer and locks it. The lock is held for milliseconds, since the citer's embedding happens before its transaction. Across nodes the same caveat as every other lock-based rule applies.
- Who sees a flag:
needsReviewis shown to a reader who can read X through its labels or owns it, ignoring withdrawal (that reader could read X before, so learning it was withdrawn reveals no existence it did not know); everyone else seesnull.replacementIdis shown only to a reader who can read the replacement now. Sinks receive the unredacted row. - Fan-out: one transaction flags every citer, with one sink call each, under the 5-minute lease.
Keep sinks to a write in the
memorydatabase (an outbox) so a heavily cited memory can still be withdrawn. - Prospective only: flags start with the transition; memories that were already stale when this shipped are not flagged until a link supersedes their source (the supersession backfill does).
global is read by everyone, fleet workers included; customer-identifying detail (names, hosts, clusters,
data) belongs under customer:<org>, read only by roles granted customer:*. A write that puts likely customer
detail in a global memory is never rejected — a false positive must not lose an agent's write — but it is
flagged and quarantined until a cleared reviewer rules on it, so a true positive never reaches a reader that
reads global only (src/patterns.ts, src/quarantine.ts).
- Configuration (
customerContentCheck, absent = off):mode: off | flag(defaultflagwhen the block is present),model(optionalmodels.generatename),patterns.namesFile(absolute path) orpatterns.namesSecret(a name in the application'sscope.secrets: a scoped secret, or a declared env name), andpatterns.hosts(at most 256 of{ label, host }, an exact host name or*.for any subdomain, or{ label, cidr }, an IPv4 or IPv6 address or range). The names source is JSON{ "<org>": ["Name", "Alias", ...] }. It is read when the plugin configures (startup and every option change), never from the repository, and no message about it echoes its content or keys. There is deliberately no regular-expression option: the text is the writer's, so matching must stay linear in it (one split into words, a token trie whose cost is the longest name rather than the list's size, map lookups,net.BlockList). - The check reads every author-written field and every id the memory shows a reader: its chosen
id, title, content, tags, evidence refs and notes, and the ids it supersedes (an evidencekindis not read, so changing one alone is not a text change). A name matches as a whole token sequence (NFKC, lowercased, letter and digit runs), soAcmematchesacme-prod.example.comand notAcmeville(in a script written without spaces, a name matches only a whole run). Labels:customer:<org>per matched name, the configured label per matched host or range. - When: on a create, and on a revise whose resulting scopes include
globaland that changes the checked text or newly includesglobal; import checks its created rows by pattern only, before its locks (a batch would otherwise start a model call per row). A match setscontentFlag: { kind: 'possible-customer-detail', since, source: 'pattern', labels }(keepingsincewhile the memory stays flagged). Leavingglobalclears the flag; so does a re-check that matches nothing, except that a flag is kept, without labels, while a configured model re-checks the text. A revise that changes neither the checked text norglobalkeeps whatever state the memory is in.mode: offchecks nothing and keeps existing flags enforced. Status-only writes never touch the flag. Flags are prospective: changing the names list re-checks nothing already stored. - The model (optional) runs after the write commits and never holds up its response or a lock: a strict
one-word answer
customer | clean(30 s), at most 8 provider calls in flight per worker thread (counted until the call itself ends, even past its deadline; more are skipped with a rate-limited warning), failures logged with fixed text only. Its verdict applies in its own locked transaction only under the same settings object (a reload invalidates it), with the row stillglobaland its checked text,contentFlagandscopeConfirmedunchanged:customersets asource: 'model'flag (taking over, with itssince, a pattern flag kept for earlier text;lastEventKind: 'flagged'), andcleanclears a kept flag only if the row'srevisionis also unchanged, since clearing publishes. Between commit and verdict a model-only true positive is readable byglobalreaders, and a check lost to a restart or skipped leaves it so: the patterns are the synchronous guarantee. - Quarantine: see rule 1 and rule 5. Clearance is the
customer:*read wildcard; an exactcustomer:<org>grant does not clear, since a flag can name any customer. The flag only removes access, never grants it.contentFlag.labelsis shown only to a cleared reader: an owner without clearance sees the flag but not which customer it matched (the flag's existence still tells a writer that a name is on the list; that is inherent in telling the owner at all). A flagged memory cannot be cited, pinned, watched or superseded by an uncleared reader (same 404 or 400 as missing), and a review flag whose source is quarantined is hidden from them. A flagged replacement breaks the supersession guarantee that label readers can read the replacement: they still read the superseded predecessor (hidden from default search), withsupersededByshown as null because the replacement's id is checked text that may name the customer. - Review: the owner clears a flag by revising the detail out or rescoping out of
global(its own write grants apply). A reviewer —super_user, or a principal holdingcustomer:*read and write — may revise someone else's flagged memory when the payload is onlyscopesand every resulting label is acustomer:label (the owner'sneedsReviewsurvives it), withdraw it, or call confirm-scope. Ownership alone never qualifies: an owner without clearance confirming its own flag would publish exactly what the flag withheld. A flagged memory that supersedes aglobalone cannot leaveglobal(the superset rule), so its review is confirm-scope or a withdrawal, which restores its predecessor.scopeConfirmed: { by, at }describes the current text: a revise that re-runs the check (the checked text changed, orglobalis new) or leavesglobal, and a model flag, clear it, so a later mention of the same customer is flagged again. - Across nodes the locks are node-scoped under the default
lockScope: node(Lifecycle), so a revise that flags a memory on one node and a confirm-scope or modelcleanof the same revision on another can both commit, and replication can keep the revise's text with the other write's cleared flag. - Cost, measured on Harper 5.3.2 (one node, local, a loaded host, 2,000
globalmemories): the quarantine group added about 0.4 ms to a 3 ms search withoutqand stayed within run-to-run noise for a hybrid search; aPUTwith the check on (2,000 organizations) was within noise of the check off; the replacement read that lets an uncleared reader see a superseded memory added about 0.4 ms to its 0.5 msGET, and anincludeSuperseded=truesearch of 10 results stayed within noise (2.1 against 1.9 ms). The matcher alone took 0.85 ms p50 for 20,000 characters against 5,000 organizations × 3 names and 64 host rules, 4.2 ms for 100,000, and 1 ms for 20,000 characters of a word that starts 50,000 names.
Mounted at urlPath (default Memory, with the per-caller watch stream at <urlPath>Events, default
MemoryEvents); every HTTP route class stays hidden = true, while MemoryMcpResource (registered at
MemoryMcp with static path = '/MemoryMcp', no prototype CRUD methods, and MemoryEndpoint's static
verb and connect overrides so direct HTTP, SSE, or WebSocket requests to /MemoryMcp return 405) is
mounted with hidden = false only when the plugin option mcp is true (default false).
| Route | Purpose |
|---|---|
PUT /Memory/<id> |
create (201 { id, revision, candidates, contentFlag }), idempotent retry (200), or revise with If-Match (200) |
GET /Memory/<id> |
read ({ memory } + ETag: "<revision>") |
DELETE /Memory/<id>?reason= |
withdraw with If-Match: "<revision>" (tombstone, returns { memory }) |
GET /Memory/?q=&tag=&scope=&limit=&includeSuperseded=&includeExpired=&flagged=&watch= |
search (no q: newest first; flagged=true: content-flagged memories the caller can read; watch=true watches every returned id) |
GET /Memory/?order=created&after=<createdAt>,<id>&limit= |
list oldest first by createdAt, then id, one page after the cursor (no q) |
GET /Memory/?…&includeWithdrawn=true |
also list withdrawn memories (super_user only; the response carries includeWithdrawn: true) |
GET /Memory/<id>/candidates |
older memories this one may replace ({ candidates }; owner, its delegate or super_user) |
POST /Memory/<id>/supersedes |
link existing memories as superseded by this one ({ ids, revisions? } + If-Match; { memory }) |
GET /Memory/bootstrap?q=&budget=&entryOverhead=§ionItemOverhead=&watch= |
session-start context (watch=true watches every returned id) |
POST /Memory/<id>/pin · /unpin |
per-reader pin / unpin (empty or {} body) |
POST /Memory/<id>/confirm |
owner's review: clears needsReview, records confirmed (returns { memory }) |
POST /Memory/<id>/confirm-scope |
customer reviewer's ruling: clears the content flag, records scopeConfirmed (returns { memory }) |
POST /Memory/watch |
watch readable memories ({ ids, ttlSeconds? } → { watched, expiresAt }) |
GET /Memory · /Memory/<id> (SSE / WebSocket) |
Level 1 record-change subscription ({ id, kind, revision }) |
GET /MemoryEvents (SSE / WebSocket) |
Level 2 per-caller watch and needsReview event stream ({ id, kind, revision }) |
POST /Memory/import |
bulk import (super_user) |
Errors are { error, code } (plus index for an import row). Only errors the plugin creates carry
their own message; anything else is a 500 internal_error. Harper's retryable failures — 503 (an index
rebuilding or lagging, a database closing), 423 (a record lock not granted within 30 s), the 409 of
a lock lease that lapsed before commit, and transient embedding backend failures (429, 5xx, timeout,
connection failure) on writes — become a 503 unavailable to retry, with a rate-limited warning
in the log. Any other 409 from Harper (for example a sink inserting a row that exists) is a 500. Import
embeds the rows it will create before its transaction, in requests of at most 100,000 characters, and holds its locks with a 5-minute
lease, since a batch writes up to 100 rows and flags their citers inside one transaction.
GET /Memory/?q=&tag=&scope=&limit=&includeSuperseded=&includeExpired=&watch=— hybrid: HNSW overembedding(query embedded with the same model) and the@fullTextindex, fused with reciprocal rank fusion (k = 60,2 × limitper lane). If embedding the query string fails, search and bootstrap fall back to the full-text lane and log a rate-limited warning. A full-text query rejection (Harper's 400Full-text query on 'text' is invalid) instead serves successful vector results, with a fixed warning at most once per minute per worker thread and no prompt or raw error in that warning. Empty successful results count as a healthy lane; two failed lanes still error. Unexpected lane errors still propagate. When both fail, unexpected vector errors take precedence; an embedding failure takes precedence over a recognized query rejection, but not an unexpected full-text error. A vector dimension mismatch defers to the full-text error while retaining its model-rejection diagnostic. This preserves a retryable embedding outage as 503. Lane collection waits for both outcomes, including their existing index-wait bounds. Both lanes carry the status, expiry (validUntilnull or> nowunlessincludeExpired=true) and readable-scope conditions, including the customer-content quarantine.scope=narrows within the readable set; it never widens it.flagged=trueadds the indexedcontentFlagged = true, the review queue of a cleared reviewer (an uncleared caller sees only its own). Results includesupersededBywhenincludeSupersededis set, andneedsReviewunder the Provenance visibility rule; withdrawn memories never appear (except to asuper_userpassingincludeWithdrawn=true, below). The vector lane has no distance cutoff, so it contributes the nearest readable memories whatever their distance. Passingwatch=truerecords aMemoryWatch(with the default 1 h TTL) for every memory id returned in the result. A search withq(REST, or MCPrecall) also carries each registered search annotator's value for a hit (Extension points); a listing withoutqcarries none.- Full-text query boundary (observed locally on Harper 5.3.2, native fulltext 0.5.0):
Array.from({ length: 65 }, (_, i) => 'w' + i).join(' ')is only 249 characters but is rejected; 64 distinct analyzed terms succeed. NativeanyreportsE_INVALID_ARGUMENT: search text produces more than 64 terms; Harper hides that reason in its generic 400, which the resource would otherwise normalize to 500. Period-separated terms also hit the limit; repeating one word 65 times does not. Punctuation-only, single-letter, code-like and URL samples, and a single token up to 1000 characters did not reject.matchesusesany, so this reproduction is unrelated to prefix expansion. The native runtime reports 64 query terms, 256 clauses and 65536 UTF-8 query bytes; the Memory index has two text fields. Integration tests pin the 64/65 boundary and hook queries against the actual index. Production's version was reported as 5.3.2, not independently inspected. Other error shapes fail closed; Harper #3447 requests safe native diagnostics and documentation of the budget. GET /Memory/?order=created&after=<createdAt>,<id>&limit=— withoutq, lists oldest first bycreatedAt, then id, under the same status, expiry, tag, scope and readable-set conditions as a search.afteris the last row of the previous page (createdAtas ISO, then id). The query leads with a plaincreatedAt >= t(a floor date on the first page) plus the tie-breakcreatedAt > t or id > lastIdand the usual conditions. For a walk of everything (asuper_userwith notag,scopeorflagged) it setsenforceExecutionOrder, so Harper scans thecreatedAtindex from the cursor whatever the other conditions estimate; for a narrower listing Harper starts from its most selective condition and sorts the few matching rows, since pinning the scan would make a caller who reads 3 rows of a large store read the whole index. Every page holds exactlylimitrows even when an import batch shares one timestamp, and nothing is skipped or repeated. A page inside a group of rows sharing onecreatedAtstill reads and sorts that group by id. Measured on Harper 5.3 (one node, local): walking 5,000 memories that include a 2,000-row same-timestamp group took 101 pages at limit 50, 3 ms p50, 13 ms p95 and 27 ms at worst (inside the group); 484 ms in all. A memory created during a walk with an earliercreatedAt(an import preserving its source time) lands behind the cursor; walk again to see it.orderwithq, orafterwithoutorder=created, is 400. Neither condition usesequalsoncreatedAt: Harper 5.3's count estimate forequalson a Date attribute throws.includeWithdrawn=true(any list or search) admitswithdrawnto the status condition, for asuper_useronly: any other caller gets 403include_withdrawn_forbiddenbefore any query runs, rather than a silently partial answer. It exists for a full export (Copy between instances); owners read their own withdrawn memories by id. A server that predates the parameter ignores it, so the response acknowledges it withincludeWithdrawn: trueand a client that needs every row refuses a page without that. A request without it builds the conditions it always did.- Searchable at commit: both indexes are Harper derived indexes maintained after commit (
audit: trueputs@embed's HNSW index on the native plane). Harper 5.3 makes a query that passeswaitForIndexMillisecondswait out the next coverage publication after any commit in the database, indexed table or not, so awatch=truerecall's ownMemoryWatchrows made the next recall wait 400–1000 ms. Search therefore waits only for a recent write the caller can read. Before its lanes run it reads theupdatedAtindex over the last 10 s (RECENT_WRITE_WINDOW_MS), up to 100 rows, and checks each withcanRead. This is a yes/no probe that returns no rows, so it is not a post-filtered read; checking in code keeps a narrow caller from walking another tenant's bulk write, and a full page counts as a bulk write in progress. A readable row, or a full page: each lane passeswaitForIndexMilliseconds(5 s) and covers every write committed before the search began; an index that is already current does not wait. Otherwise the lanes pass no wait andmaxIndexLagMilliseconds3 s (INDEX_LAG_MS, Harper's default), so Harper serves coverage at most 3 s old, and a lane whose coverage it cannot certify that fresh (503DERIVED_INDEX_LAGGING: start-up, a rebuild, a stalled publisher) runs once more with the wait.@updatedTimeis the transaction's timestamp, taken at its first lock or read, so a row the probe missed committed in time to be covered unless its transaction stayed open more than 7 s (the window less the lag): a host sink, or a wait behind another writer's lock. Harper ends a transaction open paststorage.maxTransactionOpenTime(30 s by default). A memory replicated from another node keeps its origin'supdatedAt, and one already older than the window on arrival can be missed for one coverage publication (~1 s). Measured on Harper 5.3.2 (one node, local, 200 memories): a search right after awatch=truesearch took 483 ms p50 and 937 ms p95 when every search waited, 5.5 ms p50 and 8.9 ms p95 with the probe. A caller without a write of its own searched in 7.3 ms p50 with an empty window and 6.9 ms with 99 rows it cannot read in it (25 searches each). A coverage wait that exceeds 5 s returns 503unavailable. - One vector space: each row records the
embeddingModelthat embedded it, and the vector lane only compares rows of the configured model. ChangingembeddingModeldoes not re-embed existing rows; they stay reachable through full-text until revised. A change of input bound or truncation (a fallback included) does not re-embed either: those rows share one vector space, their vectors cover different lengths of head, and a row gets the new bound when its content is next revised. A model whose dimensions differ from the indexed vectors is not supported yet: Harper's HNSW plane fixes its dimensions at the first vector, skips (and logs) vectors of another size, and rejects a query vector of another size. Search then serves the full-text lane alone and logs the rejection once per mismatch episode for that model. Switching dimensions needs a re-embedding migration, which v0 does not provide. GET /Memory/bootstrap?q=&budget=&entryOverhead=§ionItemOverhead=&watch=— the caller's pinned memories (up to 50MemoryPinrows forprincipal.id, newest pin first, whose target memory is active, unexpired and readable by the caller), then the caller's own memories (for a delegate, also those inuser:<actsFor>it may read, not the principal's whole corpus:ownViewCondition) that need review (sectionneedsReview: up to 20 the caller may see, newest flag first, through the indexedneedsReviewSince, from at most the 100 newest flags, so hidden flags cannot make a bootstrap scan without bound), then the sections other plugins registered (Extension points), then the caller's recent memories, then memories relevant toq, deduplicated and packed into a character budget (title + content; default 8000).entryOverhead(0..10000, default0) is added to each packed memory's size andsectionItemOverhead(same range, defaultentryOverhead) to each contributed item's, so a client that frames every entry passes its framing and asks for exactly the room it renders into;usedincludes the charge. They are two numbers because a contributed item carries no provenance attributes and frames smaller. Expired memories (validUntil <= now) are excluded from every section. A memory or contributed item that does not fit is skipped rather than ending the pack. Passingwatch=truerecords aMemoryWatch(default 1 h TTL) for every memory id returned across all sections. Used at session start.- Ranking may add recency decay and retrieval/citation counts later; decay is a ranking function only, never deletion.
Three levels, built in order:
- Record changes:
Memory(/Memoryand/Memory/<id>) supports Harper real-time subscriptions over WebSocket and SSE (Accept: text/event-stream). Every event is checked against the subscriber's read rule (canRead) at delivery time:ScopeGrantrows and custom principal resolver results are cached for up to 5 s and invalidated via the worker hub'sScopeGrantsubscription, which carriesactsFortoo (a resolution that began before an invalidation is not cached), so a scope narrowing or grant revocation revokes visibility mid-subscription; Harper user role assignments oncontext.userare captured when the connection opens and take effect on reconnect. Forwithdrawn, read access checks the memory's scope labels ignoringstatus = withdrawn(matching Provenance), so label readers and watchers learn that a shared memory they could read was withdrawn (GET /Memory/<id>still returns 404 to non-owners; a content-flagged memory's withdrawal reaches cleared readers only). Events carry{ id, kind, revision }— never content — withkindone ofcreated,revised,superseded,withdrawn,restored, orflagged(needsReviewset, or a model's content flag; aflaggedrow that carriesneedsReviewreaches only a reader who can see that flag), so a reader re-reads through the guardedGET. Subscriptions are live-only (omitCurrent: true) with at-most-once delivery and no replay across disconnects; if Harper core re-emits existing rows on a table reload (copyApply,harper-pro#495) beyond the per-connection 1,024-entry deduplication window, rows may replay their storedlastEventKind, which clients handle idempotently by comparingrevision. If a slow subscriber accumulates more than 256 undrained events in its client buffer (or its filter stalls for 30 s with 256 pending events), or if the worker'sMemory,MemoryWatch, orScopeGrantsubscription ends, active streams close so the client reconnects and re-reads watched ids (comparingrevision) or re-bootstraps (watch=true) to catch up. - Watches:
POST /Memory/watch { ids, ttlSeconds? }(1–100 ids;ttlSecondsinteger in1..86400, default 3600) andwatch=trueon search and bootstrap recordMemoryWatchrows (<principalId>:<memoryId>,expiresAt @expiresAt), keeping whicheverexpiresAtis later when a watch already exists (on one node; see Lifecycle for concurrent watches on two) and returning the earliest effectiveexpiresAtacross the watched ids. Each worker indexes active watches per principal in memory (lazily pruning expired entries on lookup and sweeping expired entries whenever a principal's bucket reaches 4,096 watches).POST /Memory/watchrequires read access on every requested id and rejects unreadable ids with the same 404 (memory_not_found) as missing ones before writing any watch. A per-caller event stream at/MemoryEvents(<urlPath>EventswhenurlPathis customized; identity comes fromcontext, with no principal in the path —/MemoryEvents/<principal>returns 404) delivers{ id, kind, revision }over SSE or WebSocket whenever:- a memory with an unexpired
MemoryWatchfor the caller changes and passes the caller's delivery-time read check, or - one of the caller's own memories (
ownerId === principal.id, or for a delegate one inuser:<actsFor>:inOwnView; a delegation added while the stream is open reaches this at reconnect, a revoked one stops delivery from the next event) is flaggedneedsReview(kind: 'flagged', orrestored/createdwith a newly foldedneedsReviewflag visible to the caller), or a create or revise newly content-flags it (any flag on a created row; on a revise, the flag'ssinceequals that write'slastEditor.revisedAt, so editing an already flagged memory is not news; two writes in one millisecond can repeat the notice). The notification fires when "a memory you relied on changed, or your own memory needs review", not on every write. A watcher that cannot read a memory once it is content-flagged hears nothing more about it, as after a scope narrowing. - MCP resource subscriptions (
resources/subscribe): Harper core'sresources/subscribehandler (subscribeToResourceincomponents/mcp/resources.ts) only acceptsharper+rest://andhttp(s)://URIs and rejects custommcpResourcesURI schemes such asmemory:///{id}with-32602 resource is not subscribable: memory:///{id}. Per design we do not patch Harper core: the needed core hook is filed at HarperFast/harper#3250, and MCP clients should subscribe to/MemoryEvents(calling thewatchtool orrecallwithwatch: true) instead.
- a memory with an unexpired
- Standing queries: notify when a new memory would now rank in a result a session already used. Depends on core post-commit derived-index delivery (harper#2489). Not in v0.
Behind the plugin option mcp (boolean, default false), handleApplication mounts MemoryMcpResource
(static path = '/MemoryMcp', static hidden = false, extending MemoryEndpoint with no prototype CRUD
methods) to expose MCP tools and a resource template on Harper's native /mcp endpoint (src/mcp.ts),
each delegating to the same lifecycle, search, and watch functions and enforcing the same permission checks
as REST:
static mcpTools:remember— create a memory (PUT; generates a UUID whenidis omitted, or idempotently retries an identical create).rememberandrevisereturn the storedscopes,status,needsReviewandcontentFlagbeside{ id, revision, created, candidates }; flagged writes also include a fixedquarantineNoteexplaining that readers without customer clearance cannot read the memory while its customer-content flag remains set.revise— revise an existing memory (PUTwith requiredrevisionmapped toIf-Match: "<revision>").recall— hybrid search (q,tags/tag,scopes/scope,limit,includeSuperseded,includeExpired,flagged, and optionalwatch: trueto watch every returned id).get— fetch a single memory byid({ memory }, or 404 if missing/unreadable).withdraw— withdraw an owned memory (DELETEwith requiredrevisionand optionalreason).pin/unpin— pin or unpin a memory for session bootstrap.confirm— clearneedsReviewon an owned memory and recordconfirmed(optionalrevisionmapped toIf-Match).confirm-scope— a customer reviewer's confirm-scope (optionalrevisionmapped toIf-Match).watch— watch1..100readable memory ids ({ ids, ttlSeconds? }→{ watched, expiresAt }).
static mcpResources:uriTemplate: 'memory:///{id}'(mimeType: 'application/json',method: 'mcpReadMemory'), whose read callsgetMemoryand returns the guarded memory view (orundefinedwhen missing/unreadable, which Harper's MCP layer surfaces as a JSON-RPC read error).- Tool and resource handlers are bound on
MemoryMcpResource.prototypeundermcp*method names (mcpRemember,mcpRevise,mcpRecall,mcpGet,mcpWithdraw,mcpPin,mcpUnpin,mcpConfirm,mcpConfirmScope,mcpWatch,mcpReadMemory), while every HTTP route class stayshidden = true, so neither Harper's MCP discovery nor OpenAPI export auto-generates generic REST verb tools or unguardedharper+rest://templates alongside the explicit surface.
registerMemorySink({ write(memory, context), withdraw(memory, context) })returns an unregister function. Sinks are awaited in order inside the lifecycle transaction for every row an operation changes (create, revise, confirm, the supersede, restore and review-flag side effects →write; withdraw →withdraw; import callswithdrawfor a row imported as withdrawn), and only after every lock the operation needs is held, so an attempt the lock plan restarts never reaches a sink. A write the sink makes throughcontextto a table in thememorydatabase commits or rolls back with the memory, and a sink that throws aborts the operation. Harper commits each database separately, so a write to another database can be lost if the process dies between the two commits, and a create retry would not repair it. Anything that must not be lost goes through an outbox table declared in thememorydatabase. A sink registered or unregistered during a dispatch takes effect from the next one. An idempotentPUTretry calls no sink. harper-eye uses it for the interim Reflex source-row handoff until memory is a Reflex lane.scanMemories({ after?, includeInactive? })— an async iterable ofMemoryrows as stored (the view's fields with their stored types, plusembeddingandembeddingModel), oldestcreatedAtfirst, then id. By default onlyactiverows unexpired when the scan starts;includeInactive: trueyields every row, superseded, withdrawn and expired ones included.after: { createdAt, id }, the last row a caller saw, resumes strictly after it with the?order=createdtie-break, so rows sharing acreatedAtare neither skipped nor repeated. It takes no principal and applies no scope rule: it is the one read outside the guarded routes, for trusted in-process plugin code building derived data, and no route reaches it. It reads in pages of 100 (SCAN_PAGE) with a context of its own (authorize: false), so a consumer that pauses between rows holds no read open. It is a walk, not a snapshot or a change feed: a row revised, withdrawn or expiring behind the cursor is not revisited, a row created with an earliercreatedAt(an import keeping its source time) lands behind it, and expiry makes no sink call. A derived store therefore stays current from a sink registered before its walk starts, comparesrevisionbefore applying a scanned row over sink-driven state, and after a gap in which its sink did not run walks again from the start withincludeInactive.getMemories(ids)— a promise of theMemoryrows with those ids, shaped asscanMemoriesyields them (stored fields,embeddingandembeddingModelincluded) and in every status, asscanMemories({ includeInactive: true }): the caller applies eligibility. Rows follow the order ofids, a repeated id once, and an id with no row is left out. It reads in queries of at most 100 ids (SCAN_PAGE), not a get per id, takes no principal and applies no scope rule, likescanMemories. A derived plugin that re-reads members it already tracks calls it, instead of keeping a second copy of memory text and vectors that withdrawal,contentFlagand rescoping would also have to clear. Anything but an array of strings rejects with aTypeError. It returns every row at once, embeddings included, so pass a cluster's ids, not a whole walk's. A failed read rejects the whole call, never a partial list. It reads each row as it is when its query runs: rows are not a snapshot of one moment, so a plugin building output from several rows fences its publication against sink invalidations and rechecks eligibility.getLockScope()— thelockScope(node|cluster) the running plugin in this domain applies, read on every call through the published runtime'slockScope(), so the next call after a reload sees the new value. It never falls back to the calling copy's own settings, which a transient-validation load also applies without publishing: with no running plugin in the domain, or one whose runtime has nolockScope(0.1.5 or earlier, warned once), it isundefined. It is for a plugin whose sink-derived state is correct only when lifecycle writes to one memory are serialized across nodes: undernode, writers on two nodes each run their sinks against what they read (Lifecycle). Such a plugin checks it before registering its sink, refusing to start unlesscluster, and again in the sink, where only a 0.1.5 or earlier runtime leaves it undefined (only the running plugin dispatches sinks) and a reload tonodeshows. The sink sees the current setting, not the scopelockRowslocked with, so a write that locked atnodebefore a reload toclusterpasses the check; the every-node rule's quiesce covers it.clusteris this node's setting: it does not prove record locks exist (without them every lifecycle write answers 503, so no sink runs under a weaker lock) or that other nodes match. Apps load concurrently, so a plugin in another app can run before agent-memory publishes and readundefined; one listed after agent-memory in the same app'sconfig.yamlloads after it.registerBootstrapSection({ name, render })returns an unregister function.render({ principal, q, budgetChars, itemOverheadChars, context })(sync or async) returns{ text }, an array of{ title?, content }items in priority order, or nothing. Sections render afterpinnedandneedsReviewhave packed, sobudgetCharsis the budget they left, net of one item's overhead (itemOverheadChars, the request'ssectionItemOverhead): the title + content room for a single item, so a one-item section that fills it fits exactly, and a section returning several items spendsitemOverheadCharsmore for each beyond the first. Sections render concurrently, so each is told the same remainder. Their items pack, in registration order, betweenneedsReviewandrecent, each counted as title + content +itemOverheadCharsand skipped when it does not fit, never truncated;{ text }is one item, and agent-memory measures it. A packed item is an element ofmemoriesthat agent-memory builds as exactly{ title, content, section: <name> }. Clients frame it as stored data like any memory. It carries noid, so clients never watch, cache or deduplicate it as a memory andwatch=truewatches memory ids only; and noauthor,scopesorrevision, so contributed text cannot borrow a principal's attribution. A section that throws, rejects, or returns anything else is omitted and logged at most once a minute per name; bootstrap still answers.namematches^[A-Za-z][A-Za-z0-9_-]{0,63}$, is not a built-in section, and is unique in the domain; otherwise registration throws aTypeError. Bootstrap has no MCP tool.registerSearchAnnotator({ name, annotate })returns an unregister function.annotate({ principal, hits, context })(sync or async) returns aMapfrom memory id to a JSON value, or nothing; each hit whose id the map holds with a value other thanundefinedgains it ashit[name]. It runs for searches withq(REST and MCPrecall), not for listings (theorder=createdwalk feeds Copy between instances, whose import rejects unknown fields) or bootstrap'srelevantsearch.hitsis a frozen copy of the result list holding the result objects themselves, which annotators must not change; agent-memory merges by position in its own list, so an annotator cannot add, remove or reorder hits. One that throws, rejects, or returns something without agetmethod is skipped and logged at most once a minute per name, and the search still answers.namefollows the section rule and is not a field a hit carries (the view's fields andscore), checked by the registering copy: a plugin of a later version that adds a hit field of that name sees it overwritten, so pick a name specific to the plugin. With no annotator registered, results go out untouched.- Contributed output is the contributor's to authorize. Sections and annotators get the resolved principal,
and agent-memory does not filter what they return.
canReadLabelsis the label test only: ownership, withdrawal and the customer-content quarantine (contentFlag; clearance issuperUseror thecustomer:*read wildcard inprincipal.read) are rule 1's other parts, andscanMemoriesandgetMemoriesyield flagged and private rows. Text derived from several memories is shown only to a reader of all of them, checked against the sources' current state: a rescope, flag or withdrawal reaches the contributor through its sink before its derived text is rebuilt, so it must stop serving that text then, not after the rebuild. - Contributors (sections, annotators,
scanMemoriesandgetMemoriescallers) are trusted in-process code, as sinks are: agent-memory contains their failures, a throw or a rejection, but does not defend against the values they return or the objects they are handed. They are awaited with no time bound, so a slow one delays the response. setPrincipalResolver(fn)— see Identity.resolvePrincipal(context)— the caller behind a request context, resolved exactly as the plugin's own routes resolve it (custom resolver, else the Harper user and itsScopeGrantrows,actsForincluded, read on every call). It resolves to a frozen{ id, superUser, source, actsFor, read, write }(actsFornull unless delegating).readandwriteare frozen lists of the effective grant patterns:user:<id>(anduser:<actsFor>) first, then exact labels, then namespace wildcards (customer:*), role and user rows merged. A pattern that grants nothing is not listed: invalid, in theusernamespace, past the 256 cap, or inside a granted wildcard (customer:acmeundercustomer:*).superUser: truereads and writes every label, and its lists do not say so: check the flag. It rejects with aMemoryError(statusCode401,codeprincipal_required) when there is no caller, 400 for an id that cannot formuser:<id>, and with the original error when a grant read or the custom resolver fails. It reads grants in a context of its own, so it neither checks nor clears the caller's pendingauthorizeflag. A principal is a snapshot: a caller that keeps one past its request keeps revoked grants with it.canReadLabels(principal, labels)— rule 1's label test:truewhen any label matches a read pattern (exact, or in a wildcard's namespace; a wildcard never covers bareglobal), or the principal is super_user. Non-string labels, and a missing label list, match nothing. It does not apply the memory-only rules (owner, withdrawn, content flag);canReadapplies those and then delegates here. It accepts only a principalresolvePrincipalreturned: the compiled matchers are an own, non-enumerable symbol property of it (src/scopes.tsmatchersOf), so a JSON, spread, prototype-derived or hand-built copy throws aTypeError. A JSON copy can still derive scopes for another system. The matchers are not part of the contract; in-process code that reaches them through reflection is trusted like any other plugin code.- Across apps. An app may import the package without listing the plugin while agent-memory runs as another
app on the same Harper. Copies of the package meet in a per-worker registry,
process[Symbol.for('@harperfast/agent-memory')](src/runtime.tsdomain), keyed by theresourcesregistry anddatabasesview the copy's ownharperimport carries. It usesprocessbecauseglobalThisdiffers per app under thevmandcompartmentloaders. All live apps on a worker share that pair. A deploy-validation candidate loads into its ownResources, and an app with branched databases has its owndatabases, so neither reaches the live plugin. Within a domain:- Sinks, the principal resolver, bootstrap sections and search annotators live in the domain, so every copy
registers where the plugin dispatches, whatever order the apps load in. The last
setPrincipalResolverfrom any app wins. The section and annotator lists are added on first use (??=), since an older copy may have created the domain object without them. handleApplicationpublishes{ version, resolvePrincipal, canReadLabels, scanMemories, getMemories, lockScope }(the plugin's own functions) onceconfigure()succeeds, and never for anisTransientValidationscope. The exportedresolvePrincipal,scanMemoriesandgetMemoriescall it when present, else run locally;getLockScopehas no local answer and returnsundefined. A runtime withoutscanMemoriesis 0.1.4 or earlier, which never renders sections or runs annotators: registering one beside it warns once per runtime (warnIfOldRuntime). A registration made before any plugin published turns the domain'sruntimeinto an accessor whose setter runs the same check (noteExtensionRegistered), since an older plugin publishes with a plain assignment; reads and writes ofruntimebehave as before. A scan orgetMemoriesbeside an older runtime runs locally.canReadLabelsis answered by the copy that compiled the principal (compiledHere), so a principal resolved before the plugin published keeps working. A different major version warns once and still delegates. The plugin's own routes call their functions directly; onlysrc/index.tsforwards.- A second plugin instance in a domain throws before mounting: one resolver and one sink list cannot serve two
hosts' registrations. Each worker decides independently which instance loaded first. The check,
configure()and the publish run with noawaitbetween them. - The anchor, the domain object,
MemoryRuntimeand the sink, resolver, section and annotator call shapes are a protocol between package versions. They only gain members; an incompatible change needs a new anchor key. - A copy whose
harperhas noresourceswas loaded without an app scope (moduleLoadernativeorcompartment, ordependencyLoader: native). It keeps a private domain, as every copy did before, so two such copies in different apps never meet. Mixed with a scoped plugin, whichever side comes second warns once in its own app's log: the plugin when it publishes after such a registration (publishRuntime), the copy when it registers or callsgetLockScopeafter the plugin published (noteRegistration,runningLockScope). In-app hosting is unaffected under the default loaders and undernativeandcompartment, where the host and the plugin share one module instance.dependencyLoader: nativealready split them before this change; now it warns. - The registry is per worker thread, so an
isolatedagent-memory app is out of reach. A copy from 0.1.3 or earlier keeps its registrations to itself. A registration lasts as long as the worker, even when the app that made it fails to load afterwards, as it already did for an in-app host. Harper restarts workers to replace an app, so a domain never sees a second runtime from a redeploy.
- Sinks, the principal resolver, bootstrap sections and search annotators live in the domain, so every copy
registers where the plugin dispatches, whatever order the apps load in. The last
- Options (
config.yamlunder the plugin's key):urlPath,mcp(defaultfalse),embeddingModel(defaultdefault),embeddingTruncate(defaultfalse),limits.{contentChars, titleChars, embeddingInputChars, embeddingFallbackChars}(default 20000 / 200 / 6000 / 4000; the last two 2–100000; seeMemoryabove), andsupersession.{mode, similarity, judge}(off | suggest | auto, defaultsuggest; 0.82; none;autowithoutjudgeis rejected as malformed), andcustomerContentCheck.{mode, model, patterns}(see Customer content; absent = off). The plugin's table embedder embeds with the model recorded on the row. A changed option is checked for well-formedness before it applies; a malformed change is logged and the previous configuration keeps serving (model compatibility is not checked — see "One vector space"). ChangingurlPathmounts the new path and removes the old one if the plugin still owns it.
POST /Memory/import (super_user only) takes { rows: [...] }, at most 100 rows, in one transaction:
every row commits or none does. Each row: { id, ownerId, title?, content, tags?, evidence?, scopes, status, supersedes?, supersededBy?, createdAt?, validUntil?, withdrawnAt?, withdrawnReason?, contentFlag?, needsReview? }.
- Rows carry the client-chosen
iddirectly (^[A-Za-z0-9_-]{16,64}$; the Flair import client derivesidas the first 32 hex characters ofsha256('flair:' + flairId)). - Owner, links and
createdAtare preserved, and the superset rule is not applied, because the source's history is. Links are format-checked only, so a link may name a row in a later batch. Awithdrawnrow stays withdrawn; any other row's status andsupersededByare derived from the graph as it stands (stored rows plus this batch), and a stored row that a new row supersedes is marked superseded. So whatever the batch order, and however a run is interrupted or replayed, no memory is hidden behind a missing or withdrawn replacement. The client still sends successors first. - A created row whose scopes include
globalgets the customer-content pattern check (Customer content), unless it carries acontentFlag({ kind, since, source, labels? }, allowed only on aglobalrow), which is stored as given: the source's model or names list may have matched what the target's check would not, and dropping the flag would publish a quarantined memory. A carriedneedsReview({ since, cause, sourceId, replacementId? }) is stored with itsneedsReviewSinceand stands over a flag the same batch would raise (first cause stands).nullmeans not carried. Neither applies to an existing row, and neither enters theunchangedcomparison below. - A stored row the batch turns superseded flags its active citers, stored or created in the same batch, except the rows that supersede it (Provenance). Rows created superseded or withdrawn are history, not transitions, and flag nothing.
- A re-run is idempotent by the same identical-fields rule as
PUT: an existing id with matchingownerId, editable fields (title,content,tags,scopes,evidence,validUntil), andsupersedes(pluswithdrawnReasonwhen the import row iswithdrawn) isunchanged; a row whoseidexists with different fields is reported asconflictand is not overwritten, so a native revision made after import survives, and so does a link added since (itssupersedesdiffers). The response is{ results: [{ id, result: created | unchanged | conflict }] }; an invalid row fails the batch with 400 and itsindex.
The mapping is a pure function of the Flair rows and the principal map: a re-run over the same snapshot sends identical rows.
- Owner: Flair
agentId→ Harper username through--principal-map. An unmapped agent's memories are skipped and reported, never guessed. Becauseidis derived from the Flair id alone, remapping an agent after an import reports those rows asconflictrather than overwriting or duplicating them. - Scopes: Flair treats only the literal
privateas owner-only, soprivate→[user:<owner>]and every other value (shared, absent, legacy) →[global], unless a reviewed classification replaces[global](below). - Lifecycle:
archived→withdrawnwithwithdrawnAt=archivedAt. Flair's archive is a restorable basement, mostly filled by age-based hygiene, so the row is kept out of default search, not dropped. A non-withdrawn row issupersededwhen a non-withdrawn imported row supersedes it;supersededBynames the latest such row. A withdrawn row never carriessupersededBy, so withdrawing its replacement later cannot restore it.validTo→validUntil. - Links:
supersedesis rewritten to target ids. A link to a row that is missing from the source, skipped, itself, or on a cycle is dropped and reported. So is a link that breaks the supersession reader rule above: a replacement that is notglobalmust carry every label of the old memory (globalcounts as readable by every imported principal, as the scope mapping already assumes). No sent row references a row outside the import set. - Order: successors before the rows they supersede, so after an interrupted run every committed
supersededrow has its replacement committed, and the CLI reports any imported row whose replacement came backconflict(dependsOnConflict). - Not imported:
ephemeralrows (the only Flair tier withexpiresAt; the target has no expiry), rows outside the target's content or tag limits (skipped.invalid),durability, embeddings (the target re-embeds through@embed), and Flair bookkeeping fields.
Flair mixed Harper lessons with customer investigation write-ups, and global is readable by fleet
workers, whose role reads global only. A customer incident is therefore imported as two memories: the
record, labeled customer:<org> (readable by holders of customer:* or that label), and a sanitized
lesson, labeled global, that cites the record as memory evidence, so superseding or withdrawing the
record flags the lesson (Provenance). The invariant: in this mode a non-private row reaches the target
as global only through an operator-reviewed decision, and every machine failure lands on
customer:unclassified, never on global.
- Two runs:
--classify --report <file>proposes, writes the report, and contacts no target; a run whose classifier errors on every row (a missing CLI or key) writes nothing. The operator edits the report;--decisions <file>imports with exactly what it says and makes no model calls, so a re-run with the same file sends identical rows. - What is classified: every row a plain import would send as
global:active,supersededandwithdrawnalike, because scopes outlive status. Private rows are never sent to the classifier. - Labels: one
customer:<slug>per organization the classifier names. Names go through--org-map(alias → slug); an alias found literally in the row adds its organization even when the model missed it. An unmapped name becomesslugify(name)and is listed for review. A classifier error, an answer outside the schema, or more than 16 distinct labels make the rowcustomer:unclassifiedalone.customer:unclassifiedreplaces the label of a name with no usable slug, or of a generated slug shared by two different names or equal to a mapped slug (either would merge two customers' readers); the row keeps its other labels, because any one label grants read. It is also added when the model finds customer detail but names no organization, even if an alias matched. - Before links: decided scopes enter
mapFlairMemoriesbeforesupersedesis resolved, so the reader rule above judges the labels actually sent: the link from a customer successor to aglobalmemory is dropped asnarrowing. - Binding: each decision carries a fingerprint over every Flair field the mapping reads plus the
resolved owner. A source row that changed since classification (content, tags, archive, expiry, links,
owner) refuses the whole apply, and the decisions must cover exactly the rows that would be classified.
Decided scopes are
["global"]orcustomer:<slug>labels, nothing else. A lesson needs a customer-labeled row, must not name an organization the row lists, and is sent only for a row the mapping leavesactive. - Lessons (
--lessons): drafted only foractive, unexpired, customer-labeled rows; a lesson from a superseded or archived record would publish knowledge its author already replaced. The draft is dropped (and kept in the report with a reason) when it names a source organization or a mapped alias, or when the classifier, run again on the draft, still finds customer detail or fails. A lesson row is{ id: sha256('lesson:' + sourceId)[:32], ownerId: source owner, title, content, scopes: [global], evidence: [{ kind: memory, ref: sourceId }], createdAt: source createdAt, validUntil: source validUntil }and carries no tags, which can name customers. - Sources first: every source batch commits before any lesson is sent, and a lesson is sent only if
its source did not come back
conflictand a super_user read shows itactive(anunchangedresult does not prove that; the import equality check ignores stored status). A source withdrawn in the target between that read and the lesson batch is not caught; the import window already forbids edits. An edited lesson comes backconflictand a removed one stays in the target: re-applying never withdraws what an earlier apply published. - Backends (
scripts/lib/model-backend.mjs, one interface for every backend):claude(default) runsclaude -p --safe-modewith no tools, no MCP servers and no session persistence, the memory on stdin;anthropiccalls the Messages API withANTHROPIC_API_KEYat effortmedium. Both return JSON-schema output that the script validates again, and a failure carries only a code. Admin-managed policy hooks still run under--safe-mode; where one forwards prompts, useanthropic. - Who sees what: owners read their own customer records whatever their role, so a fleet principal
must not own any. A lesson's evidence shows the record's id to every reader; its
needsReviewflag shows only to readers of the record. Rows an earlier plain import sent asglobalcome backconflictand are not relabeled. - Customer data: stdout and stderr carry ids, counts, scope labels and error codes only. The report (titles, organization names, rationales, lesson drafts) is written 0600 and never overwritten.
An operator CLI that moves a deployment's memories to another app running the plugin (a test deployment to its
permanent home), as a super_user of both. The invariant: the target ends with the source's rows and the
source's derived state (status, supersededBy, review and content flags), and no memory becomes readable to a
reader that could not read it on the source, provided both apps map usernames to the same people and grant no
broader reads (labels are copied verbatim; that precondition is documented, not checked).
- Read: the whole source through the created-order cursor with
includeSuperseded,includeExpiredandincludeWithdrawn, refusing a page that does not acknowledgeincludeWithdrawn. Held in memory to order it, up to 256 Mi characters of row JSON, past which it refuses before writing. - Row: the view's import fields (
contentFlagandneedsReviewincluded), never the embedding: the target re-embeds with its own model. Not carried, because import cannot set them:revision,author/lastEditor,confirmed,scopeConfirmed(the target's own pattern check may flag the row again, which fails closed),autoSupersedes(the links themselves are insupersedes),withdrawnBy, pins, watches, andScopeGrantrows (actsForincluded). - Order: successors first (Kahn's order over live rows'
supersedes; withdrawn rows supersede nothing), so no batch ever supersedes a row already stored. Import would turn such a row superseded and flag its active citers, raising reviews the source's owners had already answered. Rows left on a cycle refuse the copy. - Strict target: before writing, every target row must be a source id and match the intended row; a memory of
the target's own, or one that differs, refuses the copy. Import's
unchangedcompares editable fields only and a batch can change the status of a stored row it supersedes, so only a target holding nothing but this copy keeps both from hiding a difference. A target row may carry a content flag the source row lacks (its own check matched), never lack one. - Verify: after the batches, the same comparison over the whole target, then an exact one over the whole source
again (a write or a model's content verdict that landed after the read, up to 30 s after its write, would
otherwise be lost while the copy reported success); exit 0 only when both match. A re-run of an interrupted copy
passes the check and imports the rest; the rows already there come back
unchanged. - One target node: successor-first order relies on each batch seeing the earlier ones, so
--tonames one node; through a balancer over several, a predecessor could reach a node its replacement has not replicated to and be storedactive. - Output: ids, counts, scope labels and codes; credentials come from
AGENT_MEMORY_FROM_AUTHORIZATIONandAGENT_MEMORY_TO_AUTHORIZATION, never argv.
An operator CLI, run as a super_user from the operator's own machine, that turns the write-time
detection into a one-off, reviewed cleanup of history that arrived without its links.
- Report: walks every non-withdrawn memory oldest first through the created-order cursor (superseded
ones too: an existing C → B can still gain B → A), asks the server for each one's candidates (as many
at once as
--concurrency, a 503 retried), and asks a judge to classify each (newer, older) pairsupersedes | refines | unrelatedwith a one-sentence rationale. The walk keeps ids only; bodies are read per judged pair through a 256-entry cache, so a large store does not have to fit in the operator's heap. Judging runs client-side because the target need not have a generative model; candidates come from the server so the threshold and scope rules stay in one place. The report holds ids, titles,createdAt, similarity, verdict, rationale, a content fingerprint per side (sha256(title \n content), 16 hex) andapprove: false. It never holds content. The report path is claimed before the walk (its temp file opened and a hard link tried), so a path that cannot be written fails before any judge call; the report is then published 0600 through that temp file and an exclusive hard link, so a crash leaves no truncated file and an existing report is never replaced. A judge that fails on every pair writes no report. - Apply (
--apply <decisions file>): only pairs withverdict: supersedesandapprove: true, newest replacement first, so when several approved replacements cover one memory its pointer and its citers' flags name the latest. Before each link it re-reads both memories, skips the pair if either fingerprint changed (content the judge never saw), is missing or withdrawn, or is already linked, and sends both revisions it just read (If-Matchandrevisions), re-reading on 412. The whole decisions file is validated before the first write; a refused pair is reported with the server's code and the run continues. Re-running links nothing new. - Judges (
scripts/lib/model-backend.mjs, written for the import classifier too):claude(default) runs the local Claude Code CLI with--safe-mode(no CLAUDE.md, hooks, plugins or MCP servers; managed policy hooks still run), the prompt on stdin, JSON-schema output, and an environment without the scripts' ownAGENT_MEMORY_*andFLAIR_*credentials orANTHROPIC_API_KEY(so it always runs on the operator's CLI login);anthropiccalls the Messages API with structured output and server-side fallbacks, keyed only fromANTHROPIC_API_KEY;stubdecides by a marker in the newer memory (tests). Failures surface as codes, never the prompt or reply. - Output: stdout and stderr carry ids, counts and codes only.
claude-plugin/ (published at the repository root via .claude-plugin/marketplace.json) integrates
agent-memory into Claude Code without adding harness-specific routes to the Harper server:
-
Distribution: versioned by commit, not by a manifest string. Claude Code replaces an installed plugin (
plugins/cache/agent-memory/agent-memory/<version>/) only when the version it computes changes:plugin.json'sversion, else the marketplace entry's, else the marketplace's git commit. Neither manifest carries aversion, so every commit tomainis a plugin release thatclaude plugin updatedelivers, and one version never names two different plugin trees. A fixed string kept every install on the 0.1.0 code through #50 and #57; an independent semver with a CI bump check was rejected because two PRs can bump to the same number and both merge. The cost is that a server-only commit also gives installs a new (identical) copy.test/plugin-manifest.test.jsfails if either manifest gains aversion. Onboarding scripts live inscripts/onboarding/, outside the plugin payload; the Codex installer copies the plugin runtime into the Codex home rather than pointing at Claude's cache, whose replaced versions Claude Code deletes 14 days after an update. -
Remote HTTP MCP (
claude-plugin/.mcp.json,claude-plugin/bin/headers.mjs):.mcp.jsondeclares Harper's native Streamable HTTP endpoint directly ("type": "http","url": "${AGENT_MEMORY_URL}/mcp","headersHelper": "node \"${CLAUDE_PLUGIN_ROOT}/bin/headers.mjs\"") rather than wrapping REST calls in a localstdioproxy.- Trade-off: pointing directly at
/mcpkeeps a single tool schema source of truth on the server (remember,revise,recall,get,withdraw,pin,unpin,confirm,confirm-scope,watch) and authenticates each request with the caller's Harper-user-bound credential (BearerorBasic) read byheadersHelperfromprocess.envor~/.config/agent-memory/env, soAGENT_MEMORY_TOKENis never exported toCLAUDE_ENV_FILEwhere modelBashcommands could read it. In exchange, the host Harper app must enablemcp: true, andAGENT_MEMORY_URLmust be present in the process environment (or Claude Code'ssettings.jsonenvblock) at startup because Claude Code expands${AGENT_MEMORY_URL}in.mcp.jsonbeforeSessionStarthooks run. WhenAGENT_MEMORY_URLis found only in~/.config/agent-memory/env,SessionStartinjects a configuration notice reminding the user to export it in their shell or settings.
- Trade-off: pointing directly at
-
Bootstrap hook (
SessionStart:startup,resume,compact): callsGET /Memory/bootstrap?budget=<chars>&watch=true(injection budget4000characters viaAGENT_MEMORY_BOOTSTRAP_BUDGET, timeout3000ms viaAGENT_MEMORY_BOOTSTRAP_TIMEOUT_MS), and the injection is an in-order prefix of the response within that budget. The hook requestsbudget= its budget less the wrapper and banners, withentryOverheadandsectionItemOverheadset to its own framing, computed once fromformatMemoryRecordfor an entry at stated sizes: a 64-character id (the server's longest; URL-safe, so never escaped), a 32-character author and nolastEditor, 64 characters of scopes, a 6-digit revision, the longest built-in section, a title; and for a section item, a 64-character section name. Every packed entry within those sizes, whose attributes and body need no escaping that lengthens them, fits, so the response is injected whole. The sizes trade fill for coverage: ~310 characters per memory against ~180–220 measured on typical entries, so at the boundary an entry within ~100 characters of the budget is not packed. An entry beyond them (alastEditor, long principal ids, many scopes, attribute values that escaping lengthens, body text that sanitizing lengthens) and a server older than these parameters, which packsbudgetof title + content, both fall back toformatInjectionBlock({ ordered: true }): it stops at the first entry that does not fit rather than skipping ahead to a smaller, lower-priority one, and a lone entry too large to fit is truncated. When the budget cannot hold one entry at those sizes the hook sends no overhead. The offline cache path keeps skipping ahead: the cache was packed for an earlier request, not this one. The hook also keeps the tail it did not inject inwatchedIds, so a later flag on one of those memories is not mistaken for a flag on the caller's own memory. The hook recordsinjectedIds,watchedIds, and per-id{ revision, status, needsReview, author, ownerId, scopes }for all watched and injected memories in atomic per-session state (state.jsonwritten via temp-file +rename), and updates the local read-through cache (bootstrap-cache.json).startupandcompactresetinjectedIdsso bootstrap context is re-established;resumepreservesinjectedIds. -
Recall hook (
UserPromptSubmit):buildRecallQueryinspects at most the first 32000 prompt characters, normalizes Unicode with NFKC, then strips line-delimited backtick/tilde fenced code blocks (including an unclosed trailing fence) and URLs. Inline fence markers and diagnostic underlines remain prose; markers inside a code line do not close a block. It keeps at most 32 Unicode letter/number runs, joins them with spaces and truncates toQUERY_MAX_CHARS(1000characters) without a dangling surrogate. Splitting punctuation prevents code-like tokens from hiding many analyzed terms; 32 leaves headroom under the measured 64-term budget.eval-recall.mjsuses the same builder and gate. Empty sanitized queries skip the network while still processing change notices, and so does a query with no distinctive term (hasDistinctiveTerm: a letter-containing term outside a short conversational/stopword list, at least 4 characters, or 2 in Han, Kana or Hangul). Search fuses ranks only and its vector lane has no distance cutoff, so1, 2, 3oryeswould otherwise inject the nearest, unrelated memory. Code-only/URL-only prompts therefore skip automatic recall, as does a prompt whose first 32000 characters contain no usable prose; shortened text also changes embedding input, so ranking equivalence is not promised. Other search clients keep their original query. The hook callsGET /Memory/?q=<query>&limit=<k>&watch=true(kdefaults to5viaAGENT_MEMORY_RECALL_LIMIT, bounding per-promptMemoryWatchupserts to<= kactive hits; local RocksDBeval-recall.mjsbenchmark withwatch=true:p50 ~ 4-8 ms,p95 ~ 10-18 ms) under a single sharedAbortSignaldeadline (default1500ms viaAGENT_MEMORY_RECALL_TIMEOUT_MS, clamped to<= 2000ms and backed by a3s command timeout inhooks.json). Only active (status === 'active') memories not yet instate.injectedIdsare injected as recalled records withinAGENT_MEMORY_RECALL_BUDGET(default2000characters). A hit carrying asummaryTreeannotation (from@harperfast/summary-tree) getstopic="<title>" topicId="<id>", and the topic's model-generated summary (at most 400 characters) renders once per injection as an unattributed[topic-<nonce> ...]block before its first hit, so the summary is never attributed to a hit's author. The block is placed beside that first hit or not at all in the injection, so it never follows a hit that references it. A topic id over 128 characters is ignored. On timeout, network error, or server error, or whenstate.jsonis corrupt, the hook skips recall and always exits0(hook.mjsinstalls top-leveluncaughtException,unhandledRejection, andEPIPEguards and callsprocess.exit(0)after flushing stdout). Writingstate.jsonorbootstrap-cache.jsonis best-effort in both hooks (persist): a failed write is logged, and the injection the server returned is still printed, at the cost of a possible repeat next turn. SettingAGENT_MEMORY_RECALL=offskips the per-prompt network search when users do not want prompts sent in query strings or to the server's embedding provider, while still draining<session>.events.jsonlto surface pending live change notices. -
Live changes (
/MemoryEventsfollower + in-band revision diffing):- Trade-off vs.
GET /Memory/changes?since=<ts>: combining a per-session/MemoryEventsSSE follower with in-band revision diffing consumes PR #9's existing server surface with zero new server routes or timestamp change-log indexes, deliveringwithdrawn,superseded,revised, andflaggedevents in real time via/MemoryEvents(with in-band recall diffing as a fallback forrevisedandflaggedon active memories) without adding a second per-prompt HTTP round-trip on theUserPromptSubmitlatency path. SessionStartunlinks any prior.stopsentinel, garbage-collects inactive sessions as a unit only when all of their session files (state.json,events.jsonl,follower.lock,.stop) are older thanAGENT_MEMORY_CACHE_MAX_AGE_MS(default24hours; endpoint-directory GC only inspects 16-hex plugin directories containing plugin files and removes empty directories with non-recursivermdirSync), and spawns a detached per-session follower (claude-plugin/lib/follower.mjs,stdio: 'ignore',unref()). The follower acquires<session>.follower.lockwithO_EXCL(taking over locks whose heartbeatmtimeis older than3 * AGENT_MEMORY_HEARTBEAT_MS), verifies lock ownership (pidandstartedAt) before every heartbeat and on exit, connects toGET /MemoryEvents(Accept: text/event-stream) with jittered reconnect, appends incoming{ id, kind, revision, ts }frames to<session>.events.jsonl(O_APPEND), and periodically renewsPOST /Memory/watchfor active watched IDs in batches of<= 100before the 3600 s watch TTL expires (falling back to per-ID watch requests, tracking pruned404/ withdrawn / superseded IDs in memory sostate.jsonremains single-writer from hooks only, and applying exponential backoff on renewal failures).SessionEndwrites<session>.stopso the follower aborts any active stream or backoff timer and exits cleanly.- On
UserPromptSubmit, the hook drains<session>.events.jsonlfromstate.eventsOffsetusing bounded byte-offset reads (openSync/readSync, resetting to offset0if the file was truncated or recreated) and comparesrevision,status, andneedsReviewon memories returned by recall againststate.revisions. When an injected memory (injectedIds) issuperseded,revised,withdrawn, orflagged(needsReview), or when one of the caller's own memories isflagged(which/MemoryEventsdelivers to the owner), the hook emits an attributed[memory-change-<nonce> ...]notice built directly fromstate.revisionsand the event payload, requiring zero extra per-noticeGET /Memory/<id>calls. Notices are marked notified instate.notifiedRevisionsonly after they fit within the turn's character budget; any overflow notices are persisted instate.pendingNoticesfor subsequent turns.
- Trade-off vs.
-
Untrusted-data injection format (
claude-plugin/lib/format.mjs): every injected memory and change notice carriesid,author(author.principalId, pluslastEditorwhen different),scopes,revision, andneedsReview("true"or"false"); a contributed bootstrap section entry (noid) carries onlysection, so derived text never borrows an identity or attribution. All of it sits inside a per-call random nonced wrapper (<agent-memory-data-<nonce> role="stored-data" notice="...">...</agent-memory-data-<nonce>>and[memory-<nonce> ...]...[/memory-<nonce>]). Attribute values escape quotes, brackets, angle brackets, and newlines, and any occurrence of the nonce or tag names in memory bodies is scrubbed so stored content cannot break out of the data block or masquerade as instructions;memoryandtopicdelimiters in bodies are escaped whatever their nonce, closed or not. -
Diagnostics log (
<cacheDir>/diagnostics.jsonl):UserPromptSubmitandSessionStarteach append one line naming their outcome, andbin/headers.mjsone when it has no credential (mcp_headers) (recordedHook→recordDiagnostic), so a recall that injects nothing can be told apart from "no relevant memories":injected,empty,deduped,over_budget,http_<status>,timeout,error(with areason),skipped,disabledandunconfigured. Lines hold only the fields inENTRY_FIELDS(timestamp, the session's state-file key, counts, latency and fixed reason/errno codes), never prompt text, the query, memory ids or content, URLs or credentials. The log sits incacheDir, not an endpoint directory, so a run with no endpoint still logs. Logging is best-effort: a hook run killed at its command timeout, ahook.mjsexit before a handler runs, a full disk, or two writers rotating at once leave no line or lose the rotated file.injectedrecords what the handler returned tohook.mjs; a stdout write that fails after it is not seen. At 256 KiB the file is renamed todiagnostics.jsonl.1; it is openedO_NOFOLLOW | O_NONBLOCKand only appended if it is a regular file.bin/diagnostics.mjssummarizes both files. -
Credentials come only from the environment,
AGENT_MEMORY_ENV_FILE, or$HOME/.config/agent-memory/env(resolveConfig). A setHOMEis never bypassed for the account's passwd home (that is read only whenHOMEis unset): a harness that replacesHOME(dispatch gives each run a private one) does so to keep host directories out of reach, and passes the credential file as a named capability (AGENT_MEMORY_ENV_FILE, plusAGENT_MEMORY_CACHE_DIRto keep the log). When the configuration is incomplete but something was set (a URL, a token orAGENT_MEMORY_ENV_FILE:partiallyConfigured),SessionStartinjects a notice naming what is missing and where it looked (shortened to the reason alone when the budget cannot hold it), instead of returning nothing. -
Read-through offline cache (
bootstrap-cache.json): stores onlypinned(section === 'pinned') and private (user:*) memories from the last successful/Memory/bootstrapsnapshot authoritatively underAGENT_MEMORY_CACHE_DIR(0700dir,0600file, bounded byAGENT_MEMORY_CACHE_MAX_AGE_MS, default24hours), excluding unpinned shared/team-scoped (global,team:*,project:*) bootstrap memories and evicting any memory IDs that later transition towithdrawnorsuperseded. Pinned shared memories remain in the user-only (0600) cache up to24hours so offline bootstrap preserves explicit user pins unless evicted by live events or purged immediately on401/403. Used only when/Memory/bootstrapfails due to network unreachability, timeout, or5xx, prepending a[STALE CACHE ...]notice. -
Routing skill & CLI utilities:
claude-plugin/skills/memory-routing/SKILL.md: routes team-relevant findings, decisions, and shared lessons toagent-memorywhile keeping machine-specific paths, worktree state, and personal workflow preferences in Claude's native file memory (~/.claude/projects/.../memory/), enforcing a strict single-home rule.scripts/import-claude-memory.mjs: migrates selected native Claude memory Markdown files intoagent-memorywith--dry-run, skippinguser-type personal profile files by default (--include-userto opt in), deriving per-principal deterministic ids (cm_<sha256(principalKey:name)[:32]>, matching^[A-Za-z0-9_-]{16,64}$; requires--userorAGENT_MEMORY_USERwhen authenticating with aBearertoken so derived IDs remain stable across token rotation), mapping frontmattertypetotype:<type>tags, preservingWhy/How to applylines incontent, defaulting scope touser:<self>, and converting[[name]]wikilinks to{ kind: 'memory', ref }evidence via a two-passPUT /Memory/<id>write (If-Matchon pass 2 when forward/cyclic links exist).scripts/eval-recall.mjs: evaluates a JSONL dataset of{ prompt, expectedIds }against the recall hook's exact search query (GET /Memory/?q=<query>&limit=<k>&watch=trueby default;--no-watchto opt out), reportingrecall@k, hit rate, timeout rate, skipped-query count, and requested-response latency percentiles (min,p50,p95,max,meanms), excluding skipped queries from the latency distribution.
Standing queries (level 3), summary-tree consolidation of long memories, Reflex lane registration (waits on Reflex #543), attention/feedback signals.