Support recursive stable pagination with max-depth#69
Open
theronic wants to merge 7 commits into
Open
Conversation
…ause fixes 15 REPL-verified bugs across 5 root-cause layers: schema-write pipeline trust, keyword-bounded index scan, cache keying, nil-tolerant ID resolution, and nil-on-failure cursor decoding. The OpenSpec change (proposal/design/specs/tasks) records the fix strategy, including the adversarially-reviewed schema-history-digest cache design and the Datomic behaviors it was verified against. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Schema-write safety (audit 1, 8, 9, 10): write-schema! throws typed
errors on parse failures instead of silently retracting the entire
schema; // and /* */ comments supported; duplicate definitions/relations
and permission/relation name collisions rejected; empty-schema guard
with {:allow-empty-schema? true} escape; arrow targets validated against
ALL subject types (declaration-order independent); parenthesized unions
flatten, paren arrow bases rejected cleanly.
Relation lookup (audit 2): relation-datoms uses a seek-datoms prefix
scan instead of the :a..:z keyword range that hid uppercase, z-prefixed
and namespaced subject types from permission evaluation.
Strict object-ID resolution (audit 4, 5, 11, 12): reads with unknown IDs
return empty results (read-relationships no longer degrades to a global
scan); writes throw :eacl/unknown-object with existence verified by
datom presence; impl/tx-relationship requires {:allow-tempids? true} to
mint tempids; make-client honors the documented :entid->object-id key
and rejects unknown option keys.
Cache freshness (audit 3): permission-path/plan cache keys carry a
schema-history digest derived from the queried db value - programmatic
schema writes, other peers, and d/as-of views all resolve fresh paths
with no writer-side signal; unclassifiable views (filter/since/history)
and all failures degrade to cache misses, never staleness.
Cursor tokens (audit 7, 6-guard): invalid/expired tokens throw
:eacl/invalid-cursor instead of silently restarting at page one; TTL is
opt-in via :cursor-ttl-seconds (default: no expiry); cursors carry a
two-part schema/paths fingerprint that throws :eacl/stale-cursor when
the queried permission's paths changed mid-pagination while ignoring
unrelated schema changes.
API contract (audit 13, 16): write-relationship!/delete-relationship!
implemented (previously AbstractMethodError); typed ex-info errors
replace asserts (unsupported consistency, not-implemented); vacuous
count-resources assert removed.
Housekeeping (audit 14, 15, 16): dead v6 namespaces deleted
(rules*, datalog, base/Relationship - all referenced storage attrs
absent from the v7 schema); parser tests renamed to run under
clj -X:test; README ID-config and quickstart examples corrected with a
Breaking Changes section; test typos fixed.
Tests: 57 tests / 860 assertions from cold (was 35/376), including
pinned Datomic behavior facts for the digest design and seeded
differential property tests (lookup == can? ground truth == paginated
union == count, forward and reverse, both engines).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The schema-history digest (D8) recomputed per fresh db basis, i.e. after
every d/transact: unrelated relationship writes paid an O(schema-history)
scan for schema-change detection. That does not scale under write load.
Replace derivation with a signaled stamp: write-schema! asserts a fresh
:eacl/schema-version squuid on the schema singleton in the same transaction
as any definition change, and the path/plan caches key on
[(.id db) version resource-type permission-name]. Reading the stamp is one
AVET lookup — no history scans — and unrelated transactions leave every
cache key untouched.
Retained from the audit fixes:
- as-of correctness (§3): as-of views read their era's version datom, so
historic bases get their own cache slots — no cross-base slot sharing.
- Cross-peer invalidation: the stamp lives in the db, so a write-schema!
on any peer invalidates all peers (works without the local eviction,
which remains as hygiene + manual recovery hatch).
- Positive view classification: filter/since/history views bypass the
caches entirely; any stamp-read failure degrades to a miss, never a
stale hit.
- Counter-elision safety: a squuid (not a counter) cannot be re-asserted
to an unchanged value by a concurrent writer.
- Cursor fingerprints keep the {:s :p} shape (:s = version string) and
survive restarts since the stamp is stored.
Consciously accepted (by design, per #74): programmatic relation/permission
datom edits (raw d/transact, d/with, excision) no longer invalidate caches
and may serve stale paths until the next write-schema! or manual
evict-permission-paths-cache!. Users must not manage EACL schema outside
the API.
write-schema! installs the :eacl/schema-version attribute just-in-time on
databases created before it existed, and skips the version bump when the
definition delta is empty (no-op rewrites keep caches and cursors hot).
Tests: schema_basis_test.clj rewritten to pin the new contract (unrelated
transacts hit the cache; invalidation works with eviction disabled; as-of
resolves historical paths; raw-edit staleness documented as intended).
Cold clj -X:test: 56 tests, 854 assertions, 0 failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tion (#74) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fix #74: only write-schema! busts the path cache (revert derived digest)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
:max-depthruntime guards with a default of50Verification
eacl.datomic.schema-testeacl.datomic.config-testeacl.datomic.parser_testeacl.datomic.impl.indexed-testeacl.spice-testNotes