Skip to content

feat(contacts): promote Auto-Scan sightings into the persistent inventory - #18

Merged
clorth0 merged 1 commit into
mainfrom
feat/scan-sightings-to-inventory
Oct 18, 2026
Merged

clorth0 merged 1 commit into
mainfrom
feat/scan-sightings-to-inventory

Conversation

@clorth0

@clorth0 clorth0 commented Oct 18, 2026

Copy link
Copy Markdown
Owner

⚠️ User-visible change: the persistent contact inventory now grows from Auto-Scan too. Previously Auto-Scan findings were scan-only (shown for the scan, then gone); now every deduped Auto-Scan sighting is also upserted into the persistent inventory, so it survives the scan, appears in the Inventory view, and is removed by Clear Inventory like any live find.

Implements Matt's decision on the parked INTENT question #15: Auto-Scan sightings should AUTOMATICALLY enter the persistent inventory in Aetherscope.

Two key spaces (evidence, pre-change)

  • inventory_key — durable: ism:{model}:{ident}, adsb:{hex} (backend/contacts.py, ism_inventory_key/adsb_inventory_key). Persisted as the contacts table PRIMARY KEY (backend/store.py _V2_SCHEMA), byte-identical so stored history is not orphaned. ContactSighting.key always carries this key.
  • scan_key — ephemeral: {model}/{id|hex|channel} for rtl_433, hex alone for ADS-B (scan_ism_key/scan_adsb_key). Lives only as _key on one phase's finding dicts in ScanSightings; never persisted, never crossed to the store.
  • They differ in dedup identity too: scan dedup is phase-lifetime, no TTL (first rtl_433 event wins and counts repeats; latest ADS-B per hex wins), while live inventory writes are gated by the 2 s per-key SightingThrottle. Observation fields come from the one derivation in contacts.py (ism_sighting filters to _ISM_INFO_FIELDS; adsb_sighting builds label/info/position from the aircraft dict). Scan sightings stored nothing before this PR — phases kept findings in memory (scan.py _phase_rtl433/_phase_adsb) and emitted them as phase_completed findings only.

Promotion point + why there

Promotion is at the contacts seam, at phase end, not per raw event and not in scan.py's store (scan.py stays store-free):

  • contacts.ScanSightings.ism_sightings() / .adsb_sightings() derive ContactSightings from the already-deduped phase findings through the existing ism_sighting/adsb_sighting — the shared inventory_key derivation, not a re-implementation. A scan event with a model but no id/channel therefore promotes exactly as a live event would (ism:{model}:), and a sighting lacking a field uses the manual path's own defaults/omissions; nothing is invented.
  • scan.AutoScanner gains an optional on_sighting callback; each rtl_433/ADS-B phase calls it once per deduped sighting after findings are built (the ephemeral scan findings and scan_decoded live display are unchanged).
  • app._record_scan_sighting is the app-side promotion point wired in _start_scan: ISM sightings get the receiver position exactly as _emit_event does, ADS-B sightings get the same enrichment as _emit_adsb, and both go through the same _record_contact → Store.record_contact write path (and the same 2 s SightingThrottle) as a live/manual find. No second writer.

Dedup / update semantics

Follows what manual re-add already does — Store.record_contact upserts: first sighting inserts (count=1, first_seen=last_seen=now); a repeat of the same inventory_key updates label/ident/info, bumps count + last_seen, and keeps the last known position (COALESCE). Manual-add never duplicates, so promotion doesn't either (pinned in test_scan_repeat_sighting_updates_not_duplicates).

One flagged consequence of the chosen seam: within one scan phase, ScanSightings collapses raw repeats, so 3 raw repeats of one device promote as one sighting (inventory count +1 per phase in which it is seen, not per raw transmission). Re-running/re-scanning bumps count and last_seen as above.

Write cost

Phase-end batching by dedup, not batched-again at scan end: Store.record_contact is a primary-key SELECT + INSERT/UPDATE + commit per call — O(1) per distinct contact, never a full-inventory rewrite — so one call per deduped sighting per phase is already the cheap shape, and no new batch/transaction writer was added to store.py. Per-phase (rather than whole-scan) promotion keeps an ISM device seen at 433 MHz in the inventory even if a later phase is stopped. Existing inventory behaviour/guardrails are untouched, and _record_contact's 3 s contacts-emit throttle still bounds inventory broadcasts. clear_inventory (auth-gated socket handler → Store.clear_contacts, a plain DELETE FROM contacts) already covered every persisted row, so promoted records are cleared with the rest — scan sightings were previously excluded only because they were never persisted (pinned in test_clear_inventory_covers_promoted_scan_records).

Geolocation precision (#16)

Promotion writes stored records, so stored precision applies — it must not inherit the broadcast coarsening:

  • ISM: position is the receiver's, attached via the existing _receiver_geotag(), i.e. gps.geolocation() coarsened by the stored gps_precision setting (default full — the operator's choice). _broadcast_contacts still coarsens only the emitted copy to the broadcast precision (~1 km default). Pinned in test_scan_promotion_stores_full_precision_broadcast_stays_coarse: stored 40.123456, -105.654321, broadcast 40.12, -105.65, stored row untouched.
  • ADS-B: position is the aircraft's own lat/lon from the sighting, stored as-is and not coarsened on broadcast either (existing fix(gps): coarse live broadcast precision by default, full is opt-in #16 rule).

Origin visibility (flag for Matt)

Records are indistinguishable by origin. The contacts table and the inventory UI carry no source concept (columns are key/kind/label/ident/info/count/first_seen/last_seen/lat/lon), so per instructions no badge/UI was invented and no source field was added: a scan-promoted contact looks exactly like a live one in the inventory (test_scan_promotion_matches_manual_add_record_shape pins the identical record shape). Flagging here in case you want a source marker as a follow-up.

Tests + mutant

TDD, failing first (new tests red on AutoScanner(on_sighting=…) / app._record_scan_sighting missing), in tests/test_contact_sighting.py, driving the real scan→contacts→app→Store path with the repo's existing FakeDecoder/FakeRecv harness from #12:

  • test_scan_sighting_appears_in_persistent_inventory — ISM + ADS-B scan sightings land in a real temp Store under their inventory_keys.
  • test_scan_repeat_sighting_updates_not_duplicates — repeat semantics above.
  • test_scan_promotion_matches_manual_add_record_shape — scan route and _emit_event route produce the same record shape/fields.
  • test_clear_inventory_covers_promoted_scan_records — the real on_clear_inventory handler removes promoted records.
  • test_scan_promotion_stores_full_precision_broadcast_stays_coarse — fix(gps): coarse live broadcast precision by default, full is opt-in #16 stored-vs-broadcast split.
  • test_scan_phases_consume_module_and_stay_scan_only (updated) — ephemeral scan view unchanged; scan.py still has no store reference.

Mutant demonstrated and restored: deleting both self._promote(…) calls in backend/scan.py reddened 5 tests (11/16 passed: the four inventory/promotion tests + shape parity); restoring returned 16/16.

Self-review (pocock code-review, separately): Standards — one writer, shared key derivation, no new backend module (architecture-names check passes; docs/architecture.md contacts paragraph updated), no frontend change. Spec — promotion, dedup, precision, clear-inventory and scan-view preservation all pinned by the tests above; origin badge deliberately not invented (flagged above).

CI snapshot

Local, CI-way, on 772491f (origin/main: #17 772491f, #1 1add1f9, #16 0777d12, #15 9137ba3, v1.0.1 50f37ae):

  • for f in tests/test_*.py; do uv run python "$f"; done — all 29 files pass (note: 29 test files on this tree, one more than the 28 quoted in the task brief). tests/test_contact_sighting.py: 16/16.
  • python3 scripts/check_html_templates.py frontend/static/aether-*.js — clean.
  • python3 scripts/check_architecture_names.py — "every backend module is named in docs/architecture.md".
  • No JS touched, so no node --check needed.
  • Remote CI: see checks on this PR.

DO NOT MERGE — squash-merge is Matt's.

@clorth0
clorth0 merged commit dacfde0 into main Oct 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant