Repository navigation
feat(contacts): promote Auto-Scan sightings into the persistent inventory - #18
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements Matt's decision on the parked INTENT question #15: Auto-Scan sightings should AUTOMATICALLY enter the persistent inventory in Aetherscope.
Two key spaces (evidence, pre-change)
inventory_key— durable:ism:{model}:{ident},adsb:{hex}(backend/contacts.py,ism_inventory_key/adsb_inventory_key). Persisted as thecontactstable PRIMARY KEY (backend/store.py_V2_SCHEMA), byte-identical so stored history is not orphaned.ContactSighting.keyalways carries this key.scan_key— ephemeral:{model}/{id|hex|channel}for rtl_433, hex alone for ADS-B (scan_ism_key/scan_adsb_key). Lives only as_keyon one phase's finding dicts inScanSightings; never persisted, never crossed to the store.SightingThrottle. Observation fields come from the one derivation in contacts.py (ism_sightingfilters to_ISM_INFO_FIELDS;adsb_sightingbuilds label/info/position from the aircraft dict). Scan sightings stored nothing before this PR — phases kept findings in memory (scan.py_phase_rtl433/_phase_adsb) and emitted them asphase_completedfindings only.Promotion point + why there
Promotion is at the contacts seam, at phase end, not per raw event and not in scan.py's store (scan.py stays store-free):
contacts.ScanSightings.ism_sightings()/.adsb_sightings()deriveContactSightings from the already-deduped phase findings through the existingism_sighting/adsb_sighting— the sharedinventory_keyderivation, not a re-implementation. A scan event with a model but no id/channel therefore promotes exactly as a live event would (ism:{model}:), and a sighting lacking a field uses the manual path's own defaults/omissions; nothing is invented.scan.AutoScannergains an optionalon_sightingcallback; each rtl_433/ADS-B phase calls it once per deduped sighting after findings are built (the ephemeral scan findings andscan_decodedlive display are unchanged).app._record_scan_sightingis the app-side promotion point wired in_start_scan: ISM sightings get the receiver position exactly as_emit_eventdoes, ADS-B sightings get the same enrichment as_emit_adsb, and both go through the same_record_contact→Store.record_contactwrite path (and the same 2 sSightingThrottle) as a live/manual find. No second writer.Dedup / update semantics
Follows what manual re-add already does —
Store.record_contactupserts: first sighting inserts (count=1,first_seen=last_seen=now); a repeat of the sameinventory_keyupdates label/ident/info, bumpscount+last_seen, and keeps the last known position (COALESCE). Manual-add never duplicates, so promotion doesn't either (pinned intest_scan_repeat_sighting_updates_not_duplicates).One flagged consequence of the chosen seam: within one scan phase,
ScanSightingscollapses raw repeats, so 3 raw repeats of one device promote as one sighting (inventorycount+1 per phase in which it is seen, not per raw transmission). Re-running/re-scanning bumps count andlast_seenas above.Write cost
Phase-end batching by dedup, not batched-again at scan end:
Store.record_contactis a primary-key SELECT + INSERT/UPDATE + commit per call — O(1) per distinct contact, never a full-inventory rewrite — so one call per deduped sighting per phase is already the cheap shape, and no new batch/transaction writer was added tostore.py. Per-phase (rather than whole-scan) promotion keeps an ISM device seen at 433 MHz in the inventory even if a later phase is stopped. Existing inventory behaviour/guardrails are untouched, and_record_contact's 3 s contacts-emit throttle still bounds inventory broadcasts.clear_inventory(auth-gated socket handler →Store.clear_contacts, a plainDELETE FROM contacts) already covered every persisted row, so promoted records are cleared with the rest — scan sightings were previously excluded only because they were never persisted (pinned intest_clear_inventory_covers_promoted_scan_records).Geolocation precision (#16)
Promotion writes stored records, so stored precision applies — it must not inherit the broadcast coarsening:
_receiver_geotag(), i.e.gps.geolocation()coarsened by the storedgps_precisionsetting (default full — the operator's choice)._broadcast_contactsstill coarsens only the emitted copy to the broadcast precision (~1 km default). Pinned intest_scan_promotion_stores_full_precision_broadcast_stays_coarse: stored40.123456, -105.654321, broadcast40.12, -105.65, stored row untouched.Origin visibility (flag for Matt)
Records are indistinguishable by origin. The
contactstable and the inventory UI carry nosourceconcept (columns are key/kind/label/ident/info/count/first_seen/last_seen/lat/lon), so per instructions no badge/UI was invented and no source field was added: a scan-promoted contact looks exactly like a live one in the inventory (test_scan_promotion_matches_manual_add_record_shapepins the identical record shape). Flagging here in case you want a source marker as a follow-up.Tests + mutant
TDD, failing first (new tests red on
AutoScanner(on_sighting=…)/app._record_scan_sightingmissing), intests/test_contact_sighting.py, driving the real scan→contacts→app→Storepath with the repo's existing FakeDecoder/FakeRecv harness from #12:test_scan_sighting_appears_in_persistent_inventory— ISM + ADS-B scan sightings land in a real tempStoreunder theirinventory_keys.test_scan_repeat_sighting_updates_not_duplicates— repeat semantics above.test_scan_promotion_matches_manual_add_record_shape— scan route and_emit_eventroute produce the same record shape/fields.test_clear_inventory_covers_promoted_scan_records— the realon_clear_inventoryhandler removes promoted records.test_scan_promotion_stores_full_precision_broadcast_stays_coarse— fix(gps): coarse live broadcast precision by default, full is opt-in #16 stored-vs-broadcast split.test_scan_phases_consume_module_and_stay_scan_only(updated) — ephemeral scan view unchanged; scan.py still has no store reference.Mutant demonstrated and restored: deleting both
self._promote(…)calls inbackend/scan.pyreddened 5 tests (11/16 passed: the four inventory/promotion tests + shape parity); restoring returned 16/16.Self-review (pocock code-review, separately): Standards — one writer, shared key derivation, no new backend module (architecture-names check passes;
docs/architecture.mdcontacts paragraph updated), no frontend change. Spec — promotion, dedup, precision, clear-inventory and scan-view preservation all pinned by the tests above; origin badge deliberately not invented (flagged above).CI snapshot
Local, CI-way, on
772491f(origin/main: #17772491f, #11add1f9, #160777d12, #159137ba3, v1.0.150f37ae):for f in tests/test_*.py; do uv run python "$f"; done— all 29 files pass (note: 29 test files on this tree, one more than the 28 quoted in the task brief).tests/test_contact_sighting.py: 16/16.python3 scripts/check_html_templates.py frontend/static/aether-*.js— clean.python3 scripts/check_architecture_names.py— "every backend module is named in docs/architecture.md".node --checkneeded.DO NOT MERGE — squash-merge is Matt's.