[WIP] feat: add comprehensive data purge for cancelled/deleted entitlements#281
Open
luis5tb wants to merge 1 commit into
Open
[WIP] feat: add comprehensive data purge for cancelled/deleted entitlements#281luis5tb wants to merge 1 commit into
luis5tb wants to merge 1 commit into
Conversation
luis5tb
changed the title
luis5tb
added a commit
to luis5tb/google-lightspeed-agent
that referenced
this pull request
Jun 11, 2026
Address review findings in the data purge feature (PR RHEcosystemAppEng#281): Safety: - Default data_purge_enabled to False (explicit opt-in for destructive ops) - Add ge=1 validation on data_retention_days and data_purge_interval_hours - Skip entitlement deletion when child record cleanup fails (prevents orphans) - Fix delete_by_order_id to skip only actively in-flight records, not already-reported ones (mark_reported_by_ids does not clear reporting_started_at, so the original IS NULL filter would permanently orphan reported records) Correctness: - Fix integration tests to patch get_data_purge_service (the actual call path) instead of a nonexistent _purge_service attribute - Replace deprecated datetime.utcnow() with datetime.now(UTC) across the entire scheduler for consistent timezone-aware datetimes - Add ORDER BY updated_at to get_expired_cancelled to prevent batch starvation from nondeterministic LIMIT Performance: - Parallelize batch purge with asyncio.gather instead of sequential loop - Add pagination loop to purge_expired_data to drain full backlog - Optimize EntitlementRepository.delete() to single DELETE WHERE Cleanup: - Replace PurgeResult.errors list with error_count int (no raw exceptions) - Add dcr_client_deleted field to PurgeResult - Document new config vars in .env.example, docs/configuration.md, docs/marketplace.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
luis5tb
force-pushed
the
worktree-appeng5150
branch
from
460acd2 to
59d409d
Compare
Implement retention-based data removal for cancelled/deleted marketplace entitlements. All deletion is deferred to a periodic scheduler rather than triggered immediately on cancel/delete events, preserving a configurable retention window (DATA_RETENTION_DAYS=90) for billing reconciliation. New components: - DataPurgeService with purge_order_data() and purge_expired_data() - UsageRepository.delete_by_order_id() with in-flight record protection - EntitlementRepository.delete() and get_expired_cancelled() - RedisRateLimiter.delete_keys_for_order() for best-effort key cleanup - Automated purge task in ReportingScheduler (DATA_PURGE_ENABLED, DATA_RETENTION_DAYS=90, DATA_PURGE_INTERVAL_HOURS=24) Safety features: child-first deletion order, semaphore-limited concurrency, max-batches loop guard, and failed order tracking in PurgeResult. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
luis5tb
force-pushed
the
worktree-appeng5150
branch
from
59d409d to
aa7c76b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement retention-based data removal for cancelled/deleted marketplace entitlements. All deletion is deferred to a periodic scheduler rather than triggered immediately on cancel/delete events, preserving a configurable retention window (DATA_RETENTION_DAYS=90) for billing reconciliation and audit.
New components:
Safety features: child-first deletion order, semaphore-limited concurrency (10), max-batches loop guard, and failed order tracking in PurgeResult.