Open
Conversation
Management command to move deployments and all related data between projects. Handles all direct and indirect relationships including Events, SourceImages, Occurrences, Jobs, Detections, Classifications, Identifications, SourceImageCollections (with mixed-collection splitting), pipeline configs, processing services, and taxa M2M links. Features: - Dry-run mode by default, --execute to commit - Per-deployment before/after snapshots with row counts - Conservation checks (source + target = original) - FK integrity and indirect access validation - Shared resource handling (clone vs reassign devices, sites, S3 sources) - Raw SQL for ProcessingService M2M to avoid ORM column mismatch Co-Authored-By: Claude <noreply@anthropic.com>
Documents the full relationship map, edge cases, and validation checklist for moving deployments between projects. Co-Authored-By: Claude <noreply@anthropic.com>
The reassign_deployments command now also recalculates: - Event cached counts (captures_count, detections_count, occurrences_count) - Both source and target project related calculated fields Co-Authored-By: Claude <noreply@anthropic.com>
- Renamed from reassign_deployments to move_project_data to better communicate the scope of the operation (all occurrences, identifications, classifications, etc. — not just deployment records) - Added conditional scope warning that shows full data weight when processed data exists, or a simple note for unprocessed transfers - Added identifier membership: users who made identifications on moved data are auto-added to the target project with Identifier role - Added default filter config copy (score threshold, include/exclude taxa, default processing pipeline) Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
- Identifier users now get their existing source project role preserved (e.g. ProjectManager stays ProjectManager), falling back to Identifier role only for users who aren't source project members - TaxaLists linked to source project are now also linked to target project (M2M add, not move — TaxaLists can be shared) - Dry-run output shows TaxaLists and role assignments that will be made Co-Authored-By: Claude <noreply@anthropic.com>
✅ Deploy Preview for antenna-ssec ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for antenna-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 35 minutes and 47 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughAdds a new Django management command Changes
Sequence Diagram(s)sequenceDiagram
participant CLI as CLI
participant CMD as move_project_data\n(Command)
participant DB as Database
participant S3 as S3 (physical data)
participant PROC as ProcessingServices
CLI->>CMD: invoke (args: source, target/create, deployment_ids, flags)
CMD->>DB: validate source, deployments, collect snapshots
CMD->>DB: begin transaction
CMD->>DB: create/select target project
CMD->>DB: clone/reassign shared resources (devices, sites, storage)
CMD->>DB: update deployments.project_id and related FKs (events, images, occurrences, jobs)
CMD->>DB: handle collections (split/clone/remove images)
CMD->>DB: clone project pipeline configs & link processing services
CMD->>DB: add taxa & identifier memberships to target
CMD->>DB: commit transaction
DB-->>CMD: transaction committed
CMD->>DB: recompute cached fields, post-move validation
CMD->>CLI: report results (pass/fail, diffs, errors)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an operator-facing management command to move one or more deployments (and their dependent data) from a source project to a target/new project, plus a written guide documenting the relationship map and validation steps for doing so safely.
Changes:
- Added
move_project_dataDjango management command implementing deployment reassignment with dry-run analysis, execution, and post-move validation. - Added planning/ops documentation describing the full relationship map, edge cases (shared resources, mixed collections), and a validation checklist.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 10 comments.
| File | Description |
|---|---|
| docs/claude/planning/deployment-reassignment-guide.md | New guide documenting deployment reassignment relationships, strategies, and validation checklist. |
| ami/main/management/commands/move_project_data.py | New management command to analyze and execute moving deployments and related data between projects. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Design spec for making projects portable between Antenna instances. Covers UUID fields, Organization model, natural keys, export/import commands, and Darwin Core integration. Includes 21 research areas spanning internal data validation, biodiversity standards (GBIF, iNaturalist, BOLD, Camtrap DP), and patterns from non-biodiversity applications (GitLab, WordPress, Notion, Jira). Status: draft — pending research and validation. Co-Authored-By: Claude <noreply@anthropic.com>
Documents concrete use cases for each model's UUID beyond export/import: Darwin Core field mappings (occurrenceID, eventID, etc.), ML pipeline coordination (Pipeline/Algorithm slug collision risk), device tracking across institutions, and scientific reproducibility requirements. Co-Authored-By: Claude <noreply@anthropic.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@ami/main/management/commands/move_project_data.py`:
- Line 344: The code is logging user.email via self.log(...) and sending it to
logger.info, which exposes PII; update the occurrences that reference user.email
(the self.log(...) call and the logger.info usage) to avoid storing emails by
either logging a non-PII identifier (e.g., user.id or user.pk), a masked email
(e.g., mask local part), or a stable hash of the email, while preserving
role_to_assign.display_name and role_source in the message; ensure both the
self.log(...) invocation and the corresponding logger.info(...) call are changed
consistently to use the non-PII value and not user.email.
- Line 333: These lines use f-strings with no interpolation (triggering F541)
and likely loop-captured variables in inner scopes (B007); replace f-string
literals like self.log(f"\n Identifiers (users with identifications on moved
data):") with plain string literals (self.log("\n Identifiers (users with
identifications on moved data):")) and similarly update the other occurrences at
the reported locations (lines showing self.log(f"...")); search for all
self.log(f"...") instances in move_project_data.py and convert those with no
placeholders to normal strings, and where B007 arises, ensure any loop variables
used inside nested functions/comprehensions are passed as defaults or copied
into a new local variable before inner usage.
- Around line 511-515: The cloned SourceImageCollection created in
SourceImageCollection.objects.create currently only copies name, project_id and
description which drops sampling metadata; update the creation to also copy the
sampling fields from the original collection by including method and kwargs
(e.g., set method=coll.method and kwargs=coll.kwargs) so the new_coll preserves
the original collection semantics when moved.
- Around line 175-187: The command currently prefers create_project when both
--create-project and --target-project are passed; update the options handling in
move_project_data (around create_project_name, target_project logic) to detect
when both options are provided and raise a CommandError (e.g., "Specify only one
of --target-project or --create-project") instead of silently preferring
create_project; keep the existing branches for the single-option cases
(resolving target_project via Project.objects.get and logging, or setting
target_project=None and create_project_name) unchanged.
- Around line 695-723: The validation wrongly compares all records in the target
project to only the moved deployments; update the *_via_project queries
(dets_via_project, cls_via_project, idents_via_project) to also filter by
deployment_ids so they count only records for the moved deployments in the
target project (e.g. add source_image__deployment_id__in=deployment_ids to
Detection/Classifications filters and
occurrence__deployment_id__in=deployment_ids to Identification filters while
keeping source_image__project_id=target_id / occurrence__project_id=target_id).
In `@docs/claude/planning/deployment-reassignment-guide.md`:
- Around line 36-37: Update the wording so it matches the actual command
behavior: change the Device and Site guidance that currently suggests setting
their FK to NULL to instead state that the command clones or reassigns Device
and Site records (do not describe NULL as an option), and update the
project-creation description to indicate the command only sets the target
project's name and owner (omit mention of description). Ensure the edits touch
the same Device/Site lines and the project creation sentences referenced (the
blocks around the Device/Site rows and the project creation paragraph) so the
doc reflects cloned/reassigned ownership and limited project fields.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: fbde3add-8c7c-4079-bf02-a2aa73a1845a
📒 Files selected for processing (3)
ami/main/management/commands/move_project_data.pydocs/claude/planning/deployment-reassignment-guide.mddocs/claude/planning/project-portability-spec.md
Fixes from Copilot and CodeRabbit review: - Mutual exclusion check for --target-project / --create-project - Pipeline config clone preserves enabled and config fields - Collection clone preserves method and kwargs fields - Validation queries scoped to moved deployments (not entire target) - Validation failure raises CommandError (triggers transaction rollback) - Project creation uses ProjectManager (create_defaults=True) - Bulk taxa linking via target_project.taxa.add(*taxa_ids) - Remove unused Taxon import and f-prefix on plain strings Add 37 automated tests covering: - Basic move, dry run, --create-project - All 6 error handling paths - Shared resource clone-vs-reassign (Device, Site, S3StorageSource) - Collection split/reassign logic - Pipeline config cloning - ProcessingService linking - Identifier role preservation - TaxaList linking, default filter copying - Edge cases (empty deployment, move all, multiple deployments) Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (1)
ami/main/management/commands/move_project_data.py (1)
346-346:⚠️ Potential issue | 🟠 MajorAvoid logging identifier email addresses.
These two lines still emit
user.emailto both command output andlogger.info, which creates unnecessary PII retention for an admin migration. Loguser.pkor a masked address instead.Also applies to: 576-576
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ami/main/management/commands/move_project_data.py` at line 346, The log currently prints full PII via user.email in the self.log call; change it to use a non-PII identifier (e.g., user.pk) or a masked email instead of user.email. Update the occurrences where user.email is used in move_project_data.py (the self.log call that formats f"{user.email}: {role_to_assign.display_name} ({role_source})" and the similar occurrence around line 576) to log user.pk or a masked value while preserving role_to_assign.display_name and role_source.
🧹 Nitpick comments (1)
ami/main/tests/test_move_project_data.py (1)
325-362: Replacemsg=parameter withassertRaisesRegexto actually verify exception messages.The
msg=parameter inassertRaisesis only used when the assertion fails (i.e., no exception is raised). It does not verify the exception message contents. UseassertRaisesRegexinstead to match both the exception type and message text.Example fix
- with self.assertRaises(CommandError, msg="does not exist"): + with self.assertRaisesRegex(CommandError, "does not exist"): _run_command(*self._base_args(), "--target-project", "99999")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@ami/main/tests/test_move_project_data.py` around lines 325 - 362, Replace uses of self.assertRaises(..., msg="...") with self.assertRaisesRegex(CommandError, r"pattern") in the tests listed (test_source_project_not_found, test_target_project_not_found, test_deployment_not_found, test_deployment_wrong_project, test_both_target_and_create, test_neither_target_nor_create) so the exception message is actually verified; for example call self.assertRaisesRegex(CommandError, r"Source project 99999 does not exist") around the _run_command in test_source_project_not_found and use appropriate regexes like r"does not exist", r"not found", r"not in source project", r"not both", and r"Must specify" in the corresponding tests when invoking _run_command.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@ami/main/management/commands/move_project_data.py`:
- Around line 593-603: The current copy step merges taxa and only conditionally
copies the pipeline, allowing stale values to persist; instead clear and replace
the target's defaults: call clear() on
target_project.default_filters_include_taxa and
target_project.default_filters_exclude_taxa, then add all taxa from
source_project.default_filters_include_taxa and
source_project.default_filters_exclude_taxa; always assign
target_project.default_filters_score_threshold =
source_project.default_filters_score_threshold and unconditionally set
target_project.default_processing_pipeline =
source_project.default_processing_pipeline (even if None) before saving the
project in move_project_data.
- Around line 143-166: The deployment ID parsing is brittle: parse
options["deployment_ids"] into a cleaned, deduplicated list (e.g., split on ',',
strip tokens, ignore empty tokens, attempt int() inside a try/except to raise
CommandError on non-integer values) and assign to a new variable like
deployment_ids_normalized (or replace deployment_ids); then use
Deployment.objects.filter(pk__in=deployment_ids_normalized) and compare
deployments.count() against len(deployment_ids_normalized) (not the raw parsed
list) so duplicates/trailing commas don't cause false failures; ensure any
ValueError from int() is converted into a CommandError with a clear message.
- Around line 173-189: After resolving target_project (after the block that sets
target_project from options or when create_project_name is None), add a
validation that rejects using the same project as both source and target: check
options.get("source_project") (or the earlier source_project variable if
present) against the resolved target_project.pk and raise CommandError("Target
project must be different from source project") if they match; ensure this check
runs only when target_project is not None (i.e., when --target-project was
provided).
- Line 411: The validation that raises CommandError must run inside the same
transaction.atomic() so failures roll back; move the validation logic currently
after the transaction.atomic() block (the block starting at the
transaction.atomic() call and ending at that block's closing) into the atomic
block before it exits—specifically relocate the validation checks (the section
that raises CommandError) into the transaction.atomic() scope in
move_project_data.py so any CommandError thrown triggers a rollback; ensure all
subsequent writes and the commit point occur only after validation completes
successfully.
---
Duplicate comments:
In `@ami/main/management/commands/move_project_data.py`:
- Line 346: The log currently prints full PII via user.email in the self.log
call; change it to use a non-PII identifier (e.g., user.pk) or a masked email
instead of user.email. Update the occurrences where user.email is used in
move_project_data.py (the self.log call that formats f"{user.email}:
{role_to_assign.display_name} ({role_source})" and the similar occurrence around
line 576) to log user.pk or a masked value while preserving
role_to_assign.display_name and role_source.
---
Nitpick comments:
In `@ami/main/tests/test_move_project_data.py`:
- Around line 325-362: Replace uses of self.assertRaises(..., msg="...") with
self.assertRaisesRegex(CommandError, r"pattern") in the tests listed
(test_source_project_not_found, test_target_project_not_found,
test_deployment_not_found, test_deployment_wrong_project,
test_both_target_and_create, test_neither_target_nor_create) so the exception
message is actually verified; for example call
self.assertRaisesRegex(CommandError, r"Source project 99999 does not exist")
around the _run_command in test_source_project_not_found and use appropriate
regexes like r"does not exist", r"not found", r"not in source project", r"not
both", and r"Must specify" in the corresponding tests when invoking
_run_command.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 21aec0e3-2104-4d9f-abb2-c75687561306
📒 Files selected for processing (3)
ami/main/management/commands/move_project_data.pyami/main/tests/__init__.pyami/main/tests/test_move_project_data.py
- Move all validation checks inside transaction.atomic() so failures trigger a full rollback instead of leaving partially-moved data - Add source==target project guard (prevents self-move corruption) - Deduplicate --deployment-ids at parse time - Remove PII (email addresses) from log output, use user IDs instead - Update deployment-reassignment-guide to match shared-vs-exclusive logic - Add tests for source==target and duplicate deployment IDs (39 total) Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Management command to move deployments and all associated data from one project to another. Motivated by a request to move northern Quebec stations (Umiujaq, Kuujjuaq, Kangiqsujuaq, Inukjuak) from "Insectarium de Montreal" into a new "Nunavik" project.
This is a comprehensive data transfer that handles 16 categories of related data:
Usage
```bash
Dry run (default) — shows full before/after analysis
python manage.py move_project_data --source-project 23 --create-project "Nunavik" --deployment-ids 84,163,284
Execute
python manage.py move_project_data --source-project 23 --create-project "Nunavik" --deployment-ids 84,163,284 --execute
```
Features
Fixes from review
--target-project/--create-projectenabledandconfigfieldsmethodandkwargsfieldsCommandError(rolls back transaction)Project.objects.create(create_defaults=True)via ProjectManagertarget_project.taxa.add(*taxa_ids)Taxonimport and f-prefix on strings without placeholdersTesting & validation
Automated tests (37 tests)
Test file:
ami/main/tests/test_move_project_data.py```bash
docker compose -f docker-compose.ci.yml run --rm django python manage.py test ami.main.tests.test_move_project_data --keepdb -v2
```
TestMoveToExistingProjectTestDryRun--executeTestCreateProject--create-projectflag, member copyingTestErrorHandlingTestSharedResourceCloningTestCollectionHandling--no-clone-collectionsTestPipelineConfigCloning--no-clone-pipelinesTestProcessingServiceLinkingTestIdentifierRolePreservationTestTaxaListLinkingTestDefaultFilterCopyingTestEdgeCasesManual validation (pre-production)
Before running on production data, validate on a staging DB or local backup:
Dry run first — review the full output, check per-deployment counts match expectations
```bash
python manage.py move_project_data --source-project 23 --create-project "Nunavik" --deployment-ids 84,163,284
```
Execute — review the 17-check validation output at the end
```bash
python manage.py move_project_data --source-project 23 --create-project "Nunavik" --deployment-ids 84,163,284 --execute
```
Post-move spot checks in Django admin or shell:
If validation fails, the transaction is rolled back automatically and a
CommandErroris raised with details. No manual cleanup needed.Remaining nits
[12/12]to[13/16]— cosmetic only, reflects that steps 13-16 (identifiers, TaxaLists, default filters) were added after the original 12-step planTest plan
Summary by CodeRabbit
New Features
Documentation
Tests