Fix double constraint creation for tables with EXCLUSION indexes#43
Merged
Conversation
A table with both a PRIMARY KEY (or UNIQUE) index and an EXCLUSION constraint index aborted the clone with: ERROR: index "..._pkey" is already associated with a constraint summary_table_count_indexes_left excluded EXCLUSION constraint indexes from the "indexes remaining" count. Those indexes are still queued and processed by a worker (which marks them done), so countIndexesLeft reached zero twice for the table and the constraint-creation phase ran twice; the second ALTER TABLE ADD CONSTRAINT on the pkey was fatal. It only manifested with --index-jobs 1 — with more workers a different PID skipped the second pass. Count all indexes so countIndexesLeft reaches zero exactly once. As defense-in-depth, skip a constraint in copydb_create_constraints when the summary already records a done_time, and change summary_add_constraint from INSERT OR REPLACE to INSERT OR IGNORE so the second pass cannot wipe it. Ports upstream dimitri/pgcopydb dimitri#965. Adds a unit test (table with PK + EXCLUSION) and an --index-jobs 1 script test that reproduces the abort without the fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A table with both a PRIMARY KEY (or UNIQUE) index and an EXCLUSION constraint index aborts the clone:
summary_table_count_indexes_leftexcluded EXCLUSION constraint indexes from the "indexes remaining" count. Those indexes are still queued and processed by a worker (which marks them done), socountIndexesLeftreached zero twice for the table and the constraint-creation phase ran twice — the secondALTER TABLE ... ADD CONSTRAINTon the pkey is fatal. It only manifests with--index-jobs 1; with more workers a different PID skips the second pass.Fix
summary_table_count_indexes_left: count all indexes socountIndexesLeftreaches zero exactly once.copydb_create_constraintswhen the summary already records adone_time.summary_add_constraint:INSERT OR REPLACE→INSERT OR IGNORE, so the second pass can't wipe thedone_timeset by the first.Ports upstream
dimitri/pgcopydbdimitri#965, adapted to this repo'scopydb_create_constraintsloop (upstream applies the guard in a per-index hook that doesn't exist here).Tests
tests/unit/17-exclusion-with-pkey— a table with both a PK and an EXCLUSION constraint, verifying both land on the target.tests/unit/script/6-exclusion-constraint-index-jobs.sh— clones with--index-jobs 1and--index-jobs 4. This reproduces the abort without the fix (verified: unit suite exits non-zero pre-fix with the exact error above, and passes with the fix).Verified locally:
PGVERSION=18 make tests/unitgreen with the fix; fails pre-fix.