Skip to content

fix: initialize Dolt replica schema before replay - #502

Merged
weicao merged 19 commits into
mainfrom
task99-dolt-init-signed-20260918
Sep 19, 2026
Merged

weicao merged 19 commits into
mainfrom
task99-dolt-init-signed-20260918

Conversation

@weicao

@weicao weicao commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Problem

When a Dolt source database already exists before replica setup, Dolt cannot use MySQL Shell copy-instance, so MyDuck starts replication without a target schema. The first streamed CREATE TABLE test.items then fails with database not found: test.

Change

  • Create only the explicit SOURCE_DSN database with CREATE DATABASE IF NOT EXISTS before START REPLICA.
  • Keep table and row creation on the binlog path and preserve all replication filters.
  • Assert the Dolt test target schema exists before generating source tables.
  • Document the DSN requirement.

This branch is based on the already reviewed signed PR #501 branch, so the PR diff is limited to this fix and is intentionally not a merge of PR #501.

Validation

Verification image: task99-dolt-init-fix-image:20260918-r3, local manifest sha256:83e6b4e0730ffb12d7f56cff3842dbb7a9959066e49530205393651ce44fcbbf, built by overlaying the changed setup scripts on the unchanged official base apecloud/myduckserver@sha256:85ce0d0d867562ebdc932b3fd71a4ddbb2cae81155b9462fdeecec293210bd94.

bash .github/scripts/replication-test.sh dolt passed with initial/incremental rows 2 -> 3, filtered test.skip, STOP/START row 4, same-volume process restart row 5, replica IO/SQL errors 0, and no leftover test resources. Full log SHA-256: a9efb74225e56cd9ac7981811a020df61f288216492eb1264f3426b29dfb4470.

A stubbed setup harness ran the setup twice against SOURCE_DATABASE=test; both emitted the same idempotent schema statement and no query-only schema statement. Log SHA-256: 4f3e56b0d86d5391c8174a89230664dd44f82f2dd11323a2b355a9f2c7d77825.

The official 85ce0d0d failure originals remain unchanged.

Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. Part 1.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File .github/workflows/backup-restore.yml.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File catalog/connpool_test.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File catalog/internal_macro.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File catalog/internal_tables.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File catalog/internal_views.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File catalog/pg_catalog_compat.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File catalog/provider.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File initialdata/file_content.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File pgserver/in_place_handler.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File pgserver/pg_catalog_compat_test.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File pgserver/pgconfig/parameters.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File pgserver/pgconfig/parameters_list.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File pgserver/stmt.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File pgserver/stmt_test.go.
Replays reviewed PR #499 tree for task #99. Original reviewed head 3246583. File pgserver/transaction.go.
Adds the remaining reviewed file for task #99. Original reviewed head 3246583.
Create the explicit SOURCE_DSN database before START REPLICA when snapshot copy is unavailable. Keep tables, rows, and filters on the binlog path.

@leon-ape leon-ape left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review of PR #502 head 78f6ea6f837f544d6eed5a11bdf0e63041c8a6e1.

This is not merge, not release, not a retag. Official image 85ce0d0d PostgreSQL and the unrepaired Dolt CI red stay fail until a new identity is proven. Overlay run ≠ GitHub Dolt job cleared.

GitHub identity — PASS

  • Base branch task99-github-signed-20260918 / parent 5e88e3ec3c1a67802206f2f44ce9f02fddaec85d (PR #501 head).
  • Head tree 7a84ba9813ecaff38354c6024c3b3da2eef89059.
  • REST commit.verification: verified=true, reason=valid, committer web-flow.
  • One commit. Files: .github/scripts/replication-test.sh, devtools/replica-setup-mysql/replica_setup.sh, docs/tutorial/replica-setup-dolt.md. Does not reopen PR #501's 16 files.
  • PR #499 and #501 remain OPEN. No auto-merge.

Diff vs stated boundary — PASS (code)

replica_setup.sh creates CREATE DATABASE IF NOT EXISTS from explicit SOURCE_DATABASE after snapshot skip and before START REPLICA. Skips empty/mysql. Rejects backtick. No CREATE TABLE / INSERT. Filters unchanged. Dolt regression asserts information_schema.schemata has test before create_source_data.

Local unsigned 3e59120b is not this GitHub SHA; the signed candidate is 78f6ea6f.

Originals — HOLD, not PASS

Attachment SHA match:

File sha256
task99-dolt-init-fix-20260918.md e7c4204f9a8527684f2cb1e58a88bb45599968779fc6905589d48f3a40fa5079
task99-dolt-image-fix-r3.log a9efb74225e56cd9ac7981811a020df61f288216492eb1264f3426b29dfb4470
task99-schema-init-stub.log 4f3e56b0d86d5391c8174a89230664dd44f82f2dd11323a2b355a9f2c7d77825

What the Dolt log independently shows: phase dumps initial 2, incremental 3, STOP/START 4, process-restart 5; Last_IO_Errno/Last_SQL_Errno 0; GTID 1-4 → 1-7; ends dolt replication regression passed.

What it does not contain: image 85ce0d0d, overlay digest 83e6b4e0…, CREATE DATABASE, schema-count assertion, test.skip query, docker/volume/network cleanup, numeric exit code.

After process-restart, Replicate_Ignore_Table is empty (was test.skip in earlier phases). Filter persistence is not shown.

Stub log shows two setup runs and the echo Initializing target schema from SOURCE_DATABASE=test.... It does not contain CREATE DATABASE IF NOT EXISTS \test`;`. Cannot independently confirm DSN-only SQL or that INCLUDE_SCHEMAS-only names were not created.

Do not treat the report text as a substitute for those missing original fields.

Out of this review

Do not merge #502 / #501 / #499. Do not retag. Do not write GitHub Dolt CI as cleared. 85ce0d0d PostgreSQL Metabase/Superset failures stay fail.

@leon-ape leon-ape left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent re-review of PR #502 head 78f6ea6f plus r4c originals.

This is a code Approve only. Not merge, not release, not a retag. Overlay run ≠ official-image Dolt CI cleared. 85ce0d0d PostgreSQL Metabase/Superset failures stay fail.

GitHub

Parent 5e88e3ec (PR #501). Tree 7a84ba98. verified=true/reason=valid. Three files. Local run source 3e59120b reports the same tree.

Independent GitHub checks on this PR: test-replication postgres/mysql/mariadb/dolt COMPLETED/SUCCESS, mergeStateStatus=CLEAN. Auto-merge is off.

r4c originals — PASS (previous holds closed)

SHA match:

  • report 70e975ddce1c740afa03af3697a5983e75024cbe1241c3997d77079b9fff40c7
  • redacted trace 0837aebc841bb87b6620b11e52102b247c04bd6b7854d5ef65d7515f3d93b922
  • stub 5e3ec31b0b32aff15e6d880fb1d63e042f4b9f116c795049fc3aa5cbe9058992

Trace independently contains: command, source tree 7a84ba98, script/setup SHA, overlay manifest 83e6b4e0… on base 85ce0d0d, query audit CREATE DATABASE IF NOT EXISTS \test`then filter thenSTART REPLICA, schema_name=test count 1 before source tables, skip count 0, phases 2/3/4/5 with GTID 1-4→1-7, Ignore_Table test.skipuntil SERVER-mode restart (empty, skip rows still 0), docker rm/volume/network then empty postcondition,script_exit_code: 0`.

Stub two runs: stdin is exactly CREATE DATABASE IF NOT EXISTS \test`;`. No query_only CREATE DATABASE.

Out of this review

Do not merge #502/#501/#499. Do not retag. Do not write the unrepaired official Dolt job as cleared.

@weicao
weicao changed the base branch from task99-github-signed-20260918 to main September 19, 2026 04:53
@weicao
weicao merged commit 031eed8 into main Sep 19, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants