Skip to content

Fix replace partition error from version >26.6#680

Open
koletzilla wants to merge 3 commits into
mainfrom
ClickHouse/fix-replace-issue
Open

Fix replace partition error from version >26.6#680
koletzilla wants to merge 3 commits into
mainfrom
ClickHouse/fix-replace-issue

Conversation

@koletzilla

@koletzilla koletzilla commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix the insert_overwrite incremental strategy failing on ClickHouse 26.6+ with Code: 36 ... refusing REPLACE PARTITION because it would silently drop the destination partition's data. The strategy intentionally replaces partitions from a source table that may have no parts for a partition (to clear shards that received no new data on distributed tables), which newer servers reject by default. The REPLACE PARTITION statement now includes allow_replace_partition_from_empty_source=1 on servers 26.6 and newer; older servers keep their existing behavior.

I went with the simplest solution: Just add the setting for this specific use case as we explicitly want this one to behave this way. Not touching other queries in case we want to have a different behaviour for them.

Avoided other possible implementations with the focus to keep it simple:

  • Checking if the setting is available by checking existing settings: Just checking by version is enough.
  • Adding it to the list of current settings applied: Many required changes to do that for just this one.

@koletzilla koletzilla marked this pull request as ready for review July 7, 2026 13:41
@koletzilla koletzilla requested a review from Copilot July 7, 2026 13:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the insert_overwrite incremental strategy for ClickHouse 26.6+ by conditionally enabling a new server setting required to allow REPLACE PARTITION operations when the source partition is empty on some shards (common in distributed scenarios).

Changes:

  • Add SETTINGS allow_replace_partition_from_empty_source = 1 to the REPLACE PARTITION ALTER TABLE statement when running on ClickHouse >= 26.6.
  • Document the fix in the changelog under “Unreleased / Bug Fixes”.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
dbt/include/clickhouse/macros/materializations/incremental/incremental.sql Adds a ClickHouse-version-gated SETTINGS allow_replace_partition_from_empty_source = 1 clause to the REPLACE PARTITION statement used by insert_overwrite.
CHANGELOG.md Adds an Unreleased changelog entry describing the ClickHouse 26.6+ fix for insert_overwrite.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@koletzilla koletzilla requested a review from BentsiLeviav July 7, 2026 13:48
@koletzilla

Copy link
Copy Markdown
Contributor Author

test_cloud error is another flaky test. Solving it here: #681

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants