Allow non matching schemas on EXPORT PARTITION#2111
Open
k-morozov wants to merge 6 commits into
Open
Conversation
k-morozov
force-pushed
the
feature/antalya/26.3/allow_non_matching_schema_export_partition
branch
from
July 24, 2026 07:07
0d6a23a to
267e79f
Compare
Signed-off-by: Konstantin Morozov <just.morozov.k@gmail.com>
Signed-off-by: Konstantin Morozov <just.morozov.k@gmail.com>
Signed-off-by: Konstantin Morozov <just.morozov.k@gmail.com>
Signed-off-by: Konstantin Morozov <just.morozov.k@gmail.com>
k-morozov
marked this pull request as ready for review
July 24, 2026 14:34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes: #1717
Problem.
EXPORT PART/EXPORT PARTITIONmatched source and destination columns positionally, exactly likeINSERT INTO dest SELECT * FROMsrc, and required the column count to match exactly. This is unnecessarily strict for a commonreal scenario: a MergeTree table's schema grows over time (new trailing columns added), but older partitions still need to be exported to an external table (Iceberg/object storage) whose schema was fixed when it was created. Such exports were rejected outright with
NUMBER_OF_COLUMNS_DOESNT_MATCH, with no way to just drop the newer columns and export the rest.Solution. Added the export_merge_tree_part_schema_mismatch_mode setting (default strict, preserving the old behavior). The new ignore_extra_source_columns_by_position mode allows the source to have more columns than the destination: the extra trailing source columns (by declared/readable position, the same order
INSERT SELECTuses) are dropped, both at synchronous validation time (verifyExportSchemaCastable, so a real mismatch still fails immediately atALTER TABLE ... EXPORTtime) and at actual data-export time (ExportPartTask::addExportConvertingActions, via a projection step before the existing positional CAST). The reverse direction — destination has more columns than source — is still always rejected in both modes.Known follow-up. Ignored columns are still fully read and decompressed from disk before being discarded in the projection step, so no I/O is saved. Skipping the read up front isn't safe as a naive truncation: a kept
ALIAS/MATERIALIZED column can legally forward-reference an ignored one, so it would need a real dependency-graph walk instead. Left as a possible future optimization
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Added the export_merge_tree_part_schema_mismatch_mode setting. With ignore_extra_source_columns_by_position (default: strict), EXPORT PART/EXPORT PARTITION allows a source table with extra trailing columns — they are simply
ignored instead of failing with NUMBER_OF_COLUMNS_DOESNT_MATCH.
Documentation entry for user-facing changes
...
CI/CD Options
Exclude tests:
Regression jobs to run: