iframe-proxy

alexey-milovidov · 2026-02-20T15:49:24Z

Linked issues

Summary

IColumn::convertToFullIfNeeded now recursively converts inner columns of compound types (ColumnTuple, ColumnArray, ColumnNullable, ColumnMap, etc.) using forEachSubcolumn
Previously only the outermost column was converted (const/replicated/sparse/LC), leaving inner sparse columns intact
This caused assertion failures when Set::appendSetElements or MergeTreeIndexAggregatorSet::update processed batches from different MergeTree parts with different sparse serialization profiles for inner tuple elements

CI report: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=97431&sha=6fe0e8c44dd5c6a12583390b4f8fc45e226a3cb4&name_0=PR&name_1=Stateless%20tests%20%28arm_asan%2C%20azure%2C%20parallel%29
#97431

Changelog category (leave one):

Critical Bug Fix (crash, data loss, RBAC) or LOGICAL_ERROR

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix assertion failure (exception in debug/sanitizer builds) in Set and MergeTreeIndexSet when processing columns with inner sparse subcolumns (e.g., Tuple columns from MergeTree parts with different sparse serialization profiles).

🤖 Generated with Claude Code

Version info

Backported to: 26.1.4.21, 25.12.7.12, 25.11.10.15

`IColumn::convertToFullIfNeeded` only converted the outermost column (const, replicated, sparse, low cardinality) but did not recurse into subcolumns of compound types like `ColumnTuple`, `ColumnArray`, `ColumnNullable`, etc. This caused assertion failures in debug/sanitizer builds when `Set::appendSetElements` or `MergeTreeIndexAggregatorSet::update` processed batches from different MergeTree parts that had different sparse serialization profiles for inner tuple elements. The fix uses `forEachSubcolumn` to recursively convert all inner columns, which generically handles all compound column types. Closes #97474 Closes #97335 https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=97431&sha=6fe0e8c44dd5c6a12583390b4f8fc45e226a3cb4&name_0=PR&name_1=Stateless%20tests%20%28arm_asan%2C%20azure%2C%20parallel%29 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

clickhouse-gh · 2026-02-20T15:50:02Z

The test creates a MergeTree table with a Tuple column where different parts have different sparse serialization profiles for inner elements, then uses `IN` with a subquery to trigger `Set::appendSetElements` which previously failed with an assertion because `ColumnTuple` inner columns were not recursively converted from sparse to full. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

alexey-milovidov · 2026-02-20T16:03:52Z

@@ -0,0 +1,27 @@
+-- Tags: no-random-merge-tree-settings


Does not reproduce.

Ok, now it does.

The assertion in `ColumnTuple::doInsertRangeFrom` only fires when the destination is non-sparse (`ColumnVector`) and the source is sparse (`ColumnSparse`), because `ColumnSparse::doInsertRangeFrom` handles both cases. Swap insert order so the non-sparse part (all_1_1_0) is read first into `set_elements`, and the sparse part comes second. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

`Set::appendSetElements` is only called when `fill_set_elements` is true, which happens when `KeyCondition` calls `buildOrderedSetInplace` for index evaluation. The IN column must be part of the ORDER BY key for this to happen. Changed `ORDER BY key` to `ORDER BY val` so that the tuple column is in the primary key and `KeyCondition` triggers the `buildOrderedSetInplace` path. Verified: aborts on unpatched debug build (exit code 134), passes on patched. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…Needed-recursive

…mpound column types

…pound column types

Backport #97493 to 25.12: Make convertToFullIfNeeded recursive for compound column types

Backport #97493 to 25.11: Make convertToFullIfNeeded recursive for compound column types

The `concat` function's `executeFormatImpl` called `convertToFullIfNeeded` which, after it was made recursive in #97493, strips `LowCardinality` from inside compound column types like `Variant`. However, the argument type is not updated to match, so `SerializationLowCardinality` tries to cast a non-LC column, triggering a LOGICAL_ERROR exception in debug builds. Fix by using only top-level column conversions (Const, Sparse, LowCardinality) instead of the recursive `convertToFullIfNeeded`, keeping the column structure consistent with the type's serialization. https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=97581&sha=320e7c9d8876b04a971bd26214b9ac0ab433c250&name_0=PR&name_1=AST%20fuzzer%20%28amd_debug%29 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…ke `Tuple` `convertToFullIfNeeded` (made recursive in ClickHouse#97493) strips `LowCardinality` from inner subcolumns of compound types (Tuple, Array, etc.) in `Set::insertFromColumns`, but `set_elements_types` in `Set::setHeader` only stripped top-level `LowCardinality`. This created a column/type mismatch: `ColumnVector<Int8>` in the column vs `LowCardinality(Int8)` in the type. When `KeyCondition::tryPrepareSetColumnsForIndex` unpacked the tuple to build index conditions for `IN` subqueries, `castColumn` tried to interpret the plain column as `ColumnLowCardinality`, causing a LOGICAL_ERROR exception in debug builds. Fix by using `recursiveRemoveLowCardinality` for `set_elements_types` in both `Set::setHeader` and the static `Set::getElementTypes`, matching what `convertToFullIfNeeded` does to columns. https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=97831&sha=6f56284303c16ecc5f5a437491c661d5019f2977&name_0=PR&name_1=AST%20fuzzer%20%28amd_debug%29 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Backport #97493 to 26.1: Make convertToFullIfNeeded recursive for compound column types

cho11y · 2026-05-19T12:13:13Z

Backport labels have been automatically added based on https://github.com/ClickHouse/clickhouse-core-incidents/issues/1705

Labels added:

v25.12-must-backport

These labels ensure backport PRs are properly created for each affected release version. The release automation will create backport PRs as needed based on these labels.

cho11y · 2026-05-26T12:33:57Z

clickhouse-gh Bot added pr-critical-bugfix pr-must-backport Pull request should be backported intentionally. Use this label with great care! labels Feb 20, 2026

This was referenced Feb 20, 2026

[CI crash] Type mismatch during set insertion in CreatingSetsTransform #97474

Closed

Logical error: '(isConst() || isSparse() || isReplicated()) ? getDataType() == rhs.getDataType() : typeid(*this) == typeid(rhs)' (STID: 2508-47c9) #97335

Closed

alexey-milovidov commented Feb 20, 2026

View reviewed changes

alexey-milovidov marked this pull request as draft February 20, 2026 16:04

alexey-milovidov and others added 2 commits February 20, 2026 17:08

alexey-milovidov marked this pull request as ready for review February 20, 2026 18:15

Avogar self-assigned this Feb 20, 2026

Avogar approved these changes Feb 20, 2026

View reviewed changes

alexey-milovidov added 2 commits February 21, 2026 05:05

Merge remote-tracking branch 'origin/master' into HEAD

4955ad0

Merge remote-tracking branch 'origin/master' into fix-convertToFullIf…

b039c03

…Needed-recursive

alexey-milovidov added this pull request to the merge queue Feb 22, 2026

Merged via the queue into master with commit 4913faf Feb 22, 2026
148 checks passed

alexey-milovidov deleted the fix-convertToFullIfNeeded-recursive branch February 22, 2026 15:39

robot-clickhouse added the pr-synced-to-cloud The PR is synced to the cloud repo label Feb 22, 2026

robot-clickhouse-ci-1 added the pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR label Feb 22, 2026

robot-clickhouse added a commit that referenced this pull request Feb 22, 2026

Backport #97493 to 25.11: Make convertToFullIfNeeded recursive for co…

e640cb6

…mpound column types

This was referenced Feb 22, 2026

Backport #97493 to 25.11: Make convertToFullIfNeeded recursive for compound column types #97627

Merged

Cherry pick #97493 to 25.12: Make convertToFullIfNeeded recursive for compound column types #97628

Merged

robot-ch-test-poll4 mentioned this pull request Feb 22, 2026

Cherry pick #97493 to 26.1: Make convertToFullIfNeeded recursive for compound column types #97630

Merged

robot-clickhouse added a commit that referenced this pull request Feb 22, 2026

Backport #97493 to 26.1: Make convertToFullIfNeeded recursive for com…

23592f0

…pound column types

robot-ch-test-poll4 mentioned this pull request Feb 22, 2026

Backport #97493 to 26.1: Make convertToFullIfNeeded recursive for compound column types #97631

Merged

clickhouse-gh Bot added a commit that referenced this pull request Feb 22, 2026

Merge pull request #97629 from ClickHouse/backport/25.12/97493

4c608c1

Backport #97493 to 25.12: Make convertToFullIfNeeded recursive for compound column types

clickhouse-gh Bot added a commit that referenced this pull request Feb 22, 2026

Merge pull request #97627 from ClickHouse/backport/25.11/97493

3aab2f7

Backport #97493 to 25.11: Make convertToFullIfNeeded recursive for compound column types

alexey-milovidov mentioned this pull request Feb 22, 2026

Fix exception in concat with Variant containing LowCardinality #97654

Merged

3 tasks

robot-clickhouse-ci-1 added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Feb 22, 2026

This was referenced Feb 24, 2026

Logical error: Bad cast from type A to B (STID: 0250-3d48) #97701

Closed

Fix LOGICAL_ERROR exceptions from recursive convertToFullIfNeeded with LowCardinality inside compound types #97831

Merged

azat mentioned this pull request Feb 24, 2026

Bad cast from type DB::ColumnVector<int> to DB::ColumnLowCardinality with enable_join_runtime_filters=1 #97847

Closed

Avogar added a commit that referenced this pull request Feb 24, 2026

Merge pull request #97631 from ClickHouse/backport/26.1/97493

b507e15

Backport #97493 to 26.1: Make convertToFullIfNeeded recursive for compound column types

Algunenano mentioned this pull request Feb 24, 2026

Logical error: Bad cast from type A to B (STID: 4949-6525) #97854

Closed

cho11y Bot added v25.12-must-backport and removed pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR labels May 19, 2026

robot-ch-test-poll2 added pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore labels May 19, 2026

cho11y Bot added v26.2-must-backport and removed pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR labels May 26, 2026

robot-clickhouse-ci-1 added the pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR label May 26, 2026

robot-clickhouse added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label May 26, 2026

Sunbelt Computer Software

PL/B Language Development and Support

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Make convertToFullIfNeeded recursive for compound column types#97493

Make convertToFullIfNeeded recursive for compound column types#97493
alexey-milovidov merged 6 commits into
masterfrom
fix-convertToFullIfNeeded-recursive

alexey-milovidov commented Feb 20, 2026 •

edited by robot-ch-test-poll

Loading

Uh oh!

clickhouse-gh Bot commented Feb 20, 2026 •

edited

Loading

Uh oh!

alexey-milovidov Feb 20, 2026

Uh oh!

alexey-milovidov Feb 20, 2026

Uh oh!

Uh oh!

cho11y Bot commented May 19, 2026

Uh oh!

cho11y Bot commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!

Conversation

alexey-milovidov commented Feb 20, 2026 • edited by robot-ch-test-poll Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linked issues

Summary

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Version info

Uh oh!

clickhouse-gh Bot commented Feb 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexey-milovidov Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

alexey-milovidov Feb 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cho11y Bot commented May 19, 2026

Uh oh!

cho11y Bot commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

alexey-milovidov commented Feb 20, 2026 •

edited by robot-ch-test-poll

Loading

clickhouse-gh Bot commented Feb 20, 2026 •

edited

Loading