{{ message }}
Add debug virtual row assertions to MergingSortedAlgorithm#106565
Merged
Conversation
Contributor
Contributor
LLVM Coverage ReportChanged lines: Changed C/C++ lines covered by tests: 22/43 (51.16%) | Lost baseline coverage: none · Uncovered code |
alexey-milovidov
approved these changes
Jun 6, 2026
This was referenced Jun 7, 2026
groeneai
added a commit
to groeneai/ClickHouse
that referenced
this pull request
Jun 7, 2026
- src/.../MergingSortedAlgorithm.cpp: collapse the 6-line gate explanation next to `if (apply_virtual_row_conversions)` into a one-line invariant note. The full root-cause explanation (Union of read-in-order subtrees, per-source conversions, default-zero fallback in setVirtualRow) lives in the PR description and the original commit message of 28c0151. - tests/.../04323_virtual_row_two_shards_pk_alias.sql: collapse the 11-line header comment into a one-line "Regression test for STID 2651-3359 (debug assertion added in ClickHouse#106565)" - test name and the SET list make the purpose self-evident, full context lives in ClickHouse#106664. Also corrects the outdated test-file PR reference noted by clickhouse-gh inline (ClickHouse#106565 introduced the assertion; the test reproduces the assertion path covered by this PR, not ClickHouse#106654 which is unrelated). No code-behavior changes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 task
grantholly-clickhouse
pushed a commit
to grantholly-clickhouse/ClickHouse
that referenced
this pull request
Jun 8, 2026
The debug assertions added in ClickHouse#106565 fired across many unrelated PRs (STID 2651-3359), including master, with the message `Virtual row boundary violated in MergingSortedAlgorithm ... the virtual row announced Decimal64_'0.000000000' but the source then produced ...`. Trigger query (`tests/queries/0_stateless/04229_distributed_cte_timestamp_rename_103508.sql` under random `read_in_order_use_virtual_row=1`): WITH A AS ( SELECT * FROM dist_t_two_shards WHERE timestamp >= toDateTime64('2023-01-01', 9) AND timestamp < toDateTime64('2025-01-01', 9) ) SELECT timestamp, id FROM A ORDER BY timestamp DESC LIMIT 10 SETTINGS distributed_product_mode = 'allow', optimize_read_in_order = 1, read_in_order_use_virtual_row = 1; The pipeline that hits the assertion is MergingSortedTransform 2 -> 1 BufferChunks * 2 Union ExpressionTransform * 2 -> ReverseTransform -> VirtualRowTransform -> MergeTreeSelect (shard 1) ExpressionTransform * 2 -> ReverseTransform -> VirtualRowTransform -> MergeTreeSelect (shard 2) The outer `MergingSortedTransform` is built by `SortingStep::mergingSorted` from the Union branch in `optimizeReadInOrder.cpp` (the case at line 1613, `sorting->convertToFinishSorting(*max_sort_descr, use_buffering, false);`), which deliberately passes `apply_virtual_row_conversions = false` because no single conversion fits all children of the Union. Each child, however, still emits virtual-row chunks carrying its own per-source conversion in `MergeTreeReadInfo::pk_block`, with the storage's primary-key column names (`timestamp`). Inside `setVirtualRow`, when `apply_virtual_row_conversions` is false the per-source rename is skipped, so `pk_block` keeps the inner name `timestamp` while the outer merge header has the analyzer-renamed `__table1.timestamp`. The lookup `findByName(col.name)` fails and the column falls back to `createColumnConstWithDefaultValue(1)` -- a Decimal64 zero. The debug assertions in `rememberVirtualRowBoundary` / `checkVirtualRowBoundary` then treat that zero as the announced boundary and abort once the source produces its real first row. Outside debug builds the merge still produces correct output because the virtual-row 1-row chunk is consumed via `skip_last_row = true` in `mergeImpl` / `mergeBatchImpl` and never reaches the output, so the only visible damage is the debug abort. Gate `rememberVirtualRowBoundary` and `checkVirtualRowBoundary` on `apply_virtual_row_conversions = true`. When the flag is false the recorded boundary is meaningless, so simply do not compare against the next chunk. The runtime path (`setVirtualRow` + `skip_last_row`) is unchanged. Add `tests/queries/0_stateless/04323_virtual_row_two_shards_pk_alias.sql` covering the two-shard union shape with `read_in_order_use_virtual_row = 1` both ASC and DESC, and with `read_in_order_use_virtual_row_per_block = 1`. The test fails on master with the assertion and passes with this fix. Caused by: ClickHouse#106565 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
Followup for #106429 submitting separately so it's easier to backport prev PR with actual fix