Reject make_distributed_plan over a direct read from a text index by groeneai · Pull Request #108818 · ClickHouse/ClickHouse · GitHub
Skip to content

Reject make_distributed_plan over a direct read from a text index#108818

Open
groeneai wants to merge 1 commit into
ClickHouse:masterfrom
groeneai:groeneai/fix-text-index-make-distributed-plan-4792-6322
Open

Reject make_distributed_plan over a direct read from a text index#108818
groeneai wants to merge 1 commit into
ClickHouse:masterfrom
groeneai:groeneai/fix-text-index-make-distributed-plan-4792-6322

Conversation

@groeneai

Copy link
Copy Markdown
Contributor

Related: #108256

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix a LOGICAL_ERROR (Column ... already added for reading) when running a query with make_distributed_plan = 1 over a MergeTree table that does a direct read from a text index (hasToken, hasAnyTokens, hasAllTokens). Such reads are now rejected at planning time instead of aborting the server.

Description

Found by the AST fuzzer: AST fuzzer (amd_debug, targeted), STID 4792-6322.
CI report: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=108806&sha=70025015949d86cc3d68f7f9f0440ae1a0b6bf18&name_0=PR&name_1=AST%20fuzzer%20%28amd_debug%2C%20targeted%29

Logical error: 'Column __text_index_idx_hasToken_2a3fd3c5b6be32586f14537fc9a3946c already added for reading'

at src/Processors/QueryPlan/ReadFromMergeTree.cpp:4554, on the
buildQueryPipeline -> convertToDistributed -> optimizeTreeSecondPass path.

A direct read from a text index adds ephemeral virtual columns
(__text_index_idx_...) to the read and a separate index-read step. A
distributed worker fragment cannot reproduce these: the column is not in the
table and ReadFromDistributedPlanSource cannot materialize it.

buildQueryPipeline runs the second optimization pass once via optimize()
and, for make_distributed_plan, again via convertToDistributed(). The text
index optimization is not idempotent (it appends the virtual column to
all_column_names), so the single-stage re-optimization re-adds the same
column and aborts with the LOGICAL_ERROR above (a server abort in
debug/sanitizer builds). The multi-stage path instead fails at execution with
NOT_FOUND_COLUMN_IN_BLOCK.

The fix rejects the unsupported combination cleanly at planning time in
checkDistributedReadSupported, next to the existing rejections for pinned
block-number boundaries and the part-order virtual columns. This mirrors the
recently merged #108256, which rejects projections for distributed reads for
the same reason. Reproducer:

SET allow_experimental_full_text_index = 1, make_distributed_plan = 1;
CREATE TABLE t (k UInt64, s String, INDEX idx s TYPE text(tokenizer = 'splitByNonAlpha'))
ENGINE = MergeTree ORDER BY k;
INSERT INTO t SELECT number, 'uniform' FROM numbers(258);
SELECT k FROM t PREWHERE hasToken(s, 'uniform') WHERE hasToken(s, 'uniform');

A direct read from a text index adds ephemeral virtual columns
(__text_index_idx_...) to the MergeTree read plus a separate index-read step.
A distributed worker fragment cannot reproduce these: the virtual column is
not in the table and ReadFromDistributedPlanSource cannot materialize it.

buildQueryPipeline runs the second optimization pass once via optimize() and,
for make_distributed_plan, again via convertToDistributed(). The text index
optimization is not idempotent (it appends the virtual column to
all_column_names), so the single-stage re-optimization re-adds the same column
and aborts with the LOGICAL_ERROR "Column ... already added for reading"
(server abort in debug/sanitizer builds). The multi-stage path instead fails
at execution with NOT_FOUND_COLUMN_IN_BLOCK.

Reject the unsupported combination cleanly at planning time in
checkDistributedReadSupported, next to the existing rejections for pinned
block-number boundaries and part-order virtual columns. Found by the AST
fuzzer (STID 4792-6322).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@groeneai

Copy link
Copy Markdown
Contributor Author

@groeneai

Copy link
Copy Markdown
Contributor Author

cc @kssenii @CurtizJ — could you review this? It rejects make_distributed_plan = 1 over a direct read from a text index in checkDistributedReadSupported (next to the existing pinned-block / part-order rejections), the same way #108256 rejects projections. Without it the single-stage path re-runs the non-idempotent text index optimization and aborts with Column ... already added for reading (AST fuzzer, STID 4792-6322); the multi-stage path fails with NOT_FOUND_COLUMN_IN_BLOCK.

@alexey-milovidov alexey-milovidov added the can be tested Allows running workflows for external contributors label Jun 29, 2026
@clickhouse-gh

clickhouse-gh Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Workflow [PR], commit [867d48c]

Summary:

job_name test_name status info comment
AST fuzzer (amd_debug, targeted) FAIL
Logical error: Block structure mismatch in A stream: different columns: (STID: 0993-27f0) FAIL cidb, issue

AI Review

Summary

This PR rejects make_distributed_plan when a ReadFromMergeTree step has direct text-index read tasks, moving an existing late failure into planning. The guard runs after direct text-index optimization creates index_read_tasks and before distributed read conversion, and the added stateless test covers the fuzzer shape, WHERE, PREWHERE, and a non-distributed control. I found no blocking correctness issues.

Final Verdict

Status: ✅ Approve

No required actions.

@clickhouse-gh clickhouse-gh Bot added the pr-bugfix Pull request with bugfix, not backported by default label Jun 29, 2026
@clickhouse-gh

clickhouse-gh Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

LLVM Coverage Report

Metric Baseline Current Δ
Lines 85.40% 85.40% +0.00%
Functions 92.60% 92.60% +0.00%
Branches 77.60% 77.60% +0.00%

Changed lines: Changed C/C++ lines covered by tests: 9/9 (100.00%) | Lost baseline coverage: none · Uncovered code

Full report · Diff report

@groeneai

Copy link
Copy Markdown
Contributor Author

CI finish ledger — 867d48c

Every failure below has an owner: a fixing PR (ours or external), or a full-effort fix task whose fixing-PR link will be posted here when it opens. Only CH Inc sync is exempt.

Check / test Reason Owner / fixing PR
AST fuzzer (amd_debug, targeted) / Block structure mismatch in UnionStep stream (STID 0993-27f0) chronic trunk fuzzer bug (const vs non-const __table1.year in UnionStep; 14 master hits + 115 PRs / 30d; unrelated to this PR's makeDistributed/text-index diff) #107719 (ours, open)
CH Inc sync - CH Inc sync (private, not actionable)

This PR's own fix (the make_distributed_plan + text-index direct-read LOGICAL_ERROR, STID 4792-6322) and its regression test 04417 are green. The lone red check is the pre-existing chronic UnionStep block-structure-mismatch (STID 0993-27f0), owned by open fixing PR #107719.

Session id: cron:our-pr-ci-monitor:20260629-183000

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

can be tested Allows running workflows for external contributors pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants