iframe-proxy

groeneai · 2026-06-10T14:23:00Z

A non-equi JOIN ON predicate over a Nothing-typed column (e.g. from ARRAY JOIN []) aborted the server with a LOGICAL_ERROR:

Logical error: Unexpected expression in JOIN ON section. Expected boolean (UInt8), got 'Nothing'.

toBoolIfNeeded() coerced a non-UInt8 residual condition to boolean by wrapping it in and(x, true). That works for every type except a non-nullable Nothing: and(Nothing, true) stays Nothing (the bottom type), so the residual mixed-join expression had output type Nothing, which HashJoin::validateAdditionalFilterExpression rejects. This fuzzer-found bug exists since v24.5.

Fix: cast a Nothing-typed condition to UInt8 (Nullable(UInt8) when nullable). The column has no values, so the predicate matches no rows. The multi-condition path in concatConditions had the same gap and is now also routed through toBoolIfNeeded.

Reproducer (from the issue):

SELECT 1 FROM t1 tx ARRAY JOIN [] AS a0 LEFT JOIN t1 ON t1.c0 = a0 AND t1.c0 <> a0;

Closes #78761

Changelog category (leave one):

Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fix a LOGICAL_ERROR ("Unexpected expression in JOIN ON section. Expected boolean (UInt8), got 'Nothing'") when a non-equi JOIN ... ON predicate references a Nothing-typed column, such as one produced by ARRAY JOIN [].

Version info

Merged into: 26.7.1.432

A non-equi JOIN ON predicate over a column of type Nothing (for example the column produced by ARRAY JOIN []) aborted the server with Logical error: Unexpected expression in JOIN ON section. Expected boolean (UInt8), got 'Nothing'. toBoolIfNeeded() in JoinStepLogical coerced a non-UInt8 residual condition to boolean by wrapping it in and(x, true). That works for every type except a non-nullable Nothing: and(Nothing, true) stays Nothing because Nothing is the bottom type. The residual mixed-join expression then had output type Nothing, which HashJoin::validateAdditionalFilterExpression rejects. Cast a Nothing-typed condition to UInt8 (Nullable(UInt8) when the input is nullable) instead. The column has no values, so the predicate matches no rows. The multi-condition path in concatConditions had the same gap and is now also routed through toBoolIfNeeded. Closes ClickHouse#78761 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

groeneai · 2026-06-10T14:23:23Z

Pre-PR validation gate

#	Question	Answer
a	Deterministic repro?	Yes, 100% on master (debug). `CREATE TABLE t1 (c0 Int32) ENGINE=MergeTree ORDER BY c0; INSERT INTO t1 VALUES (1),(2),(3); SELECT 1 FROM t1 tx ARRAY JOIN [] AS a0 LEFT JOIN t1 ON t1.c0 = a0 AND t1.c0 <> a0;` aborts every time with the LOGICAL_ERROR.
b	Root cause explained?	`ARRAY JOIN []` yields a column of type `Nothing`. The non-equi residual predicate `t1.c0 <> a0` is `notEquals(Int32, Nothing) -> Nothing`. `toBoolIfNeeded()` coerces non-`UInt8` conditions to boolean via `and(x, true)`, but `and(Nothing, true)` stays `Nothing` (the bottom type). The mixed-join expression's output column is then `Nothing`, and `HashJoin::validateAdditionalFilterExpression` requires `UInt8` -> LOGICAL_ERROR.
c	Fix matches root cause?	Yes. The fix is in `toBoolIfNeeded` (the exact function that emits the `and(..., true)` wrapper seen in the error). A `Nothing`-typed condition is cast to `UInt8` (`Nullable(UInt8)` when nullable) instead of being wrapped in `and`. The multi-condition path in `concatConditions` is now also routed through `toBoolIfNeeded`. No symptom-guarding; the coercion is fixed where it was incomplete.
d	Test intent preserved / new tests added?	New regression test `04327_join_on_nothing_typed_residual` added: single/multi/mixed residual predicates, INNER, and the ConcurrentHashJoin (`max_threads=4`) path from the issue stack, plus a non-empty `ARRAY JOIN [2,3]` case asserting correct results (no behavior change for valid inputs).
e	Both directions demonstrated?	Yes. Rebuilt the original (pre-fix) binary: all variants abort (LOGICAL_ERROR / signal 6), incl. the ConcurrentHashJoin path. Rebuilt with the fix: all exit 0 with correct results. The new test passes via `clickhouse-test` (50 runs; the only failures were unrelated `Invalid time zone: zoneinfo/UTC` infra noise that an unrelated control test reproduces identically).
f	Fix is general, not a narrow patch?	Both residual-filter construction paths (single condition via `toBoolIfNeeded`, multiple conditions via `concatConditions`) are covered. Traced upstream: the bad `Nothing` type originates from the empty-array `ARRAY JOIN` and propagates through `and`; the coercion helper is the correct, single place to handle it. Note: `RIGHT`/`FULL` JOIN with a `Nothing` column hits a separate, pre-existing `NOT_IMPLEMENTED` (`insertDefaultInto` for `Nothing`) that reproduces with equi-only `ON` and is out of scope for this LOGICAL_ERROR fix.

Session id: cron:clickhouse-worker-slot-5:20260610-135400

groeneai · 2026-06-10T14:23:53Z

cc @vdimir @KochetovNicolai — could you review this? It fixes a LOGICAL_ERROR ("Expected boolean (UInt8), got 'Nothing'") in the logical join planner: toBoolIfNeeded coerced a non-equi JOIN ON residual predicate via and(x, true), which leaves a non-nullable Nothing-typed predicate (e.g. from ARRAY JOIN []) still typed Nothing; the fix casts it to a boolean instead. Closes #78761.

clickhouse-gh · 2026-06-10T14:28:00Z

Workflow [PR], commit [5b37fe2]

Summary: ❌

Performance Comparison: Performance dashboard

job_name	test_name	status	info	comment
AST fuzzer (amd_debug, targeted)		FAIL
	Logical error: 'or_argument_nodes.size() > 1' (STID: 2508-3fad)	FAIL	cidb

AI Review

Summary

This PR fixes JOIN ... ON residual predicate coercion for Nothing-typed expressions by casting Nothing / Nullable(Nothing) to UInt8 / Nullable(UInt8) in toBoolIfNeeded, and by routing multi-condition conjunctions through the same helper. I did not find any remaining correctness issues in the current head: the planner change matches HashJoin::validateAdditionalFilterExpression's UInt8 contract, and the regression test now pins enable_analyzer = 1 so it reaches the fixed planner path instead of failing earlier in the old analyzer.

Missing context / blind spots

⚠️ I did not rerun the reproducer locally because this checkout has no build directory or clickhouse binary; confidence here is based on code inspection plus the PR's existing CI and coverage evidence.

Final Verdict

✅ No new findings.

The test creates a local MergeTree, runs deterministic JOINs over an ARRAY JOIN [] Nothing-typed residual predicate, and checks stable row values. There is no parallel-replicas incompatibility: the queries produce identical correct results under parallel replicas (verified locally with cluster_for_parallel_replicas, the JOIN dispatched via ReadFromRemoteParallelReplicas, 15/15 runs across 2-6 replicas and both parallel_replicas_local_plan modes). Remove the unnecessary exclusion so the test gets parallel-replicas coverage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The regression test for ClickHouse#78761 failed on the old-analyzer CI variant (Stateless tests amd_llvm_coverage, old analyzer, ...) with Code: 47. DB::Exception: Missing columns: 'a0' ... (UNKNOWN_IDENTIFIER) The old analyzer (TreeRewriter/ExpressionAnalyzer) cannot resolve an ARRAY JOIN alias used inside a JOIN ON clause, so the query errors out before reaching the planner. The LOGICAL_ERROR this test guards against, and its fix in JoinStepLogical::toBoolIfNeeded, only exist in the new analyzer's planner. Pin enable_analyzer = 1 so the test exercises the fixed code path on every CI variant instead of erroring on the old one. Verified locally on a debug build: with the old analyzer forced (--client-option enable_analyzer=0) the test fails without this line and passes with it; 20/20 runs pass under the previously red configuration. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

groeneai · 2026-06-10T21:18:58Z

Pre-PR validation gate (follow-up fixup `a019401b625d`)

This addresses the only PR-caused CI failure on the prior HEAD: the new regression test 04327_join_on_nothing_typed_residual failed on the Stateless tests (amd_llvm_coverage, old analyzer, s3 storage, DatabaseReplicated, WasmEdge, parallel) variant.

#	Question	Answer
a	Deterministic repro?	Yes. On a debug build, run the test through `clickhouse-test` with the session forced to the old analyzer (`--client-option enable_analyzer=0`): the test fails 100% with `Code: 47. UNKNOWN_IDENTIFIER: Missing columns: 'a0'` — byte-for-byte the CI red. Equivalently, `clickhouse-client --query "SELECT 1 FROM t tx ARRAY JOIN [] AS a0 LEFT JOIN t ON t.c0 = a0 AND t.c0 <> a0 SETTINGS enable_analyzer = 0"` errors on demand.
b	Root cause explained?	The old analyzer (`TreeRewriter`/`ExpressionAnalyzer`) cannot resolve an `ARRAY JOIN` alias (`a0`) referenced inside a `JOIN ON` clause, so the query aborts in `collectUsedColumns` before the planner ever runs. The `LOGICAL_ERROR` this test guards against, and its fix in `JoinStepLogical::toBoolIfNeeded`, exist only in the new analyzer's planner. So on the old-analyzer CI variant the test query errored at name-resolution time rather than exercising the fix.
c	Fix matches root cause?	Yes. The fix pins `SET enable_analyzer = 1;` at the top of the test, so it always runs on the analyzer that contains the bug and the fix, on every CI variant — including the old-analyzer job. This is the dominant idiom in the suite (~700 stateless tests use it). No production code changed for this; the C++ fix is unchanged.
d	Test intent preserved / new tests added?	Preserved. The same six query shapes (single/multi/mixed residual, INNER, ConcurrentHashJoin, and the non-empty-array correctness check) still run and still assert the same results. Pinning the analyzer does not weaken what the test checks — it ensures the test reaches the guarded code path.
e	Both directions demonstrated?	Yes, through the actual `clickhouse-test` runner under the previously-red `enable_analyzer=0` variant: pre-fix (no `SET`) → `[FAIL]` with `Code: 47 UNKNOWN_IDENTIFIER: Missing columns: 'a0'`; with-fix → `[OK]`, 20/20 runs pass. The only failures under full randomization were `Code: 36 Invalid time zone: zoneinfo/UTC`, proven environmental (an unrelated control test `00001_select_1` reproduces the identical tz error at the same rate locally); zero JOIN/`Nothing`/`UInt8` errors and zero output diffs.
f	Fix is general, not a narrow patch?	N/A for this follow-up (test-only analyzer pin). The underlying C++ fix in this PR already covers both affected code paths in `JoinStepLogical` — `toBoolIfNeeded` (single residual) and `concatConditions` (multi residual) — and preserves nullability via `Nullable(UInt8)` when the input is nullable.

Session id: cron:clickhouse-worker-slot-18:20260610-203300

groeneai · 2026-06-11T11:10:00Z

CI finished for HEAD a019401b625d. No PR-caused failures. Remaining reds are both unrelated to this change:

AST fuzzer (amd_debug, targeted) / Logical error 'Column identifier ... is already registered' (STID 4697-4326): chronic trunk AST-fuzzer bug that hits many unrelated PRs and master; tracked separately.
Stateless tests (arm_asan_ubsan, azure, sequential) Server died / Segfault: known 2026-06-10/11 Azure ASan-allocator OOM infra outage (95+ unrelated PRs in 24h; stack has no ClickHouse frames).

The PR's own regression test 04327_join_on_nothing_typed_residual passes on every variant. Ready for review.

groeneai · 2026-06-30T22:20:33Z

CI finish ledger — `d441299`

Every failure below has an owner: a fixing PR (ours or external). Only CH Inc sync is exempt.

Check / test	Reason	Owner / fixing PR
AST fuzzer (amd_debug, targeted) / Logical error `or_argument_nodes.size() > 1` (STID 2508-3fad)	logical error (trunk, analyzer)	#107113 (ours, open)
AST fuzzer (arm_asan_ubsan) / Block structure mismatch in A stream (STID 0993-27f0)	logical error (trunk, UnionStep)	#107719 (ours, open)
Stress test (amd_tsan) / Hung check failed, possible deadlock	deadlock (chronic trunk)	#108212 (ours, merged) / #105905 (ours, open)
Stress test (arm_tsan) / Hung check failed, possible deadlock	deadlock (chronic trunk)	#108212 (ours, merged) / #105905 (ours, open)
CH Inc sync	private fork-sync mirror	CH Inc sync (private, not actionable)

This PR (Nothing-typed JOIN ON residual predicate fix) is approved by @ vdimir. The earlier 02354_vector_search_rescoring failures cleared after merging master (picked up #108812). None of the above are caused by this PR's diff.

Session id: cron:our-pr-ci-monitor:20260630-220000

clickhouse-gh · 2026-07-01T14:20:08Z

LLVM Coverage Report

Changed lines: Changed C/C++ lines covered: 26/26 (100.00%) · Uncovered code

Full report · Diff report

nikitamikhaylov added the can be tested Allows running workflows for external contributors label Jun 10, 2026

clickhouse-gh Bot added the pr-bugfix Pull request with bugfix, not backported by default label Jun 10, 2026

clickhouse-gh Bot reviewed Jun 10, 2026

View reviewed changes

Comment thread tests/queries/0_stateless/04327_join_on_nothing_typed_residual.sql Outdated

groeneai and others added 2 commits June 10, 2026 20:34

vdimir self-assigned this Jun 29, 2026

vdimir approved these changes Jun 29, 2026

View reviewed changes

vdimir and others added 2 commits June 29, 2026 14:20

Merge branch 'master' into fix-join-on-nothing-typed-residual

c9ee170

Merge branch 'master' into fix-join-on-nothing-typed-residual

d441299

groeneai mentioned this pull request Jun 30, 2026

Fix logical error in correlated subquery decorrelation with set operations #108554

Open

Merge branch 'master' into fix-join-on-nothing-typed-residual

5b37fe2

vdimir added this pull request to the merge queue Jul 2, 2026

Merged via the queue into ClickHouse:master with commit c8de357 Jul 2, 2026
172 of 174 checks passed

robot-ch-test-poll mentioned this pull request Jul 2, 2026

Logical Error: Unexpected expression in JOIN ON section. Expected boolean (UInt8), got 'Nothing' #78761

Closed

robot-clickhouse-ci-2 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jul 2, 2026

Sunbelt Computer Software

PL/B Language Development and Support

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix LOGICAL_ERROR for Nothing-typed JOIN ON residual predicate#106981

Fix LOGICAL_ERROR for Nothing-typed JOIN ON residual predicate#106981
vdimir merged 6 commits into
ClickHouse:masterfrom
groeneai:fix-join-on-nothing-typed-residual

groeneai commented Jun 10, 2026 •

edited by robot-ch-test-poll

Loading

Uh oh!

groeneai commented Jun 10, 2026

Uh oh!

groeneai commented Jun 10, 2026

Uh oh!

clickhouse-gh Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

Uh oh!

groeneai commented Jun 10, 2026

Uh oh!

groeneai commented Jun 11, 2026

Uh oh!

groeneai commented Jun 30, 2026

Uh oh!

clickhouse-gh Bot commented Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!

Conversation

groeneai commented Jun 10, 2026 • edited by robot-ch-test-poll Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Version info

Uh oh!

groeneai commented Jun 10, 2026

Pre-PR validation gate

Uh oh!

groeneai commented Jun 10, 2026

Uh oh!

clickhouse-gh Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

AI Review

Summary

Missing context / blind spots

Final Verdict

Uh oh!

Uh oh!

groeneai commented Jun 10, 2026

Pre-PR validation gate (follow-up fixup a019401b625d)

Uh oh!

groeneai commented Jun 11, 2026

Uh oh!

groeneai commented Jun 30, 2026

CI finish ledger — d441299

Uh oh!

clickhouse-gh Bot commented Jul 1, 2026

LLVM Coverage Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

groeneai commented Jun 10, 2026 •

edited by robot-ch-test-poll

Loading

clickhouse-gh Bot commented Jun 10, 2026 •

edited

Loading

Pre-PR validation gate (follow-up fixup `a019401b625d`)

CI finish ledger — `d441299`