iframe-proxy

groeneai · 2026-06-29T13:17:15Z

Changelog category (leave one):

CI Fix or Improvement (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

...

Description

Reverts the merge of #105591 (commit d56213874eb), reversing changes made to 6519f67a9eb.

#105591 broke 02354_vector_search_rescoring on master across all build configs (amd_debug, amd_asan_ubsan, amd_tsan, amd_llvm_coverage, arm_binary) with a deterministic Code: 44. ILLEGAL_COLUMN ("The _distance column is an internal virtual column of vector search and cannot be referenced directly in queries"). CI randomized-settings diagnosis: "Runs: 350, Failed: 350, Passed: 0. Test also fails without randomized settings (not a randomization issue)."

Report (amd_debug, parallel): https://s3.amazonaws.com/clickhouse-test-reports/json.html?REF=master&sha=d56213874eb9a8630fb1dd4862525efcf564d30f&name_0=MasterCI&name_1=Stateless%20tests%20%28amd_debug%2C%20parallel%29

Root cause is a base-skew semantic conflict: #105591 was developed and CI-tested on a base that predated the _distance guard, then merged onto a master that already contained it.

Throw ILLEGAL_COLUMN when _distance is selected directly in vector search queries #108423 (merged 7625ed4f8f2 @ 2026-06-28 02:32 UTC) added a hard ILLEGAL_COLUMN guard in both passes of useVectorSearch.cpp (and in ReadFromMergeTree.cpp), making _distance an internal-only virtual column. It added 02354_vector_search_incident1654.sql (customer incident StorageKafka: extended configuration, parallel consumers, offset tracking #1654) asserting the guard fires.
Apply exact row-positioning for vector search rescoring queries #105591 (merged d56213874eb @ 2026-06-29 11:10 UTC; PR branch tip last updated 2026-06-26) added a feature that deliberately supports referencing _distance directly, with test queries and reference output in 02354_vector_search_rescoring.sql that expect those queries to succeed. Its CI ran on a guard-free base and passed.

The guard from #108423 pre-empts #105591's new code path at useVectorSearch.cpp:236, so #105591's own added queries throw ILLEGAL_COLUMN once both changes are on master. Master bisect confirms: 6519f67a9eb (pre-#105591, guard already present) was green for the existing tests; d56213874eb (#105591) is red across all configs.

Reverting #105591 (the later, feature PR) restores master to green while preserving the customer-incident #1654 guard. The product decision of whether _distance should be user-referenceable is left to the vector-search owners to reconcile and re-land #105591 on top of the #108423 guard.

Local verification on the reverted branch (build id 16173c3a04...):

02354_vector_search_rescoring output matches reference exactly
02354_vector_search_rescoring_and_prewhere passes
02354_vector_search_incident1654 guard still fires ILLEGAL_COLUMN for both queries

Note: an earlier version of this description attributed the _distance guard to #107985; that was incorrect. #107985 ("Add ProfileEvents to observe distributed query plan execution") is unrelated. The guard was added by #108423, as pointed out by @Algunenano below.

Version info

Merged into: 26.7.1.244

…es" (ClickHouse#105591) This reverts the merge of PR ClickHouse#105591 (commit d562138), reversing changes made to 6519f67. ClickHouse#105591 broke 02354_vector_search_rescoring on master across all build configs with a deterministic Code 44 ILLEGAL_COLUMN ("The _distance column is an internal virtual column of vector search and cannot be referenced directly"). Root cause is a base-skew semantic conflict between two PRs merged 65 minutes apart, neither of which saw the other in CI: - ClickHouse#107985 (merged 10:05 UTC) added a hard ILLEGAL_COLUMN guard in both passes of useVectorSearch.cpp, making _distance internal-only. It closed customer incident clickhouse-core-incidents#1654 and added test 02354_vector_search_incident1654 asserting the guard. - ClickHouse#105591 (merged 11:10 UTC) added a feature that deliberately supports referencing _distance directly, with test queries and reference output that expect it to succeed. The guard from ClickHouse#107985 pre-empts ClickHouse#105591's new code path, so ClickHouse#105591's own tests fail on master. Reverting ClickHouse#105591 (the later, feature PR) restores master to green while preserving the customer-incident ClickHouse#1654 guard. The product decision of whether _distance should be user-referenceable is left to the vector-search owners to reconcile and re-land. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

groeneai · 2026-06-29T13:18:14Z

Pre-PR validation gate (click to expand)

#	Question	Answer
a	Deterministic repro?	Yes. On master (`d56213874eb`/`f39e0967ab2`) the query `WITH [0.0,0.0] AS r SELECT id, round(_distance,2) FROM tab_multi_granule ORDER BY L2Distance(vec,r) LIMIT 4 SETTINGS vector_search_with_rescoring=0` throws `Code: 44 ILLEGAL_COLUMN` every time. CI diagnosis: 350/350 failures, "fails without randomized settings (not a randomization issue)."
b	Root cause explained?	Yes. Base-skew semantic conflict: #108423 (merged 2026-06-28) added a hard `ILLEGAL_COLUMN` guard in both passes of `useVectorSearch.cpp` (`_distance` is internal-only, closing customer incident #1654). #105591 (merged 2026-06-29 11:10 UTC; branch tip last updated 2026-06-26) added a feature + tests that deliberately reference `_distance` directly and expect success. #105591's CI ran on a guard-free base and passed; the #108423 guard at `useVectorSearch.cpp:236` then pre-empts #105591's path once both are on master, so #105591's own tests throw.
c	Fix matches root cause?	Yes. Reverting #105591 (the later feature PR) removes the conflicting feature + tests and restores master to the pre-#105591 green state, while keeping the customer-incident #1654 guard intact. This directly resolves the conflict; the product decision is deferred to the vector-search owners to re-land #105591 atop the guard.
d	Test intent preserved / new tests added?	Yes. The #108423 guard test `02354_vector_search_incident1654` is preserved and still fires `ILLEGAL_COLUMN` for both its queries. No test weakened; this is a clean revert of a feature PR, not a band-aid.
e	Both directions demonstrated?	Yes. Without revert: master red across all configs (CIDB, reproduced locally to Code 44). With revert (build id `16173c3a04...`): `02354_vector_search_rescoring` output matches reference exactly, `02354_vector_search_rescoring_and_prewhere` passes, `02354_vector_search_incident1654` guard still fires.
f	Fix is general across code paths?	Yes. The revert restores all 13 files of #105591 byte-for-byte to the pre-merge parent `6519f67a9eb`; no sibling code path is left in a half-reverted state. Verified `git diff 6519f67a9eb` is empty for every affected source/test file.
g	Fix generalizes across inputs?	N/A (clean revert of a feature PR, not a code-path bug fix with input variants).
h	Backward compatible?	Yes. #105591 was never in a stable release; reverting it has no backward-compatibility impact. `Settings.cpp` changes are reverted to the pre-#105591 state.
i	Invariants and contracts preserved?	Yes. The revert returns `useVectorSearch.cpp`, `MergeTreeRangeReader.cpp`, `MergeTreeDataSelectExecutor.cpp`, `RangesInDataPart.h` etc. to their exact pre-#105591 contracts. No commit after #105591 touched any of these files, so the revert has no follow-on conflicts.

[Edit: corrected the guard PR in row b from #107985 to #108423 per Algunenano; #107985 is unrelated.]

Session id: cron:clickhouse-worker-slot-1:20260629-124700

groeneai · 2026-06-29T13:18:26Z

cc @shankar-iyer @rschu1ze: master is currently red on 02354_vector_search_rescoring across all configs (deterministic Code: 44 ILLEGAL_COLUMN). It's a base-skew conflict: #108423 (merged 2026-06-28) made _distance internal-only to close incident #1654, then #105591 (merged 2026-06-29 11:10 UTC; branch tip last updated 2026-06-26) added a feature + tests that reference _distance directly and expect it to work. #105591's CI ran on a guard-free base and passed; the guard pre-empts the feature, so #105591's own tests fail once both are on master. This reverts #105591 to restore green while preserving the #1654 guard; the feature can be re-landed on top of the guard. Could you confirm the revert is the right call vs. another reconciliation?

[Edit: corrected the guard PR from #107985 to #108423 per Algunenano below; #107985 ("Add ProfileEvents to observe distributed query plan execution") is unrelated.]

clickhouse-gh · 2026-06-29T13:21:12Z

Workflow [PR], commit [e531152]

AI Review

Summary

This PR cleanly reverts #105591, restoring all changed source, docs, and stateless test files to the pre-feature parent 6519f67a9eb while preserving the _distance direct-reference guard added by #107985. I did not find any new correctness, compatibility, or test-coverage findings in the current diff.

Final Verdict

Status: ✅ Approve

Algunenano · 2026-06-29T13:23:18Z

@groeneai The PR that added the guards was #108423, not #107985 (which is unrelated)

Algunenano

Verified locally

groeneai · 2026-06-29T14:22:04Z

Algunenano self-assigned this Jun 29, 2026

Algunenano added the can be tested Allows running workflows for external contributors label Jun 29, 2026

clickhouse-gh Bot added the pr-ci label Jun 29, 2026

Algunenano mentioned this pull request Jun 29, 2026

Apply exact row-positioning for vector search rescoring queries #105591

Merged

1 task

Algunenano approved these changes Jun 29, 2026

View reviewed changes

Algunenano added this pull request to the merge queue Jun 29, 2026

Merged via the queue into ClickHouse:master with commit 1e41bf3 Jun 29, 2026
46 of 70 checks passed

groeneai mentioned this pull request Jun 29, 2026

Fix server crash on lazy remote source with unbuilt materialized CTE #108547

Merged

robot-ch-test-poll4 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jun 29, 2026

groeneai mentioned this pull request Jun 29, 2026

Add allow_lossy_numeric_supertype setting for numeric Variant common types #107236

Open

skuznetsov mentioned this pull request Jun 29, 2026

Re-land exact row filtering for vector search rescoring queries #108846

Open

Sunbelt Computer Software

PL/B Language Development and Support

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Revert "Apply exact row-positioning for vector search rescoring queries" (#105591)#108812

Revert "Apply exact row-positioning for vector search rescoring queries" (#105591)#108812
Algunenano merged 1 commit into
ClickHouse:masterfrom
groeneai:groeneai/revert-105591-vector-rescoring-distance-conflict

groeneai commented Jun 29, 2026 •

edited by robot-clickhouse-ci-2

Loading

Uh oh!

groeneai commented Jun 29, 2026 •

edited

Loading

Uh oh!

groeneai commented Jun 29, 2026 •

edited

Loading

Uh oh!

clickhouse-gh Bot commented Jun 29, 2026 •

edited

Loading

Uh oh!

Algunenano commented Jun 29, 2026

Uh oh!

Algunenano left a comment

Uh oh!

Uh oh!

groeneai commented Jun 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Sunbelt Computer Software

PL/B Language Development and Support

Uh oh!

Conversation

groeneai commented Jun 29, 2026 • edited by robot-clickhouse-ci-2 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Description

Version info

Uh oh!

groeneai commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

groeneai commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clickhouse-gh Bot commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

AI Review

Summary

Final Verdict

Uh oh!

Algunenano commented Jun 29, 2026

Uh oh!

Algunenano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

groeneai commented Jun 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

groeneai commented Jun 29, 2026 •

edited by robot-clickhouse-ci-2

Loading

groeneai commented Jun 29, 2026 •

edited

Loading

groeneai commented Jun 29, 2026 •

edited

Loading

clickhouse-gh Bot commented Jun 29, 2026 •

edited

Loading