{{ message }}
Throw ILLEGAL_COLUMN when _distance is selected directly in vector search queries#108423
Merged
Conversation
…arch queries
The `_distance` virtual column is internal to the vector search optimization and
populated only when the optimized plan rewrites the query. Previously, referencing
it directly in SELECT with an ORDER BY distance function caused a LOGICAL_ERROR
("Vector column unexpectedly already replaced") because the optimizer tried to add
`_distance` to the read list while it was already there from the user's SELECT.
Now this case throws a user-facing ILLEGAL_COLUMN error with a clear message
directing users to use the distance function (L2Distance, cosineDistance) in
ORDER BY instead.
The check is placed in both the first pass (tryUseVectorSearch) and the second pass
(optimizeVectorSearchSecondPass) for defense-in-depth coverage.
Closes: ClickHouse/clickhouse-core-incidents#1654
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
clickhouse-gh
Bot
added
the
pr-bugfix
![clickhouse-gh[bot]](https://avatars.githubusercontent.com/in/928874?s=60&v=4){kind=small}
| if (read_from_mergetree_step->isVectorColumnReplaced()) | ||
| throw Exception(ErrorCodes::ILLEGAL_COLUMN, | ||
| "The `_distance` column is an internal virtual column of vector search and cannot be referenced directly in queries. " | ||
| "Use the distance function (e.g. `L2Distance`, `cosineDistance`) in ORDER BY instead"); |
Contributor
There was a problem hiding this comment.
The suggested fix is misleading for the incident this PR handles: the failing query already has ORDER BY L2Distance(...); the direct reference is in the select list. As written, users can follow the instruction and still get the same error.
Please change the diagnostic in all three copies, or factor it, to tell users to select the distance expression instead of _distance, e.g. SELECT L2Distance(...) AS distance ... ORDER BY distance.
|
|
Contributor
LLVM Coverage ReportChanged lines: Changed C/C++ lines covered by tests: 16/22 (72.73%) | Lost baseline coverage: none · Uncovered code |
alexey-milovidov
approved these changes
Jun 28, 2026
robot-ch-test-poll4
added
the
pr-synced-to-cloud
robot-clickhouse-ci-1
added
the
pr-must-backport-synced
robot-ch-test-poll4
added a commit
that referenced
this pull request
Jun 28, 2026
Cherry pick #108423 to 26.4: Throw `ILLEGAL_COLUMN` when `_distance` is selected directly in vector search queries
robot-clickhouse
added a commit
that referenced
this pull request
Jun 28, 2026
…selected directly in vector search queries
robot-ch-test-poll4
added a commit
that referenced
this pull request
Jun 28, 2026
Cherry pick #108423 to 26.5: Throw `ILLEGAL_COLUMN` when `_distance` is selected directly in vector search queries
robot-clickhouse
added a commit
that referenced
this pull request
Jun 28, 2026
…selected directly in vector search queries
robot-ch-test-poll4
added a commit
that referenced
this pull request
Jun 28, 2026
Cherry pick #108423 to 26.6: Throw `ILLEGAL_COLUMN` when `_distance` is selected directly in vector search queries
robot-clickhouse
added a commit
that referenced
this pull request
Jun 28, 2026
…selected directly in vector search queries
robot-clickhouse
added
the
pr-backports-created
clickhouse-gh Bot
added a commit
that referenced
this pull request
Jun 28, 2026
Backport #108423 to 26.6: Throw `ILLEGAL_COLUMN` when `_distance` is selected directly in vector search queries
alexey-milovidov
added a commit
that referenced
this pull request
Jun 29, 2026
Backport #108423 to 26.5: Throw `ILLEGAL_COLUMN` when `_distance` is selected directly in vector search queries
shankar-iyer
added a commit
that referenced
this pull request
Jun 29, 2026
Backport #108423 to 26.4: Throw `ILLEGAL_COLUMN` when `_distance` is selected directly in vector search queries
This was referenced Jun 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
_distancevirtual column is internal to the vector search optimization and populated only when the optimized plan rewrites the query. Previously, referencing it directly in SELECT with an ORDER BY distance function caused a LOGICAL_ERROR ("Vector column unexpectedly already replaced") because the optimizer tried to add_distanceto the read list while it was already there from the user's SELECT.Now this case throws a user-facing ILLEGAL_COLUMN error with a clear message directing users to use the distance function (L2Distance, cosineDistance) in ORDER BY instead.
The check is placed in both the first pass (tryUseVectorSearch) and the second pass (optimizeVectorSearchSecondPass) for defense-in-depth coverage.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
Vector search queries that SELECT from the
_distancecolumn now return a proper error instead of failing with aLOGICAL_ERROR.Version info
26.7.1.18626.6.2.9,26.5.4.22,26.4.5.78