Implements SQL truth-value predicates IS TRUE, IS FALSE, IS UNKNOWN and their IS NOT variants. Closes ClickHouse/ClickHouse#99597.#99997
Conversation
|
Workflow [PR], commit [8adb439] Summary: ❌
AI ReviewSummaryThis PR adds parser support for SQL truth-value predicates ( Final VerdictStatus: ✅ Approve |
alexey-milovidov
left a comment
There was a problem hiding this comment.
Good change, almost ready!
…m` and `isDistinctFrom` operators.
|
@alexey-milovidov I checked the failing tests, and they do not seem to be caused by the new changes in this PR. Please let me know whether I should investigate them or treat them as unrelated CI failures. |
|
@alexey-milovidov just a kind reminder on this PR. If any further updates are needed from my side, I’m ready to make them. |
|
@alexey-milovidov sorry for the ping — quick question as this is my first time. Do I need to do anything after approval for the PR to be merged, or does it just take some time? |
|
We will take it from here. |
|
Fix for the integration test: #101897 |
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Now we have to fix the hung queries in the Stress test... |
…mkarimeddin-fix-99597-is-true-false-unknown
|
I fixed the hung check, will update the branch. |
|
Hi @alexey-milovidov , gentle follow-up on this PR. Thanks again for the review and help here. Please let me know if anything else is needed from my side. |
|
I will take it from here. The CI is not green, we have to fix the CI first. |
|
Merged latest master into the branch and pushed ( CI failures analysis (from the previous run)The two failing checks are unrelated to this PR (which only touches the SQL parser):
@groeneai, investigate both unrelated failures:
|
|
@alexey-milovidov — investigated both failures. Both are pre-existing trunk issues, unrelated to this PR's parser-only diff. 1. Stress test (arm_msan) —
|
|
Correction on the MSan analysis above — I was over-inclusive with my pattern matching. The Filtering to the actual STID test_name: SELECT toStartOfDay(check_start_time) AS day, count() AS failures, any(check_name) AS check
FROM default.checks
WHERE test_name = 'MemorySanitizer: use-of-uninitialized-value (STID: 1478-2063)'
AND test_status IN ('FAIL', 'ERROR')
AND pull_request_number = 0
AND check_start_time > now() - INTERVAL 60 DAY
GROUP BY day ORDER BY day DESCZero master hits since 2026-04-10, so STID 1478-2063 itself does appear to be resolved (likely consistent with The CI failure linked from this PR is on SHA Same story for the TL;DR: Both failures are stale CI from 2026-04-10. The new run on |
LLVM Coverage Report
Changed lines: 94.34% (50/53) | lost baseline coverage: 1 line(s) · Uncovered code |
There was a problem hiding this comment.
@groeneai if there is a set index on a merge tree boolean column and IS TRUE or IS UNKNOWN predicate (if the column is nullable) is used, is the index used? Add a test case, please.
There was a problem hiding this comment.
Filed #105865 with a regression test. Short answer: yes, the set skip index is already used in all six forms.
The new predicates lower to functions that the set index recognises in MergeTreeIndexConditionSet::atomFromDAG and then evaluates per-granule in mayBeTrueOnGranule:
b IS TRUE->isNotDistinctFrom(b, true)b IS FALSE->isNotDistinctFrom(b, false)b IS UNKNOWN->isNull(b)b IS NOT TRUE->isDistinctFrom(b, true)b IS NOT FALSE->isDistinctFrom(b, false)b IS NOT UNKNOWN->isNotNull(b)
With three granules holding only true, only false, and only NULL respectively, the set index drops the two unrelated granules for each positive form and the matching granule for each IS NOT form. The new test asserts that via ProfileEvents['SelectedMarks'].
One small follow-up not addressed by this PR: EXPLAIN indexes = 1 prints Condition: true for the isNotDistinctFrom / isDistinctFrom forms because KeyCondition only special-cases isNull / isNotNull (for IS UNKNOWN / IS NOT UNKNOWN it prints (b isNull) / (b isNotNull)). The pruning still happens via the actions DAG path, but teaching KeyCondition about the new operators would make the EXPLAIN output more informative. Happy to send a separate PR for that if you'd like.
Sibling of `04290_set_index_is_true_false_unknown` covering the leading primary-key path on a `Nullable(Bool)` column. `KeyCondition` recognises the `isNull` / `isNotNull` lowered forms and drops one granule for `IS UNKNOWN` and `IS NOT UNKNOWN`. The other four predicates lower to `isNotDistinctFrom` / `isDistinctFrom`, which are not recognised by `KeyCondition`, so the test pins this current behaviour. Follow-up to ClickHouse#99997. Requested in ClickHouse#105865.
…NKNOWN Verifies that the set skip index prunes granules for the new truth-value predicates from ClickHouse#99997 on a `Nullable(Bool)` column. The new test populates three granules of eight rows each (only true, only false, only NULL) and checks via `ProfileEvents['SelectedMarks']` that the index drops the two unrelated granules for each positive form and the matching granule for each `IS NOT` form. Follow-up to ClickHouse#99997.
The `swap-clickhouse-jdbc.py` module docstring claimed the NoREC oracle's `IS TRUE` postfix is rewritten to `= TRUE`, but the implementation (and the inline comment, and the PR description) correctly rewrite it to `!= 0`. The `= TRUE` variant was tried and rejected; the docstring was left behind. Also clarify the rationale in both the docstring and the Dockerfile comment. `IS TRUE`/`IS FALSE` are now implemented in ClickHouse's parser (since #99997), but `IS TRUE` parses as `<expr> <=> true`, i.e. strict equality with `1`, which does not match `WHERE <expr>` truthiness (any non-zero numeric). Verified locally that `5 IS TRUE` = 0 while `WHERE 5` matches and `5 != 0` = 1, so the `!= 0` rewrite remains correct and necessary - using native `IS TRUE` would silently produce false-positive oracle mismatches. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Fixes #99597
Changelog category (leave one):
Changelog entry:
Added support for SQL truth-value predicates
IS TRUE,IS FALSE,IS UNKNOWN,IS NOT TRUE,IS NOT FALSE, andIS NOT UNKNOWNfor nullable boolean expressions.Documentation entry for user-facing changes
Motivation: Allows unambiguous testing of nullable boolean columns for all three SQL truth values (true, false, null/unknown). Unlike
= true, theIS TRUEpredicate returnsfalse(notNULL) when the value isNULL.Example use: