Per IEEE 754 / SQL JOIN ON semantics, NaN never compares equal to NaN,
but ClickHouse's hash table compared float keys bitwise via bitEquals
(memcmp). Two NaN values with matching bit patterns therefore wrongly
joined - INNER JOIN ON NaN = NaN returned rows instead of zero, and
ANTI JOIN dropped the NaN-keyed left row instead of keeping it.

Fold NaN rows into the JOIN-key null map so they are skipped on both
build and probe sides, mirroring how NULL keys behave. Done at three
layers:

* HashJoin / ConcurrentHashJoin: extend the build- and probe-side
  null_map after extractNestedColumnsAndNullMap via a new helper
  JoinCommon::extendJoinKeyNullMapWithFloatNaNs.

* FullSortingMergeJoin: in FullMergeJoinCursor::setChunk, OR NaN
  positions into the per-key null_map so nullableCompareAt
  short-circuits.

* BuildRuntimeFilterTransform: drop NaN rows before they enter the
  bloom/exact runtime filter, so the runtime filter never carries a
  NaN entry that would mis-prune a probe-side NaN.

GROUP BY / DISTINCT / uniq / IN keep their existing NaN-coalescing
behaviour - this changes only the JOIN ON path.

Closes: ClickHouse#106531

Per @PedroTadim review on PR ClickHouse#106540: cover BFloat16 in addition to
Float32 / Float64 / Nullable(Float64). The fix already dispatches on
BFloat16 in all three NaN-detection paths (extendJoinKeyNullMapWithFloatNaNs,
foldFloatNaNsIntoNullMap, filterOutNaNs); this commit just exercises that
dispatch end-to-end.

The new block covers BFloat16 keys for hash INNER / LEFT / ANTI and
full_sorting_merge INNER, mirroring the Float32 grid.

The previous helper only recognised a top-level `ColumnFloat32`/`ColumnFloat64`/
`ColumnVector<BFloat16>`. Runtime filters can be built over `Nullable(Float*)`,
`LowCardinality(Nullable(Float*))`, or - for multi-key `LEFT ANTI` - a
temporary `Tuple` of common types, so a probe-side `NaN` was still pre-pruned
by `ExactNotContainsRuntimeFilter` before the join saw the row.

Recursively traverse those wrappers and OR-mark every row whose float payload
is `NaN`, then filter the original (still-wrapped) column once. Already-NULL
rows do not need to be dropped: the runtime filter stores `NULL` natively
(`argument_can_have_nulls`), so there is no bitwise match for them on the
probe side.

`isNaN` is templated and works on `Float32`, `Float64` and `BFloat16`, so the
fix covers all three element types under any of the three wrappers.

Test additions in `04318_106531_join_on_nan_keys.sql` cover, with
`enable_join_runtime_filters = 1` and `join_algorithm = 'hash'`:

* `Nullable(Float64)` `ANTI`/`INNER`
* `LowCardinality(Nullable(Float64))` `ANTI`
* `Tuple(Float64, Float64)` `LEFT ANTI` (multi-key path)

`extendJoinKeyNullMapWithFloatNaNs` (HashJoin / ConcurrentHashJoin path)
and `foldFloatNaNsIntoNullMap` (full_sorting_merge path) only dispatched
on top-level `ColumnFloat32` / `ColumnFloat64` / `ColumnVector<BFloat16>`.
A `ColumnTuple` key fell through, so `tuple(NaN) = tuple(NaN)` (whether
planner-generated or written explicitly as
`ON tuple(l.k) = tuple(r.k)`) still produced a wrong match because the
hash table or merge cursor compared the tuple payload bitwise.

Mirror the recursion landed at 1b3d5dc in
`BuildRuntimeFilterTransform::filterOutNaNs`: unwrap `Nullable` (respect
existing null map for the element-Nullable case),
unwrap `LowCardinality` via `convertToFullColumnIfLowCardinality`, and
recurse into `ColumnTuple` (a row is dropped if ANY float element is
`NaN`, matching scalar tuple equality semantics: `(a, b) = (c, d)` iff
`a = c AND b = d`, with `NaN != NaN`).

`extendJoinKeyNullMapWithFloatNaNs` first runs a type-only walk
(`containsFloatPayload`) so a fresh null map is still allocated lazily;
purely integer/string/date keys keep the previous fast path.

Tuple-keyed `JOIN ON` test cases added to
`04318_106531_join_on_nan_keys.sql` for `hash` and `full_sorting_merge`,
covering plain `tuple(Float64)`, multi-element `tuple(Float64, Float64)`
where only one element is `NaN`, `tuple(Nullable(Float64))` (preserves
NULL-safe tuple equality), and `tuple(LowCardinality(Nullable(Float64)))`.

Demonstrated both directions on a debug build: pre-fix (HEAD~) the
tuple-keyed `INNER JOIN` returns the matched row plus the wrongly-joined
NaN rows; post-fix it returns only the matched row. ANTI returns the
NaN-keyed rows after the fix.

Closes: ClickHouse#106531

The HashJoin / FullSortingMergeJoin / BuildRuntimeFilter NaN-as-null
treatment did not extend to `SET join_algorithm = 'direct'`. The MergeTree
direct path builds the right-side lookup with `IN` (which intentionally
keeps coalescing NaN per this PR's design) and then maps found rows back
to probe keys via `ColumnsHashing::HashMethodHashed`, which compares the
key payload bitwise. Two `NaN` keys with identical bit patterns therefore
matched for `INNER` / `LEFT SEMI` and were suppressed for `LEFT ANTI`.

Force every probe row whose key holds a `NaN` to the not-found path
(sentinel selector + `null_map = false`) inside `getByKeys`, mirroring
`JoinCommon::extendJoinKeyNullMapWithFloatNaNs`. The recursion walks
`Nullable` / `LowCardinality` / `Tuple` so the same shapes covered on
the hash path work here too. `joinKeyContainsFloatPayload` and
`markFloatNaNRowsAsNull` are exposed in `JoinCommon` so the helper is
shared with the hash and merge paths.

New tests in `04318_106531_join_on_nan_keys.sql` cover
`SET join_algorithm = 'direct'` over MergeTree-keyed `Float64` and
`Nullable(Float64)` for `INNER` / `LEFT` / `LEFT SEMI` / `LEFT ANTI`.

Address-requested in clickhouse-gh[bot] inline review on commit
a08706c (PR ClickHouse#106540) at 2026-06-06T05:03:46Z.

Closes: ClickHouse#106531

…umnConst

Addresses 5 follow-up review items on PR ClickHouse#106540 (issue ClickHouse#106531):

1. arm_tidy parameter-name mismatch: align JoinUtils.h declaration with
   .cpp definition (mutable_null_map). Fixes the
   readability-inconsistent-declaration-parameter-name warning that
   broke the arm_tidy build on commit c183aff.

2. ColumnNullable branch must respect the outer null map. A
   ColumnNullable may carry arbitrary trash bytes in its nested column
   for rows where the null map says NULL. If those bytes happen to be a
   NaN bit pattern, the previous markFloatNaNRowsAsNull marked the
   whole row in the join null map and broke null-safe tuple equality
   (tuple(NULL) = tuple(NULL) wrongly became a non-match). The
   recursion now threads is_null down so an already-NULL row is left
   alone.

3. MergeJoinTransform foldFloatNaNsIntoNullMap and
   BuildRuntimeFilterTransform filterOutNaNs gain a type-only
   joinKeyContainsFloatPayload guard. Most full-sorting-merge and
   runtime-filter keys are integer/string/date and never carry a NaN,
   so we now skip the per-chunk ColumnUInt8/Filter allocation in those
   cases.

4. Defence-in-depth ColumnConst handling in joinKeyContainsFloatPayload
   and markFloatNaNRowsAsNull (and the parallel helpers in
   MergeJoinTransform and BuildRuntimeFilterTransform). Direct joins
   pass the original probe key column without convertToFullColumnIfConst,
   so a constant NaN can reach this helper as
   ColumnConst(ColumnFloat*(NaN), N). Marking all rows when the
   underlying scalar is NaN keeps the constant-probe semantics aligned
   with the materialised path.

Test: 04318_106531_join_on_nan_keys gains a Tuple(Nullable(Float64))
null-safe-match case where NULL slots carry a NaN trash payload.
Pre-fix this returned 0 (BUG); post-fix it returns 1.

Address three follow-up CRs from clickhouse-gh[bot] on commit 25b677f
(PR ClickHouse#106540, NaN-as-null treatment for JOIN ON):

CR 1 — `ColumnSparse` was not unwrapped in `joinKeyContainsFloatPayload` /
`markFloatNaNRowsAsNullImpl` (`JoinUtils.cpp`), `markNaNRows`
(`MergeJoinTransform.cpp`), and `markNaNRowsImpl`
(`BuildRuntimeFilterTransform.cpp`). HashJoin and full_sorting_merge
materialise sparse columns upstream, but direct joins and runtime-filter
builds can see them raw. Added a `column.isSparse()` branch in each helper
that calls `convertToFullColumnIfSparse` and recurses on the dense form.

CR 2 — The `ColumnConst` branch in `markFloatNaNRowsAsNullImpl` recursed
into the one-row data column while keeping the full-size `mutable_null_map`,
so the leaf branch's `chassert(data.size() == mutable_null_map.size())`
would trip in debug/ASan builds. Rewrote the branch to evaluate the scalar
into a one-row `scalar_mask`, then spread the mark across all rows not
excluded by `is_null`. Mirrors the pattern already used in
`BuildRuntimeFilterTransform.cpp::markNaNRowsImpl`. Same fix applied to
`MergeJoinTransform.cpp::markNaNRows`.

CR 3 — Re-confirms CR 2 on the `HashJoin.cpp:792` thread; addressed by
the same JoinUtils.cpp rewrite.

Test additions to `tests/queries/0_stateless/04318_106531_join_on_nan_keys`:
- Constant `NaN` probe key (hash) covering the rewritten ColumnConst branch
  via the helper's contract.
- Build-side sparse `Float64` key with runtime filter enabled, exercising
  the new `ColumnSparse` unwrap in `markNaNRowsImpl`.

Local verification (debug build, x86-64-v2): 30/30 passes via
`clickhouse local --queries-file` on the test file, sibling
`03214_join_on_tuple_comparison_elimination_bug` continues to pass.

The legacy MergeJoin (used by `SET join_algorithm = 'partial_merge'`) sorts
both sides and merges by `nullableCompareAt`, which falls through to
`compareAt` and compares Float32/Float64/BFloat16 bitwise. Two NaNs with
identical bit patterns then form an equal range, so a NaN row on either
side spuriously matches another NaN row, violating IEEE 754
`NaN != NaN`.

The other algorithms patched in this PR (HashJoin, ConcurrentHashJoin,
GraceHashJoin via runtime filter, DirectKeyValueJoin, FullSortingMergeJoin)
all fold NaN into the join-key null map. MergeJoin's sort-then-merge
contract makes the same fold here require a schema change (wrap float keys
in `Nullable` for the cursor view) that the surrounding output path is not
set up for, so we reject instead.

The constructor throws `NOT_IMPLEMENTED` with a message pointing the user
to `hash` / `parallel_hash` / `grace_hash` / `full_sorting_merge`. The new
`MergeJoin::isSupported(table_join, right_sample_block)` overload is used
by `AUTO` and `PARTIAL_MERGE` selection in `PlannerJoins` and
`ExpressionAnalyzer` so existing AUTO queries with float keys silently
fall back to HashJoin (which now handles NaN correctly) instead of
throwing mid-query when JoinSwitcher would attempt to switch to
MergeJoin.

A new type-only helper `JoinCommon::joinKeyTypeContainsFloatPayload`
walks `Nullable` / `LowCardinality` / `Tuple` so all four wrapper shapes
are covered.

Test cases added to `tests/queries/0_stateless/04318_106531_join_on_nan_keys.sql`:
Float64, Nullable(Float64), LowCardinality(Nullable(Float64)) and
Tuple(Float64, Float64) all expected to throw `NOT_IMPLEMENTED` under
`partial_merge`; an Int32 partial_merge case verifies non-float keys
still work.

Pre-fix repro: `SET join_algorithm = 'partial_merge'; SELECT v, w FROM lpm INNER JOIN rpm ON lpm.k = rpm.k`
with `(nan, 'nan-l'), (1.5, 'm')` left-side and `(nan, 999), (1.5, 100)`
right-side returns BOTH `m\t100` AND `nan-l\t999` (BUG: NaN matches NaN).
Post-fix: throws Code 48 NOT_IMPLEMENTED. AUTO with float keys correctly
falls back to HashJoin and returns only `m\t100`.

clickhouse-gh[bot] CR id 4442281467 / comment 3367XXX (partial_merge
algorithm gap) on ClickHouse#106540.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…r-join

Per `@vdimir` `CHANGES_REQUESTED` review on PR ClickHouse#106540, restore
the legacy `partial_merge` (`MergeJoin`) behaviour for float `JOIN ON`
keys (`NaN` matches `NaN` bitwise) and document the gotcha instead of
throwing `NOT_IMPLEMENTED`. The previous commit (`b201d205cacc`) was
backward incompatible and rejected existing user queries.

Reverted:
- `MergeJoin::MergeJoin` constructor `NOT_IMPLEMENTED` throw on float
  key types.
- `MergeJoin::isSupported(table_join, right_sample_block)` overload.
- `JoinCommon::joinKeyTypeContainsFloatPayload` helper (now unused).
- Test cases asserting `serverError NOT_IMPLEMENTED` for the four float
  key shapes; the test now verifies that `partial_merge` returns the
  legacy bitwise-equal output.

Added a docs `:::note Float keys` block in the `join_algorithm` setting
description so users can pick a `NaN`-safe algorithm (`hash`,
`parallel_hash`, `grace_hash`, `full_sorting_merge`, `direct`) when it
matters.

The other algorithms in this PR (`HashJoin`, `ConcurrentHashJoin`,
`GraceHashJoin` runtime filter, `DirectKeyValueJoin`,
`FullSortingMergeJoin`) keep their `NaN`-as-null fold and continue to
match IEEE 754 `NaN != NaN`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Address @vdimir review feedback on PR ClickHouse#106540 (the JOIN ON NaN-keys fix
for issue ClickHouse#106531). The previous version carried three near-identical
~80-line recursive NaN-detection cascades (in JoinUtils.cpp,
BuildRuntimeFilterTransform.cpp and MergeJoinTransform.cpp), each
re-implementing the same Nullable/LowCardinality/Tuple/ColumnConst/
ColumnSparse unwrap plus a per-float-type leaf loop.

Replace them with a single per-column primitive: ColumnVector<T>::getNanMask,
which OR-marks NaN rows for floating-point T (no-op otherwise). The shared
JoinCommon recursion (markFloatNaNRowsAsNull / joinKeyContainsFloatPayload)
keeps the wrapper unwrap in one place and calls getNanMask at the leaf;
the two transforms now reuse that shared recursion instead of private copies.

Behaviour is unchanged: NaN keys are still treated as never-matching for
JOIN ON, while null-safe tuple equality (tuple(NULL) = tuple(NULL)) is
preserved because the Nullable branch skips already-NULL rows. Net -250
lines across the four join code paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…erMap)

The generic direct (key-value) JOIN path did not honour IEEE 754 / SQL
JOIN ON NaN != NaN for IKeyValueEntity backends. DirectKeyValueJoin::joinBlock
forwarded the probe key to storage->getByKeys; StorageEmbeddedRocksDB (and
Redis / KeeperMap) serialize the key Field bitwise, so a probe NaN matched a
build-side NaN with the same bit pattern. Only DirectJoinMergeTreeEntity masked
NaN internally, so the direct algorithm overlisted NaN-safety in the
join_algorithm docs for the generic backends.

Fix: fold NaN probe rows to the not-found path inside DirectKeyValueJoin::joinBlock,
gated on offsets.empty() so only the generic backends (which return a 1:1 result
with empty offsets) are touched; DirectJoinMergeTreeEntity already masks NaN and
returns non-empty offsets, so it is left unchanged. Reuses the shared
JoinCommon::markFloatNaNRowsAsNull recursion (Nullable / LowCardinality / Tuple),
so the masking matches the hash / full_sorting_merge paths exactly. NaN rows are
remapped to a default sentinel and null_map=false, mirroring how the MergeTree
entity handles its own NaN rows, which yields the correct shape for INNER (drop),
LEFT (default / NULL), LEFT SEMI (drop), and LEFT ANTI (keep).

Adds 04318_106531_join_on_nan_keys_rocksdb_direct (tagged no-fasttest, use-rocksdb)
covering all four join kinds plus join_use_nulls, Float32 keys, non-NaN regression,
and a String key literally equal to 'nan' (must still match). Split from the parent
04318 test because EmbeddedRocksDB is not in the fast-test build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>