The stateless test 02180_group_by_lowcardinality asserts a row-by-row deterministic output from a query that is inherently non-deterministic. The test currently fails ~10–20 times per day on master under Stateless tests (amd_llvm_coverage, ParallelReplicas, s3 storage, parallel) and has been failing across 112 distinct PRs over the last 30 days.

@alexey-milovidov flagged this on #102039.

Root cause

The query is:

SELECT val, avg(toUInt32(val))
FROM t_group_by_lowcardinality
GROUP BY val
LIMIT 10
SETTINGS max_threads = 1, max_rows_to_group_by = 100, group_by_overflow_mode = 'any';

With group_by_overflow_mode = 'any', only the first max_rows_to_group_by distinct keys make it into the hash table, and which keys those are depends on the order in which rows arrive at the aggregator. LIMIT 10 without ORDER BY then selects ten of those keys in hash-iteration order.

The max_threads = 1 clause does not help under ParallelReplicas (where the table is sharded across replicas and partial aggregations are merged) or under S3 storage (where reads can be split into independent ranges), so the resulting row set varies between runs.

Locally reproduced the exact CI failure diff by changing max_threads = 1 to max_threads = 4: an extra {"val":null,"avg(toUInt32(val))":null} appears at the top and the last row drops off — identical to the diff shown in the CI report.

Fix

This test is a crash-regression test for "Avoid crash in case of GROUP BY LowCardinality(Nullable(String)) column and group_by_overflow_mode='any'" (PR #29637 / #56057). The query is preserved verbatim, including all the problematic settings; only the assertion is changed to SELECT count() FROM (...), which is always 10 (the LIMIT value).

If the underlying crash ever returns, or LIMIT stops returning the expected number of rows, the test will detect it. The specific row values were never deterministic to begin with.

Originally tracked in #36069 (2022). The -- Tags: no-random-settings workaround masked the issue for the random-settings randomization but the underlying non-determinism returned once ParallelReplicas was enabled by default in some CI configurations.

CI report (one of many): https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=102039&sha=eb773718e41db7bafe973e9a67a6a3a195b48fe9&name_0=PR&name_1=Stateless%20tests%20%28amd_llvm_coverage%2C%20ParallelReplicas%2C%20s3%20storage%2C%20parallel%29

Closes #36069

Changelog category (leave one):

• CI Fix or Improvement (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Not for changelog (CI Fix or Improvement).

Documentation entry for user-facing changes

• Documentation is unchanged.

Version info

• Merged into: 26.5.1.742