Fix UBSAN signed integer negation in countDigits for decimal values#103225
Conversation
|
cc @rschu1ze @alexey-milovidov — could you review this? It's a small UBSAN fix for |
|
Workflow [PR], commit [c25d1cc] Summary: ❌
AI ReviewSummaryThis PR fixes undefined behavior in Final VerdictStatus: ✅ Approve |
|
@groeneai, we fixed an intermittent failure of performance tests, please update the branch. |
The decimal branch of `countDigits::execute` computed the absolute value of a
negative decimal by applying unary minus directly to the signed native integer
(`-value`) before casting to unsigned. When the underlying value equals the
signed minimum (for example `INT32_MIN` for `Decimal32`), this triggers
undefined behavior: the result cannot be represented in the signed type. The
integer (non-decimal) branch a few lines below already used the correct
cast-to-unsigned-then-negate pattern.
The fuzzer caught this on master as STID 1751-507c in the `AST fuzzer
(arm_asan_ubsan)` check — UBSAN flagged `negation of -2147483648 cannot be
represented in type 'int'` at `src/Functions/countDigits.cpp:138`. Over the
last 30 days the same diagnostic appeared on master and on two unrelated PR
runs, all pointing at the same line.
The fix is to cast the signed value to `NativeT` (the unsigned equivalent of
the column's native integer type) before applying unary minus, so that the
arithmetic is well-defined modular arithmetic rather than undefined signed
overflow. The outer cast back to `NativeT` suppresses the
`-Wimplicit-int-conversion-on-negation` diagnostic that fires for narrower
types, which are subject to integer promotion to `int` before the unary minus.
Both the decimal and integer branches now use the same symmetric pattern.
A regression test `04107_count_digits_int_min_ubsan.sql` constructs
`Decimal{32,64,128,256}` columns with the signed minimum in their underlying
native integer via `reinterpret` (the normal `CAST`/`toDecimalXX` paths reject
values outside the precision range), which exercises the previously UB-prone
branch for every decimal width and the symmetric `Int8`/`Int16`/.../`Int256`
path.
Relevant CI report:
https://s3.amazonaws.com/clickhouse-test-reports/json.html?REF=master&sha=56bc5847a60449e7339fcdeecc284c19d276ab06&name_0=MasterCI&name_1=AST%20fuzzer%20%28arm_asan_ubsan%29
4158ed5 to
bb11cb5
Compare
|
@alexey-milovidov Rebased onto current master |
|
This was fixed by #105146. Let's update the branch. |
LLVM Coverage ReportChanged lines: 100.00% (8/8) · Uncovered code |
…ClickHouse#103225) The decimal branch of `countDigits::execute` computed the absolute value of a negative decimal by applying unary minus directly to the signed native integer (`-value`) before casting to unsigned. When the underlying value equals the signed minimum (for example `INT32_MIN` for `Decimal32`), this triggers undefined behavior: the result cannot be represented in the signed type. The integer (non-decimal) branch a few lines below already used the correct cast-to-unsigned-then-negate pattern. The fuzzer caught this on master as STID 1751-507c in the `AST fuzzer (arm_asan_ubsan)` check — UBSAN flagged `negation of -2147483648 cannot be represented in type 'int'` at `src/Functions/countDigits.cpp:138`. Over the last 30 days the same diagnostic appeared on master and on two unrelated PR runs, all pointing at the same line. The fix is to cast the signed value to `NativeT` (the unsigned equivalent of the column's native integer type) before applying unary minus, so that the arithmetic is well-defined modular arithmetic rather than undefined signed overflow. The outer cast back to `NativeT` suppresses the `-Wimplicit-int-conversion-on-negation` diagnostic that fires for narrower types, which are subject to integer promotion to `int` before the unary minus. Both the decimal and integer branches now use the same symmetric pattern. A regression test `04107_count_digits_int_min_ubsan.sql` constructs `Decimal{32,64,128,256}` columns with the signed minimum in their underlying native integer via `reinterpret` (the normal `CAST`/`toDecimalXX` paths reject values outside the precision range), which exercises the previously UB-prone branch for every decimal width and the symmetric `Int8`/`Int16`/.../`Int256` path. Relevant CI report: https://s3.amazonaws.com/clickhouse-test-reports/json.html?REF=master&sha=56bc5847a60449e7339fcdeecc284c19d276ab06&name_0=MasterCI&name_1=AST%20fuzzer%20%28arm_asan_ubsan%29 Co-authored-by: Groene AI <270696204+groeneai@users.noreply.github.com> Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>

The decimal branch of
countDigits::executecomputed the absolute value of a negative decimal by applying unary minus directly to the signed native integer (-value) before casting to unsigned. When the underlying value equals the signed minimum (for exampleINT32_MINforDecimal32), this triggers undefined behavior — the result cannot be represented in the signed type. The integer (non-decimal) branch a few lines below already used the correct cast-to-unsigned-then-negate pattern.The fuzzer caught this on master as STID 1751-507c in the
AST fuzzer (arm_asan_ubsan)check — UBSAN flagged:Over the last 30 days the same diagnostic appeared on master and on two unrelated PR runs, all pointing at the same line.
Relevant CI report:
https://s3.amazonaws.com/clickhouse-test-reports/json.html?REF=master&sha=56bc5847a60449e7339fcdeecc284c19d276ab06&name_0=MasterCI&name_1=AST%20fuzzer%20%28arm_asan_ubsan%29
Fix
Cast the signed value to
NativeT(the unsigned equivalent of the column's native integer type) before applying unary minus, so the arithmetic is well-defined modular arithmetic rather than undefined signed overflow. The outer cast back toNativeTsuppresses the-Wimplicit-int-conversion-on-negationdiagnostic that fires for narrower types, which are subject to integer promotion tointbefore the unary minus. Both the decimal and integer branches now use the same symmetric pattern.Regression test
04107_count_digits_int_min_ubsan.sqlconstructsDecimal{32,64,128,256}columns with the signed minimum planted into their underlying native integer viareinterpret— the normalCAST/toDecimalXXpaths reject values outside the precision range, soreinterpretis the only way to exercise the fuzzer-discovered path from SQL. The test also exercises the symmetricInt8/Int16/.../Int256paths at their signed minima to guard against regressions in the integer branch.Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
Documentation entry for user-facing changes
Version info
26.5.1.790