Fix UBSan signed integer overflow in `Decimal` arithmetic operators (STID 0937-2022) by groeneai · Pull Request #105158 · ClickHouse/ClickHouse · GitHub
Skip to content

Fix UBSan signed integer overflow in Decimal arithmetic operators (STID 0937-2022)#105158

Merged
alexey-milovidov merged 6 commits into
ClickHouse:masterfrom
groeneai:groeneai/fix-decimal-ubsan-stid-0937-2022
May 31, 2026
Merged

Fix UBSan signed integer overflow in Decimal arithmetic operators (STID 0937-2022)#105158
alexey-milovidov merged 6 commits into
ClickHouse:masterfrom
groeneai:groeneai/fix-decimal-ubsan-stid-0937-2022

Conversation

@groeneai

@groeneai groeneai commented May 17, 2026

Copy link
Copy Markdown
Contributor

The AST fuzzer (arm_asan_ubsan) hit signed integer overflow inside
Decimal<long>::operator*= / operator+= in base/base/Decimal.cpp
three times in four days (master 2026-05-13, PR #100177, PR #104545; UBSan
STID 0937-2022):

/ClickHouse/base/base/Decimal.cpp:24:88: runtime error: signed integer
overflow: 1921680119216801192 * 10 cannot be represented in type 'long'

#0 DB::Decimal<long>::operator*=(long const&)   base/base/Decimal.cpp:24
#1 DB::readDigits<false, DB::Decimal<long>>     src/IO/readDecimalText.h:108
#2 DB::readDecimalText<DB::Decimal<long>, bool> src/IO/readDecimalText.h:155
#3 DB::tryReadDecimalText<DB::Decimal<long>>    src/IO/readDecimalText.h:199
#4 DB::normalizeParameter(...)                  src/AggregateFunctions/TimeSeries/AggregateFunctionTimeseriesHelpers.cpp:58

The trigger path is the timeseries aggregate-function parameter parser,
which calls tryReadDecimalText(buf, value, 20, scale) with a string-typed
parameter and Decimal<Int64> as the underlying type. The digit
accumulator x *= 10; x += digit; overflows Int64 once the partial value
crosses 2^63.

Decimal<T>'s operator overloads are unchecked by design — ClickHouse
relies on well-defined two's-complement wrap-around in many code paths
(digit accumulation, running sums in aggregates, scale-multiplier
conversion). Code paths that need overflow detection use the explicit
helpers in base/arithmeticOverflow.h (common::addOverflow,
common::mulOverflow, etc.). The bug is that the wrap-around is technically
undefined behaviour at the C++ language level, so UBSan flags it.

Mark Decimal<T>'s compound-assignment operators, cross-type
compound-assignment operators, and free-function arithmetic operators
(+, -, *, /, unary -) with NO_SANITIZE_UNDEFINED. This matches
the existing pattern on the sibling addOverflow and negateOverflow
helpers in the same file (which were added previously for exactly this
purpose in sumWithOverflow and friends). No behaviour change in release
builds — the wrap was already happening; UBSan just no longer flags it.

Add a regression test 04254_decimal_parsing_ubsan_stid_0937_2022.sql
that exercises the readDecimalText path through
timeSeriesResampleToGridWithStaleness with 19- and 20-digit string
parameters, plus a happy-path regression guard.

Verified locally on build-asan (clang-21,
-DSANITIZE=address,undefined): before the fix, the test query aborts
with the UBSan diagnostic above; after the fix the same query returns
BAD_ARGUMENTS cleanly (the post-parse common::mulOverflow check in
tryReadDecimalText rejects the value). 20/20 randomized runs of the new
test pass; existing decimal (00700_decimal_arithm, _aggregates,
_math) and timeseries (02538_nullable_array_tuple_timeseries,
03222_create_timeseries_table, 03254_timeseries_functions*,
04106_timeseries_bucket_count_overflow) tests continue to pass.

CI report (one of three observations): https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=104545&sha=9004589d3c0c3df141a51bc5525f980149223295&name_0=PR&name_1=AST%20fuzzer%20%28arm_asan_ubsan%29

Changelog category (leave one):

  • CI Fix or Improvement (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Not for changelog. Suppress UBSan signed-integer-overflow reports inside Decimal<T> arithmetic operator overloads (+=, -=, *=, /=, %=, +, -, *, /, unary -). The operators already had wrap-around semantics; callers that need overflow detection use the explicit helpers in base/arithmeticOverflow.h. Matches the pre-existing NO_SANITIZE_UNDEFINED attribute on sibling addOverflow and negateOverflow helpers.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

session-id: cron:clickhouse-ci-task-worker:20260517-143000


Note

Medium Risk
Touches core Decimal<T> arithmetic operators by disabling UBSan undefined-behavior checks for +/-/* paths; while intended to preserve existing wrap-around semantics, it could mask real overflow-related bugs in sanitized builds if misapplied.

Overview
Suppresses UBSan signed-integer-overflow reports in Decimal<T> by marking wrap-around arithmetic operators (+=, -=, *=, cross-type variants, free +, -, *, and unary -) as NO_SANITIZE_UNDEFINED, while intentionally leaving ///=/%= instrumented to still catch divide-by-zero and INT_MIN / -1.

Adds a stateless regression test (04266_decimal_parsing_ubsan_stid_0937_2022) that exercises decimal digit accumulation via timeSeriesResampleToGridWithStaleness with oversized string parameters (expecting BAD_ARGUMENTS) plus a normal-range sanity check.

Reviewed by Cursor Bugbot for commit f9107ed. Bugbot is set up for automated code reviews on this repo. Configure here.

Version info

  • Merged into: 26.6.1.290

…STID 0937-2022)

`Decimal<T>`'s arithmetic operator overloads in `base/base/Decimal.cpp` and
`base/base/Decimal.h` perform unchecked arithmetic on the underlying integer
type. When the digit accumulator inside `readDigits` (`src/IO/readDecimalText.h`)
or a running aggregate sum crosses `2^63`, the implicit `value *= x` /
`value += x` overflows `Int64`, which UBSAN flags as undefined behaviour:

    /ClickHouse/base/base/Decimal.cpp:24:88: runtime error: signed integer
    overflow: 1921680119216801192 * 10 cannot be represented in type 'long'

Observed three times in four days on the AST fuzzer (`arm_asan_ubsan`):
master 2026-05-13, PR ClickHouse#100177, PR ClickHouse#104545 (STID 0937-2022). The trigger
path is `tryReadDecimalText(buf, value, 20, scale)` in
`AggregateFunctionTimeseriesHelpers::normalizeParameter` — parsing a
string-typed timeseries parameter into a `Decimal<Int64>` with precision 20.

ClickHouse already relies on well-defined two's-complement wrap-around in
these operators: digit accumulation in `readDigits`, running sums in
aggregate functions, scale-multiplier conversions, etc. Code paths that
need overflow detection use the explicit helpers in
`base/arithmeticOverflow.h` (`common::addOverflow`, `common::mulOverflow`,
etc.) — the operator overloads themselves are intentionally wrap-around.

Mark `Decimal<T>`'s compound-assignment operators, cross-type
compound-assignment operators, and free-function arithmetic operators
(`+`, `-`, `*`, `/`, unary `-`) with `NO_SANITIZE_UNDEFINED`. This matches
the existing pattern on the sibling `addOverflow` and `negateOverflow`
helpers in the same file. No behaviour change in release builds; UBSAN
no longer flags the wrap.

Add regression test `04254_decimal_parsing_ubsan_stid_0937_2022.sql` that
exercises the `readDecimalText` path via `timeSeriesResampleToGridWithStaleness`
with 19- and 20-digit string parameters, plus a happy-path case.

Verified locally on `build-asan` (clang-21, `-DSANITIZE=address,undefined`):
before the fix UBSAN aborted with the same diagnostic; after the fix the
same query returns `BAD_ARGUMENTS` cleanly. 20/20 randomized runs of the
new test pass; existing decimal and timeseries tests continue to pass.

CI report: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=104545&sha=9004589d3c0c3df141a51bc5525f980149223295&name_0=PR&name_1=AST%20fuzzer%20%28arm_asan_ubsan%29
@groeneai

Copy link
Copy Markdown
Contributor Author

@groeneai

Copy link
Copy Markdown
Contributor Author

cc @alexey-milovidov @yariks5s — could you review this when you have a moment? The fix is the same shape as the existing NO_SANITIZE_UNDEFINED on addOverflow and negateOverflow in this file, extended to the implicit operator overloads. UBSan-only impact in sanitizer builds (3 master+PR hits over 4 days, AST fuzzer); no behaviour change in release.

session-id: cron:clickhouse-ci-task-worker:20260517-143000

@alexey-milovidov alexey-milovidov added the can be tested Allows running workflows for external contributors label May 17, 2026
@clickhouse-gh

clickhouse-gh Bot commented May 17, 2026

Copy link
Copy Markdown
Contributor

Workflow [PR], commit [067b5de]

Summary:


AI Review

Summary

This PR suppresses UBSan signed-overflow reports for the overflow-dependent Decimal<T> arithmetic operators while leaving division and modulo instrumented, and adds a stateless regression test for the readDecimalText path through timeSeriesResampleToGridWithStaleness. The current code addresses the earlier review thread about over-broad suppression on / and %; I did not find any remaining correctness, safety, or test-coverage findings.

Final Verdict

Status: ✅ Approve

@clickhouse-gh clickhouse-gh Bot added the pr-ci label May 17, 2026
Comment thread base/base/Decimal.h Outdated
Bot review on PR ClickHouse#105158 (`base/base/Decimal.h:72`) pointed out that the
previous commit applied `NO_SANITIZE_UNDEFINED` too broadly: `operator/=`,
`operator%=` and the free `operator/` do not rely on wrap-around semantics,
so suppressing UBSan on them also disables `divide-by-zero` and
`INT_MIN / -1` diagnostics inside `Decimal` arithmetic.

Restrict the attribute to the operators that genuinely depend on
two's-complement wrap — `operator+=`, `operator-=`, `operator*=` and the
corresponding cross-type / free / unary `-` variants — and drop it from
`/=`, `%=` and the free `/`. `addOverflow` and `negateOverflow` keep the
attribute as before. Update the rationale comments in `Decimal.h` and
`Decimal.cpp` to document the narrower contract.

Regression test `04254_decimal_parsing_ubsan_stid_0937_2022.sql` still
exercises the wrap-around path through `tryReadDecimalText` (i.e. the
trigger of STID 0937-2022, which goes through `value *= 10` and
`value += digit`) and continues to pass under
`build-asan` (`-DSANITIZE=address,undefined`, 10/10 randomized runs).
Decimal `/`, `%` and `*` smoke checks return the expected values.
@groeneai

Copy link
Copy Markdown
Contributor Author

@clickhouse-gh Good catch — fixed in 355717a.

NO_SANITIZE_UNDEFINED now stays only on the overflow-dependent operators (+=, -=, *=, free + / - / *, unary -, and the existing addOverflow / negateOverflow helpers). /=, %= and the free / are no longer suppressed, so UBSan continues to report divide-by-zero and INT_MIN / -1 inside Decimal arithmetic.

The rationale comments in Decimal.h and Decimal.cpp are updated to document the narrower contract.

Verified on build-asan (clang-21, -DSANITIZE=address,undefined):

  • Regression test 04254_decimal_parsing_ubsan_stid_0937_2022.sql — the trigger for STID 0937-2022 goes through value *= 10 / value += digit in readDecimalText, both still marked NO_SANITIZE_UNDEFINED, so it continues to pass (10/10 runs with randomized settings).
  • SELECT toDecimal64(7,0) / toDecimal64(3,0) and friends return the expected results.

Session id: cron:clickhouse-ci-task-worker:20260517-171500

@alexey-milovidov alexey-milovidov self-assigned this May 17, 2026
…void collision

Master merged `04254_sqlite_escape_quote_with_quote_103860_roundtrip` while
this PR was open, claiming the `04254_` prefix that the regression test
added in `f319a4908bc` was using. Bump to the next free number (`04266_`).

No test-content changes.
@clickhouse-gh clickhouse-gh Bot added the manual approve Manual approve required to run CI label May 23, 2026
The prior commit deleted/modified workflow files in this branch as a
workaround for GitHub's workflow-scope check. Restoring them to match
`master` so the PR diff no longer contains unrelated changes.
@antonio2368

antonio2368 commented May 28, 2026

Copy link
Copy Markdown
Member
ERROR: Failed to insert data into CI DB, exception [HTTPSConnectionPool(host='play.clickhouse.com', port=443): Max retries exceeded with 
                      url: /?database=default&query=INSERT+INTO+checks+FORMAT+JSONEachRow&date_time_input_format=best_effort&send_logs_level=warning (Caused by 
                      NewConnectionError("HTTPSConnection(host='play.clickhouse.com', port=443): Failed to establish a new connection: [Errno 61] Connection 
                      refused"))] (from: Fast test (arm_darwin))

Even on retry. I guess we can override and merge this as everything else is green?

@alexey-milovidov alexey-milovidov removed the manual approve Manual approve required to run CI label May 31, 2026
@clickhouse-gh

clickhouse-gh Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

LLVM Coverage Report

Metric Baseline Current Δ
Lines 84.40% 84.40% +0.00%
Functions 91.40% 91.40% +0.00%
Branches 77.00% 77.00% +0.00%

Changed lines: Changed C/C++ lines covered by tests: 12/18 (66.67%) | Lost baseline coverage: none · Uncovered code

Full report · Diff report

@alexey-milovidov alexey-milovidov merged commit e71e5e2 into ClickHouse:master May 31, 2026
165 of 166 checks passed
@robot-ch-test-poll2 robot-ch-test-poll2 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

can be tested Allows running workflows for external contributors pr-ci pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants