{{ message }}
Repository navigation
Commit 57a4c8e
authored
fix: reject decimal promotion that changes the scale (apache#3613)
* fix: reject decimal promotion that changes the scale
The DecimalType handler in promote() guarded scale equality with
`file_type.scale == file_type.scale`, which compares the file scale to
itself and is always true. As a result any decimal-to-decimal promotion
with a widening precision was accepted regardless of the scale.
Per the Iceberg spec a decimal may only be promoted when the scale is
unchanged and the precision widens (decimal(P, S) to decimal(P2, S) with
P2 > P), matching TypeUtil.isPromotionAllowed in the Java implementation.
The bug affected both paths that use promote(): on read a differing-scale
promotion built a reader at the wrong scale and reinterpreted the stored
unscaled integers (silent data corruption), and on write it let a
DataFrame column with a different scale pass the compatibility check.
Compare the file scale to the read scale instead. The identical tautology
in the test oracle masked the defect, so fix it too and add a decimal with
a different scale to the promotion matrix, plus explicit regression tests
for the schema and read paths.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
* test: simplify decimal promotion coverage
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
---------
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>1 parent 6285004 commit 57a4c8e
3 files changed
Lines changed: 28 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
256 | 256 | | |
257 | 257 | | |
258 | 258 | | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
259 | 268 | | |
260 | 269 | | |
261 | 270 | | |
| |||

0 commit comments