ashokraminedi commented on code in PR #20130:
URL: https://github.com/apache/hudi/pull/20130#discussion_r4137769603
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/utils/SparkInternalSchemaConverter.java:
##########
@@ -349,9 +349,13 @@ private static boolean
convertIntLongType(WritableColumnVector oldV, WritableCol
} else if (newType instanceof StringType) {
newV.putByteArray(i, getUTF8Bytes((isInt ? oldV.getInt(i) :
oldV.getLong(i)) + ""));
} else if (newType instanceof DecimalType) {
+ DecimalType decimalType = (DecimalType) newType;
Decimal oldDecimal = Decimal.apply(isInt ? oldV.getInt(i) :
oldV.getLong(i));
- oldDecimal.changePrecision(((DecimalType) newType).precision(),
((DecimalType) newType).scale());
- newV.putDecimal(i, oldDecimal, ((DecimalType) newType).precision());
+ if (oldDecimal.changePrecision(decimalType.precision(),
decimalType.scale())) {
Review Comment:
Good point. Updated in `889b09ac6c27`.
I pulled the precision check into a shared `putDecimalOrNull(newV, rowId,
decimal, decimalType)` helper and now use it for INT/LONG, FLOAT, DOUBLE, and
STRING -> DECIMAL conversions.
The helper checks the result of `changePrecision()` before writing to the
target vector. On failure it writes NULL when ANSI mode is disabled, and throws
when ANSI mode is enabled.
I left the DECIMAL -> DECIMAL path unchanged since, as you noted, that
evolution is widening-only.
While checking the neighboring conversion paths, I also reproduced the
FLOAT, DOUBLE, and STRING overflow cases and captured them in #20143.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
