[
https://issues.apache.org/jira/browse/SPARK-60118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L. C. Hsieh updated SPARK-60118:
--------------------------------
Affects Version/s: 3.5.8
4.2.0
4.1.3
4.0.4
4.3.0
> Casting a string with an exponent near Int.MaxValue to decimal throws
> ArithmeticException instead of NULL / NUMERIC_OUT_OF_SUPPORTED_RANGE
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-60118
> URL: https://issues.apache.org/jira/browse/SPARK-60118
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.5.8, 4.2.0, 4.3.0, 4.1.3, 4.0.4, 4.4.0
> Reporter: L. C. Hsieh
> Priority: Major
>
> Casting a string whose exponent is near Int.MaxValue to a decimal type fails
> with a raw java.lang.ArithmeticException under all cast modes, instead of
> returning NULL (try_cast / legacy cast) or raising
> NUMERIC_OUT_OF_SUPPORTED_RANGE (ANSI cast):
> {code:sql}
> SELECT try_cast('1e2147483647' AS DECIMAL(10,2));
> {code}
> {code}
> java.lang.ArithmeticException: BigInteger would overflow supported range
> at java.base/java.math.BigInteger.reportOverflow(BigInteger.java:1178)
> at java.base/java.math.BigInteger.pow(BigInteger.java:2627)
> at java.base/java.math.BigDecimal.bigTenToThe(BigDecimal.java:4375)
> at java.base/java.math.BigDecimal.bigMultiplyPowerTen(BigDecimal.java:4510)
> at java.base/java.math.BigDecimal.setScale(BigDecimal.java:2936)
> at java.base/java.math.BigDecimal.setScale(BigDecimal.java:3003)
> at scala.math.BigDecimal.setScale(BigDecimal.scala:592)
> at org.apache.spark.sql.types.Decimal.set(Decimal.scala:155)
> at org.apache.spark.sql.types.Decimal$.apply(Decimal.scala:610)
> at org.apache.spark.sql.types.Decimal$.fromString(Decimal.scala:656)
> {code}
> Decimal.fromString / fromStringANSI fast-fail when the number of integral
> digits (precision - scale of the parsed java.math.BigDecimal) exceeds 38.
> That value is computed as an Int; for '1e2147483647' it is 1 - (-2147483647),
> which overflows to a negative number, so the check is skipped and converting
> the BigDecimal to Decimal throws.
> Other affected inputs: '-1E+2147483647', '12e2147483647', '1.5e2147483647',
> and '1e2147483648' (throws "Underflow"). '0e2147483647' currently returns 0
> while '0e100' returns NULL. The code dates back to SPARK-35841 and is the
> same on all maintained branches.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
