L. C. Hsieh created SPARK-60118:
-----------------------------------
Summary: Casting a string with an exponent near Int.MaxValue to
decimal throws ArithmeticException instead of NULL /
NUMERIC_OUT_OF_SUPPORTED_RANGE
Key: SPARK-60118
URL: https://issues.apache.org/jira/browse/SPARK-60118
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.4.0
Reporter: L. C. Hsieh
Casting a string whose exponent is near Int.MaxValue to a decimal type fails
with a raw java.lang.ArithmeticException under all cast modes, instead of
returning NULL (try_cast / legacy cast) or raising
NUMERIC_OUT_OF_SUPPORTED_RANGE (ANSI cast):
{code:sql}
SELECT try_cast('1e2147483647' AS DECIMAL(10,2));
{code}
{code}
java.lang.ArithmeticException: BigInteger would overflow supported range
at java.base/java.math.BigInteger.reportOverflow(BigInteger.java:1178)
at java.base/java.math.BigInteger.pow(BigInteger.java:2627)
at java.base/java.math.BigDecimal.bigTenToThe(BigDecimal.java:4375)
at java.base/java.math.BigDecimal.bigMultiplyPowerTen(BigDecimal.java:4510)
at java.base/java.math.BigDecimal.setScale(BigDecimal.java:2936)
at java.base/java.math.BigDecimal.setScale(BigDecimal.java:3003)
at scala.math.BigDecimal.setScale(BigDecimal.scala:592)
at org.apache.spark.sql.types.Decimal.set(Decimal.scala:155)
at org.apache.spark.sql.types.Decimal$.apply(Decimal.scala:610)
at org.apache.spark.sql.types.Decimal$.fromString(Decimal.scala:656)
{code}
Decimal.fromString / fromStringANSI fast-fail when the number of integral
digits (precision - scale of the parsed java.math.BigDecimal) exceeds 38. That
value is computed as an Int; for '1e2147483647' it is 1 - (-2147483647), which
overflows to a negative number, so the check is skipped and converting the
BigDecimal to Decimal throws.
Other affected inputs: '-1E+2147483647', '12e2147483647', '1.5e2147483647', and
'1e2147483648' (throws "Underflow"). '0e2147483647' currently returns 0 while
'0e100' returns NULL. The code dates back to SPARK-35841 and is the same on all
maintained branches.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
