sepuri sai krishna created SPARK-59686:
------------------------------------------
Summary: round and bround fail for a large rounding scale instead
of returning the rounded value
Key: SPARK-59686
URL: https://issues.apache.org/jira/browse/SPARK-59686
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.0.4, 4.1.3, 4.2.0, 3.5.8, 4.3.0
Reporter: sepuri sai krishna
h3. Problem
{{round}} and {{bround}} pass the user supplied scale straight to
{{BigDecimal.setScale}}, which
throws once the magnitude passes roughly 1e9. Neighbouring scales return the
result fine, so a
well defined answer becomes an error:
{code:sql}
SELECT round(1.5, -10000000); -- 0.0
SELECT round(1.5, -1000000000); -- java.lang.ArithmeticException
SELECT round(1.5, 100000); -- 1.5
SELECT round(1.5, 2147483647); -- java.lang.ArithmeticException
{code}
The answers are not in doubt. A scale far to the left of the decimal point
rounds any finite value
to zero, and a scale past the digits a value carries leaves it unchanged, which
is exactly what the
smaller scales above return.
h3. Two symptoms, one cause
* On the ANSI integral path the exception is wrapped by
{{MathUtils.withOverflow}} and surfaces as
{{ARITHMETIC_OVERFLOW}}. That is spurious: nothing overflowed, and the result
{{0}} is
representable.
* Everywhere else, so {{FLOAT}}, {{DOUBLE}}, {{DECIMAL}} with a negative scale,
and the integral
types in non-ANSI mode, the raw {{java.lang.ArithmeticException}} reaches the
user. It is not a
{{SparkThrowable}}, so it carries no error condition and no SQLSTATE.
h3. Also affected
{{RoundBase.dataType}} computes {{-_scale + 1}} for a negative scale. For
{{Int.MinValue}} the
negation overflows, so a {{DECIMAL}} input gets a narrower result precision
than intended.
h3. Suggested fix
Clamp the scale once, where it is read. {{_scale}} is a single {{protected lazy
val}} that every
type branch uses and that codegen interpolates, so clamping there covers
interpreted and generated
code for all numeric types.
Clamping cannot change a result. No finite value has more than 309 integral
digits, so any scale at
or below the negative bound rounds to zero, and the exact decimal expansion of
a finite double needs
at most 1074 fractional digits, so any scale at or above the positive bound
leaves the value
unchanged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
