https://bz.apache.org/bugzilla/show_bug.cgi?id=66047

--- Comment #4 from Axel Howind <[email protected]> ---
I am not sure if this is really more consistent. As I have said in a comment to
another issue, there is no exact 6.05 and no exact 7.05 in excel as excel
internally uses IEEE754 numbers. Both numbers are represented by their nearest
repesentable neighbour, and the fractional part of both representations ($1 and
$2) is < 0.05, so a consistent solution would round both numbers down. What
happens here is that the errors in the floating point representations of 7.05
($2) and 0.1 ($3) cancel each other out and lead to the result 7.1 ($5 rounded
to one decimal).

You can check this in jshell by using the double constructor of BigDecimal
(note that you should avoid that constructor at all costs for other purposes):


```
    % jshell 
    |  Welcome to JShell -- Version 18.0.1
    |  For an introduction type: /help intro

    jshell> new BigDecimal(6.05)
    $1 ==> 6.04999999999999982236431605997495353221893310546875

    jshell> new BigDecimal(7.05)
    $2 ==> 7.04999999999999982236431605997495353221893310546875

    jshell> new BigDecimal(0.1)
    $3 ==> 0.1000000000000000055511151231257827021181583404541015625

    jshell> new BigDecimal(6.05*0.1)
    $4 ==> 0.604999999999999982236431605997495353221893310546875

    jshell> new BigDecimal(7.05*0.1)
    $5 ==> 0.7050000000000000710542735760100185871124267578125
```

This is the reason why you get 6.0 and 7.1 respectively in Excel.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to