On Wed, Feb 02, 2022 at 07:00:45PM +0100, newbie nullzwei via gnumeric-list wrote:
> 1. could someone please check if it is correct that statfuns.xls in cell > F22 calculates > 15.075000000000001 as average of 33.9; 23.3; -1.3 and 4.4? I don't have access to the spreadsheet just now, but the arithmetic mean of those four values, as calculated by Julia and Python: julia> Statistics.mean([33.9, 23.3, -1.3, 4.4]) 15.075000000000001 A naive calculation in Python: >>> sum([33.9, 23.3, -1.3, 4.4])/4 15.075000000000001 Using high-precision arithmetic on floating point values: >>> statistics.fmean([33.9, 23.3, -1.3, 4.4]) 15.075 And using exact arithmetic (convert the floats to rational numbers; use exact rational arithmetic for the calculation; convert back to float): >>> statistics.mean([33.9, 23.3, -1.3, 4.4]) 15.075 So 15.075 is the exact result, but plus 1 ULP due to rounding errors is acceptable. If your school maths calculates a different value, perhaps you went to a different school than me, or you made a mistake :-) My school maths calculation gives 60.3 as the sum, and dividing by four gives 15.075. -- Steve _______________________________________________ gnumeric-list mailing list gnumeric-list@gnome.org https://mail.gnome.org/mailman/listinfo/gnumeric-list
