[issue25177] OverflowError in statistics.mean when summing large floats

Wolfgang Maier added the comment: Steven's commit here also fixed issue 24068. -- nosy: +wolma ___ Python tracker ___ ___ Python-bug

[issue25177] OverflowError in statistics.mean when summing large floats

Changes by Berker Peksag : -- resolution: -> fixed stage: -> resolved status: open -> closed type: -> behavior ___ Python tracker ___ _

[issue25177] OverflowError in statistics.mean when summing large floats

Roundup Robot added the comment: New changeset a7d2307055e7 by Victor Stinner in branch 'default': Null merge 3.5, patch was already applied to default (isuse #25177) https://hg.python.org/cpython/rev/a7d2307055e7 -- ___ Python tracker

[issue25177] OverflowError in statistics.mean when summing large floats

Steven D'Aprano added the comment: Larry, Is it too late to get this into 3.5rc1? changeset 99407:ed45a09e5a69 Thanks. -- nosy: +larry ___ Python tracker ___ _

[issue25177] OverflowError in statistics.mean when summing large floats

Roundup Robot added the comment: New changeset 4bc9405c4f7b by Steven D'Aprano in branch '3.4': Fix for issue #25177 with the mean of very small and very large numbers. https://hg.python.org/cpython/rev/4bc9405c4f7b New changeset ed45a09e5a69 by Steven D'Aprano in branch '3.5': Fixed issue #2517

[issue25177] OverflowError in statistics.mean when summing large floats

Bar Harel added the comment: Anyway, yes, it should be quite the same. I can provide some benchmarks tomorrow if you wish. -- ___ Python tracker ___

[issue25177] OverflowError in statistics.mean when summing large floats

Mark Dickinson added the comment: Note that the two input values given in the original report are not the same: [8.988465674311579e+307, 8.98846567431158e+307] != [8.988465674311579e+307] * 2. -- ___ Python tracker

[issue25177] OverflowError in statistics.mean when summing large floats

Mark Dickinson added the comment: I can reproduce here (OS X 10.9, Python 3.5), exactly as described in the original post. Python 3.5.0 (default, Sep 22 2015, 18:26:54) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more in

[issue25177] OverflowError in statistics.mean when summing large floats

Steven D'Aprano added the comment: > Has anyone confirmed that this bug actually exists? Confirmed. The initial report is not quite correct: you need three values to trigger the overflow, not two: py> x = 8.988465674311579e+307 py> statistics.mean([x]*2) == x True py> statistics.mean([x]*3) =

[issue25177] OverflowError in statistics.mean when summing large floats

Steven D'Aprano added the comment: Has anyone confirmed that this bug actually exists? I'm afraid that I cannot verify it. I get these results on three different computers: py> x = 8.988465674311579e+307 py> statistics.mean([x, x]) 8.988465674311579e+307 py> statistics.mean([x, x]) == x True r

[issue25177] OverflowError in statistics.mean when summing large floats

Mark Dickinson added the comment: > I strongly suspect that moving from float to Fraction-based ratios is going > to kill performance in the common case The existing code already converts each of the input items to Fraction; the only difference is that the old code converts the sum of those Fr

[issue25177] OverflowError in statistics.mean when summing large floats

Josh Rosenberg added the comment: Do you have any benchmarks on the before and after? I strongly suspect that moving from float to Fraction-based ratios is going to kill performance in the common case, particularly for longer input sequences, but that's a hunch only. -- nosy: +josh.r

[issue25177] OverflowError in statistics.mean when summing large floats

Changes by Steven D'Aprano : -- assignee: -> steven.daprano ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: ht

[issue25177] OverflowError in statistics.mean when summing large floats

Steven D'Aprano added the comment: On Sat, Oct 10, 2015 at 04:28:22PM +, Bar Harel wrote: > Any comments on the patch? Not yet, I've been unable to look at it, but thank you. If I haven't responded by the 17th of this month, please ping me again. -- ___

[issue25177] OverflowError in statistics.mean when summing large floats

Bar Harel added the comment: Any comments on the patch? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http

[issue25177] OverflowError in statistics.mean when summing large floats

Bar Harel added the comment: Alright, this patch passed all tests. I've changed the function sum to include a new parameter called fraction which decides whether to return a fraction or not. It might be useful in more scenarios due to the fact fractions are more accurate. I'm still unsure why n

[issue25177] OverflowError in statistics.mean when summing large floats

Bar Harel added the comment: Alright, I issued a fix, now testing it -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsub

[issue25177] OverflowError in statistics.mean when summing large floats

Steven D'Aprano added the comment: Bar, thanks for the time you put into diagnosing this error, it is definitely a bug. The intention is for mean([huge, huge]) to return huge, not raise OverflowError. I'm reluctant to say that mean() will *never* raise OverflowError, but it certainly shouldn'

[issue25177] OverflowError in statistics.mean when summing large floats

Mark Dickinson added the comment: That patch doesn't really help, I'm afraid, since it introduces problems at the other end of the floating-point range: for example, `mean([5e-324, 5e-324])` would give `0.0` with that patch (instead of the `5e-324` it currently gives). So currently, when compu

[issue25177] OverflowError in statistics.mean when summing large floats

Bar Harel added the comment: Seems like this is the only viable option. It fixes the OverflowError but comes at the cost of precision and time. Another option would be to try/except the overflow error and only then return the slower method. Regarding the data [8.988465674311579e+307, 8.98846567

[issue25177] OverflowError in statistics.mean when summing large floats

Bar Harel added the comment: Alright, Seems like the problem is bigger than I thought. PEP 238 (https://www.python.org/dev/peps/pep-0238/) mentions that problem. Using the new division, // will cause a floor while / will cause an OverflowError. An ugly way around it involves type checking or re

[issue25177] OverflowError in statistics.mean when summing large floats

Bar Harel added the comment: Yup, it indeed fixes the problem. Sorry for thinking it's intended. Seems like it's a small problem but it does affects all the uses of Integral or Rational. I'll test it with the suite hoping it has a sufficient coverage. -- __

[issue25177] OverflowError in statistics.mean when summing large floats

Bar Harel added the comment: Whoop! I see the reason for it now. By limit I don't mean the precision limit, I mean the top limit in which float converts to "inf". Seems like this bug is due to the change of python 3's division operator. Under numbers it states: "It's important that this conversi

[issue25177] OverflowError in statistics.mean when summing large floats

David MacIver added the comment: I'm not sure what you mean by float having a limit here. It's certainly finite precision, but there is still a representable value with that finite precision closest to the mean. As an example where there is an obvious correct answer that will trigger this err

[issue25177] OverflowError in statistics.mean when summing large floats

Bar Harel added the comment: I believe it's an intended behavior as python's float has a limit after all. It's hard to reach it but definitely possible. A workaround is to internally use Decimal (also take the advantage that it's implementation is now way faster) but it will cause a precision l

[issue25177] OverflowError in statistics.mean when summing large floats

Changes by Berker Peksag : -- nosy: +steven.daprano ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mai

[issue25177] OverflowError in statistics.mean when summing large floats

New submission from David MacIver: The following code produces an OverflowError: import statistics statistics.mean([8.988465674311579e+307, 8.98846567431158e+307]) The error is: File "/home/david/.pyenv/versions/3.5.0/lib/python3.5/statistics.py", line 293, in mean return _sum(data)/n