[issue18305] [patch] Fast sum() for non-numbers

Guido van Rossum added the comment: The example code is a perversion and should not be encouraged. -- resolution: -> rejected status: open -> closed ___ Python tracker ___ _

[issue18305] [patch] Fast sum() for non-numbers

Raymond Hettinger added the comment: Guido, do you care to close this one? Terry, has channeled you and thinks it may be against your wishes to turn sum() into a general purpose sequence concatenator. I also seem to recall you expresses similar thoughts back when Alex Martelli first designed

[issue18305] [patch] Fast sum() for non-numbers

Terry J. Reedy added the comment: Sergey, please stop calling the current documented behavior a bug. https://docs.python.org/3/library/functions.html#sum says 'The iterable‘s items are normally numbers ... To concatenate a series of iterables, consider using itertools.chain(). To make a chang

[issue18305] [patch] Fast sum() for non-numbers

Sergey added the comment: > This is not a common case. A common case is creating short tuples and keeping > a lot of tuples in memory. > For fast += you need keep not only a size of tuple, but also a size of > allocated memory. It's a cause of sys.getsizeof([1, 2]) > sys.getsizeof((1, > 2)).

[issue18305] [patch] Fast sum() for non-numbers

Changes by Serhiy Storchaka : -- nosy: +rhettinger ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.

[issue18305] [patch] Fast sum() for non-numbers

Serhiy Storchaka added the comment: > On the contrary, it will reduce memory usage and creation time compared to > regular tuples, because in cases like: > c = a + b > you do not have to spend time and memory for allocating and copying elements > of "a". This is not a common case. A common c

[issue18305] [patch] Fast sum() for non-numbers

Sergey added the comment: > I guess such implementation of tuple will increase memory usage > and creation time which are critical important for tuples. On the contrary, it will reduce memory usage and creation time compared to regular tuples, because in cases like: c = a + b you do not have

[issue18305] [patch] Fast sum() for non-numbers

Serhiy Storchaka added the comment: I guess such implementation of tuple will increase memory usage and creation time which are critical important for tuples. -- ___ Python tracker

[issue18305] [patch] Fast sum() for non-numbers

Sergey added the comment: Attached fasttuple.py is a Proof-of-Concept implementation of tuple, that reuses same data storage when possible. Its possible usage looks similar to built-in tuples: from fasttuple import ft a = ft([1,2]) b = a + ft([3,4]) c = b + ft([5,6]) d = b + ft([7,8])

[issue18305] [patch] Fast sum() for non-numbers

Sergey added the comment: > This "optimisation" is a semantic change. It breaks backward > compatibility in cases where a = a + b and a += b do not result > in the name a having the same value. In particular this breaks > backward compatibility for numpy users. I didn't knew that. Then I guess o

[issue18305] [patch] Fast sum() for non-numbers

Sergey added the comment: Steven D'Aprano noticed that there's an obscure case of: >>> class A(list): ... def __add__(self, other): ... return A(super(A,self).__add__(other)) ... def __radd__(self, other): ... return A(other) + self ... Where: >>> type( [1] + A([2]) )

[issue18305] [patch] Fast sum() for non-numbers

Ethan Furman added the comment: So making this a speed-up for generic objects using __iadd__ is out. The only question remaining is: is it worth special casing a few specific objects (list, tuple, str) to optimise performance? The str question has already been answered: it's special cased to

[issue18305] [patch] Fast sum() for non-numbers

Oscar Benjamin added the comment: This "optimisation" is a semantic change. It breaks backward compatibility in cases where a = a + b and a += b do not result in the name a having the same value. In particular this breaks backward compatibility for numpy users. Numpy arrays treat += differentl

[issue18305] [patch] Fast sum() for non-numbers

Changes by Ethan Furman : -- nosy: +ethan.furman ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue18305] [patch] Fast sum() for non-numbers

Sergey added the comment: This patch implements another solution to the same bug. But instead of changing default code it adds a special case optimization for lists, tuples and strings, similar to those two special cases for numbers, that are already there. === Lists === No patch: $ ./python

[issue18305] [patch] Fast sum() for non-numbers

Sergey added the comment: > I don't know about IronPython, but Jython and PyPy have same behavior as > CPython. Right. I was wrong, at least Jython and PyPy suffer from this bug too. > I think that it is worthwhile to discuss the idea at first in the > Python-Ideas mailing list [1]. Ok, wrot

[issue18305] [patch] Fast sum() for non-numbers

Ramchandra Apte added the comment: I agree with Sergey. Would be nice if it could use copy.copy (though that might change behaviour a bit) -- nosy: +Ramchandra Apte ___ Python tracker _

[issue18305] [patch] Fast sum() for non-numbers

Serhiy Storchaka added the comment: > Looks like this bug is CPython-specific, others (Jython, IronPython...) don't > have it, so people will move code that depends on the internal optimization > to other pythons that DO have it. :) I don't know about IronPython, but Jython and PyPy have same

[issue18305] [patch] Fast sum() for non-numbers

Sergey added the comment: > The issue of quadratic performance of sum(sequences, null_seq) is known I hope it's never too late to fix some bugs... :) > sum([[1,2,3]]*n, []) == [1,2,3]*n == list(chain.from_iterable([[1,2,3]]*n)) But if you already have a list of lists, and you need to join the

[issue18305] [patch] Fast sum() for non-numbers

Serhiy Storchaka added the comment: I agree with Terry. CPython deliberately disallow use sum() with lists of strings. -- ___ Python tracker ___

[issue18305] [patch] Fast sum() for non-numbers

Terry J. Reedy added the comment: Performance enhancements do not normally go in bugfix releases. The issue of quadratic performance of sum(sequences, null_seq) is known, which is why the doc says that sum() is for numbers and recommends .join for strings and itertools.chain for other sequence

[issue18305] [patch] Fast sum() for non-numbers

Changes by STINNER Victor : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://m

[issue18305] [patch] Fast sum() for non-numbers

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue18305] [patch] Fast sum() for non-numbers

New submission from Sergey: Problem === Code: sum([[1,2,3]]*100, []) takes forever to complete. Suggestion == Patch sum() function so that it did not created 100 copies of result, but created just one. Attached patch does that. Before patch: $ ./python -mtimeit --setup="x