subject:"\[issue12805\] Optimizations for bytes.join et. al"

[issue12805] Optimizations for bytes.join() et. al

2012-10-20 Thread John O'Connor

John O'Connor added the comment: I think that's a good compromise and much better than what I had originally. - John -- ___ Python tracker ___ __

[issue12805] Optimizations for bytes.join() et. al

2012-10-20 Thread Antoine Pitrou

Antoine Pitrou added the comment: I have committed the simple optimization patch, thank you! -- resolution: -> fixed stage: -> committed/rejected status: open -> closed ___ Python tracker

[issue12805] Optimizations for bytes.join() et. al

2012-10-20 Thread Roundup Robot

Roundup Robot added the comment: New changeset bfa715f98c0f by Antoine Pitrou in branch 'default': Issue #12805: Make bytes.join and bytearray.join faster when the separator is empty. http://hg.python.org/cpython/rev/bfa715f98c0f -- nosy: +python-dev ___

[issue12805] Optimizations for bytes.join() et. al

2012-10-20 Thread Serhiy Storchaka

Changes by Serhiy Storchaka : Added file: http://bugs.python.org/file27636/bytes_join_optimization_3.patch ___ Python tracker ___ ___ Python-b

[issue12805] Optimizations for bytes.join() et. al

2012-10-20 Thread Serhiy Storchaka

Serhiy Storchaka added the comment: Indeed, 1-char separator is more common for strings, but I found several b'\n'.join or b','.join even in stdlib. 3 lines of difference are only 2.3% of Objects/stringlib/join.h. Here is a patch with dropped the 1-byte separator case. -- ___

[issue12805] Optimizations for bytes.join() et. al

2012-10-20 Thread Antoine Pitrou

Antoine Pitrou added the comment: Ok, here are the benchmark results here with a 1-byte separator: 10 x 10 0.244 usec 0.202 usec +21% 100 x 10 0.325 usec 0.326 usec-0% 1000 x 10 0.691 usec 0.689 usec+0% 10 x 1000 18.2 usec14.2 usec +28% 100 x 10

[issue12805] Optimizations for bytes.join() et. al

2012-10-20 Thread Serhiy Storchaka

Serhiy Storchaka added the comment: > Ok, but does it really make a difference and in which cases? Up to 40% on Athlon and up to 70% on Atom. > In other words, do you have benchmark numbers? :) Attached results for AMD Athlon 64 X2 4600+ and Intel Atom N570 @ 1.66GHz under 32-bit Linux. You c

[issue12805] Optimizations for bytes.join() et. al

2012-10-20 Thread Antoine Pitrou

Antoine Pitrou added the comment: Ok, but does it really make a difference and in which cases? In other words, do you have benchmark numbers? :) -- ___ Python tracker ___ ___

[issue12805] Optimizations for bytes.join() et. al

2012-10-20 Thread Serhiy Storchaka

Serhiy Storchaka added the comment: In a normal loop we copy into the output buffer for each iteration the element and the separator. As you know, the processor can copy large amounts of data several times more efficiently than byte-for-byte. Therefore, filling the buffer by memset more effici

[issue12805] Optimizations for bytes.join() et. al

2012-10-19 Thread Antoine Pitrou

Antoine Pitrou added the comment: I don't understand the point of this optimization. It looks rather cryptic and very specific. Can you elaborate? -- ___ Python tracker ___

[issue12805] Optimizations for bytes.join() et. al

2012-10-19 Thread Serhiy Storchaka

Serhiy Storchaka added the comment: I adapted a patch to the current code (now it works for bytearrays too), deleted the behavior change, left optimization only for 0- and 1-bytes separator, deleted tests (bytes.join tests already exists), wrote a benchmark and after long experiments has set a

[issue12805] Optimizations for bytes.join() et. al

2012-04-08 Thread Serhiy Storchaka

Serhiy Storchaka added the comment: > Main note: the patched test adds a space to the separator, but that is not > enough to account for the difference. Sorry, I copied the wrong line. $ ./python -m timeit -s "seq=[bytes([i]*10) for i in range(256)]" "b' '.join(seq)" 10 loops, best of 3:

[issue12805] Optimizations for bytes.join() et. al

2012-04-08 Thread Terry J. Reedy

Terry J. Reedy added the comment: Side note: Windows requires that args be quoted with ", not ', to work properly, at least with these args. Main note: the patched test adds a space to the separator, but that is not enough to account for the difference. c:\Programs\Python32>python -m timeit

[issue12805] Optimizations for bytes.join() et. al

2012-04-08 Thread Serhiy Storchaka

Serhiy Storchaka added the comment: > Regardless of whether optimization (which is negligible in this case), I > don't know whether to consider such side effect desirable or undesirable. After some thought, I realized that this is an erroneous behavior. -- ___

[issue12805] Optimizations for bytes.join() et. al

2012-04-08 Thread Serhiy Storchaka

Serhiy Storchaka added the comment: > Honestly, I don't think there's much point in these optimizations. The first > one ("The sequence length and separator length are both 0") is probably > useless. This optimization changes the behavior. ... def __repr__(self): return 'B(' + super().__

[issue12805] Optimizations for bytes.join() et. al

2012-04-08 Thread Antoine Pitrou

Antoine Pitrou added the comment: Honestly, I don't think there's much point in these optimizations. The first one ("The sequence length and separator length are both 0") is probably useless. -- ___ Python tracker