[issue19506] subprocess.communicate() should use a memoryview

2013-12-07 Thread Gregory P. Smith
Changes by Gregory P. Smith : -- resolution: -> fixed stage: -> committed/rejected status: open -> closed ___ Python tracker ___ ___

[issue19506] subprocess.communicate() should use a memoryview

2013-12-07 Thread Gregory P. Smith
Gregory P. Smith added the comment: fwiw, this patch made sense. using test_sub.py on my dual-core amd64 box with an opt build is saw the times drop from ~0.90-0.97 to ~0.75-0.8. more speedup than i was really anticipating for this use case. it probably depends upon what the value of _PIPE_

[issue19506] subprocess.communicate() should use a memoryview

2013-12-07 Thread Roundup Robot
Roundup Robot added the comment: New changeset 44948f5bdc12 by Gregory P. Smith in branch '3.3': Fixes issue #19506: Use a memoryview to avoid a data copy when piping data http://hg.python.org/cpython/rev/44948f5bdc12 New changeset 5379bba2fb21 by Gregory P. Smith in branch 'default': Fixes issu

[issue19506] subprocess.communicate() should use a memoryview

2013-12-01 Thread Charles-François Natali
Charles-François Natali added the comment: Impressive speedup :-) -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscr

[issue19506] subprocess.communicate() should use a memoryview

2013-11-30 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: Best of 10 runs. Unpatched: 3.91057508099766 Patched:3.86466505300632 -- ___ Python tracker ___

[issue19506] subprocess.communicate() should use a memoryview

2013-11-30 Thread Antoine Pitrou
Antoine Pitrou added the comment: On a quad-core machine: - without Victor's patch: $ time ./python test_sub.py 0.3926395702847 real0m0.521s user0m0.412s sys 0m0.238s - with Victor's patch: $ time ./python test_sub.py 0.3856174530001226 real0m0.516s user0m0.404s sy

[issue19506] subprocess.communicate() should use a memoryview

2013-11-30 Thread Charles-François Natali
Charles-François Natali added the comment: Could someone with a dual-core machine try the attached simplistic benchmark with and without Victor's patch? I can see some user-time difference with 'time' on my single-core machine, but I'm curious to see how this would affect things were both the par

[issue19506] subprocess.communicate() should use a memoryview

2013-11-20 Thread Charles-François Natali
Charles-François Natali added the comment: > Serhiy Storchaka added the comment: > > I don't know. But a function call have a cost. I'm with Serhiy here. Writing a performance optimization without benchmark is generally a bad idea (I won't quote Knuth :-). -- __

[issue19506] subprocess.communicate() should use a memoryview

2013-11-20 Thread STINNER Victor
Changes by STINNER Victor : Removed file: http://bugs.python.org/file32714/subprocess_memoryview.patch ___ Python tracker ___ ___ Python-bugs-

[issue19506] subprocess.communicate() should use a memoryview

2013-11-20 Thread STINNER Victor
STINNER Victor added the comment: New patch without the unwanted change in listobject.c. -- Added file: http://bugs.python.org/file32720/subprocess_memoryview-2.patch ___ Python tracker

[issue19506] subprocess.communicate() should use a memoryview

2013-11-20 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: I don't know. But a function call have a cost. -- ___ Python tracker ___ ___ Python-bugs-list mail

[issue19506] subprocess.communicate() should use a memoryview

2013-11-20 Thread STINNER Victor
STINNER Victor added the comment: > What is a performance for small chunks? I have no idea. Does it matter? -- ___ Python tracker ___ ___

[issue19506] subprocess.communicate() should use a memoryview

2013-11-19 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: What is a performance for small chunks? -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Pyth

[issue19506] subprocess.communicate() should use a memoryview

2013-11-19 Thread STINNER Victor
STINNER Victor added the comment: > Hi, why does your patch change listobject.c? Oops, the change is unrelated (#19578). -- ___ Python tracker ___ __

[issue19506] subprocess.communicate() should use a memoryview

2013-11-19 Thread Antoine Pitrou
Antoine Pitrou added the comment: Hi, why does your patch change listobject.c? -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue19506] subprocess.communicate() should use a memoryview

2013-11-19 Thread STINNER Victor
STINNER Victor added the comment: Attached subprocess_memoryview.patch uses a memoryview() to avoid memory copies. The behaviour is undefined if two threads share the same Popen object and call the communicate() method at the same time. But I'm not sure that Popen is thread safe, it doesn't us

[issue19506] subprocess.communicate() should use a memoryview

2013-11-05 Thread STINNER Victor
Changes by STINNER Victor : -- type: -> performance ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://ma

[issue19506] subprocess.communicate() should use a memoryview

2013-11-05 Thread STINNER Victor
New submission from STINNER Victor: The following code copies data, whereas the copy can be avoided using a memory view: chunk = self._input[self._input_offset:self._input_offset + _PIPE_BUF] self._input_offset += os.write(key.fd, chunk) It should be replaced with: input_view = memoryview(sel