[issue11271] concurrent.futures.ProcessPoolExecutor.map() slower than multiprocessing.Pool.map() for fast function argument
Tobias Brink added the comment: I can confirm an overhead of 2 ms to 3 ms using a relatively recent Intel Core i5 CPU. I (personally) believe these 3 ms to be a pretty big overhead on modern computers and I also believe that it would be relatively simple to reduce it. I don't have much time at the moment but I will try to produce a complete proof of concept patch for the futures module in the next weeks. I'd be happy to get some comments when it is finished. -- ___ Python tracker <http://bugs.python.org/issue11271> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11324] ConfigParser(interpolation=None) doesn't work
New submission from Tobias Brink : The docs for Python 3.2 say that p = configparser.ConfigParser(interpolation=None) disables interpolation. Instead it gives this traceback when calling p.read(): Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.2/configparser.py", line 688, in read self._read(fp, filename) File "/usr/lib/python3.2/configparser.py", line 1081, in _read self._join_multiline_values() File "/usr/lib/python3.2/configparser.py", line 1091, in _join_multiline_values options[name] = self._interpolation.before_read(self, AttributeError: 'NoneType' object has no attribute 'before_read' The attached patch should fix it. -- components: Library (Lib) files: configparser_interpolation.patch keywords: patch messages: 129421 nosy: tbrink priority: normal severity: normal status: open title: ConfigParser(interpolation=None) doesn't work type: behavior versions: Python 3.2, Python 3.3 Added file: http://bugs.python.org/file20895/configparser_interpolation.patch ___ Python tracker <http://bugs.python.org/issue11324> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11324] ConfigParser(interpolation=None) doesn't work
Tobias Brink added the comment: I added a test but I am not too familiar with the Python test suite. Please check if the "test_init_takes_interpolation_none" test is necessary because the test suite also fails without it if my patch is not applied. Feel free to remove it if it is superfluous. -- Added file: http://bugs.python.org/file20957/patch_with_test.diff ___ Python tracker <http://bugs.python.org/issue11324> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11271] concurrent.futures.ProcessPoolExecutor.map() slower than multiprocessing.Pool.map() for fast function argument
New submission from Tobias Brink : I tested the new concurrent.futures.ProcessPoolExecutor.map() in 3.2 with the is_prime() function from the documentation example. This was significantly slower than using multiprocessing.Pool.map(). Quick look at the source showed that multiprocessing sends the iterable in chunks to the worker process while futures sends always only one entry of the iterable to the worker. Functions like is_prime() which finish relatively fast make the communication overhead (at least I guess that is the culprit) very big in comparison. Attached is a file which demonstrates the problem and a quick workaround. The workaround uses the chunk idea from multiprocessing. The problem is that it requires the iterables passed to map() to have a length and be indexable with a slice. I believe this limitation could be worked around. -- components: Library (Lib) files: map_comparison.py messages: 128963 nosy: tbrink priority: normal severity: normal status: open title: concurrent.futures.ProcessPoolExecutor.map() slower than multiprocessing.Pool.map() for fast function argument type: performance versions: Python 3.2 Added file: http://bugs.python.org/file20825/map_comparison.py ___ Python tracker <http://bugs.python.org/issue11271> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11271] concurrent.futures.ProcessPoolExecutor.map() slower than multiprocessing.Pool.map() for fast function argument
Tobias Brink added the comment: Playing around a bit I wrote the attached implementation which works with all iterables. -- Added file: http://bugs.python.org/file20826/new_processpoolexecutor.py ___ Python tracker <http://bugs.python.org/issue11271> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
