Dan O'Reilly added the comment:
Unfortunately, I don't really have time to continue working on this anymore (or
any of the other proposed patches I had submitted around 2014).
--
___
Python tracker
<http://bugs.python.o
Dan O'Reilly added the comment:
It's probably too late to do anything about this now, but wouldn't it make more
sense for `Pool.__exit__` to call `close`, rather than `terminate`? The vast
majority of the time, that's probably what the user of the `Pool` would want to
run
Changes by Dan O'Reilly :
--
nosy: +dan.oreilly
versions: +Python 3.3
___
Python tracker
<http://bugs.python.org/issue6766>
___
___
Python-bugs-list m
Changes by Dan O'Reilly :
--
nosy: +dan.oreilly
___
Python tracker
<http://bugs.python.org/issue8800>
___
___
Python-bugs-list mailing list
Unsubscr
Changes by Dan O'Reilly :
--
type: -> enhancement
___
Python tracker
<http://bugs.python.org/issue21423>
___
___
Python-bugs-list mailing list
Unsub
Dan O'Reilly added the comment:
This is basically the same thing that issue21367 is reporting.
--
nosy: +dan.oreilly
___
Python tracker
<http://bugs.python.org/is
Dan O'Reilly added the comment:
Hey, my first committed patch :) Thanks for helping to push this through,
Antoine!
--
___
Python tracker
<http://bugs.python.org/is
Dan O'Reilly added the comment:
Thanks, Antoine. I've opened issue22393.
--
___
Python tracker
<http://bugs.python.org/issue9205>
___
___
Python-bugs-l
New submission from Dan O'Reilly:
This is essentially a dupe of issue9205, but it was suggested I open a new
issue, since that one ended up being used to fix this same problem in
concurrent.futures, and was subsequently closed.
Right now, should a worker process in a Pool unexpectedl
Dan O'Reilly added the comment:
Is it possible to have this issue re-opened, so that the new patch is more
likely to get attention? Or should I create a new issue for the multiprocessing
patch?
--
___
Python tracker
<http://bugs.py
Dan O'Reilly added the comment:
Are any other changes needed here? I'm still not completely clear on what
Victor meant with his last comment.
--
___
Python tracker
<http://bugs.python.o
Changes by Dan O'Reilly :
Added file: http://bugs.python.org/file36481/exectest.py
___
Python tracker
<http://bugs.python.org/issue22281>
___
___
Python-bugs-list m
New submission from Dan O'Reilly:
As initially discussed on python-ideas, it would be nice if there was a way to
query the concurrent.futures Executor objects for information about their
internal state - number of total/active/idle workers, number of
total/active/waiting tasks, which
Dan O'Reilly added the comment:
>> So, concurrent.futures is fixed now. Unless someone wants to patch
>> multiprocessing.Pool, I am closing this issue.
I realize I'm 3 years late on this, but I've put together a patch for
multiprocessing.Pool. Should a process
Changes by Dan O'Reilly :
--
nosy: +dan.oreilly
___
Python tracker
<http://bugs.python.org/issue6721>
___
___
Python-bugs-list mailing list
Unsubscr
Dan O'Reilly added the comment:
A couple of small updates based on comments from Charles-François Natali:
* Use itertools.chain.from_iterable to "yield from" the result chunks instead
of a for loop.
* Add more tests with varying chunksizes.
--
Added file: http://
Dan O'Reilly added the comment:
Another updated patch. This one changes ProcessPoolExecutor behavior so that
RuntimeErrors are raised in any active Futures, and on subsequent calls to
submit after the initializer fails. This makes its behavior consistent with
ThreadPoolExe
Dan O'Reilly added the comment:
> It seems the addition of the initargs argument doesn't tackle Mark's
> objection here:
> > the initialiser and uninitialiser for the EnhancedThreadPoolExecutor
> > accept no arguments. In retrospect, it would have been better
Dan O'Reilly added the comment:
Here's an updated patch. Changes:
* Fixed what appears to have been a find/replace typo I made prior to uploading
the previous patch.
* The tracebacks from the negative unit tests are now suppressed.
* Fixed a race condition in the initializ
Changes by Dan O'Reilly :
--
title: concurrent.futures.ThreadPoolExecutor should accept an initializer
argument -> concurrent.futures.ThreadPoolExecutor/ProcessPoolExecutor should
accept an initializer argument
___
Python tracke
Dan O'Reilly added the comment:
Here's an updated patch that adds documentation and Antoine's requested code
changes.
--
Added file: http://bugs.python.org/file36306/map_chunksize_with_docs.patch
___
Python tracker
<http
Changes by Dan O'Reilly :
Added file: http://bugs.python.org/file36185/test_mult.py
___
Python tracker
<http://bugs.python.org/issue11271>
___
___
Python-bugs-list m
Changes by Dan O'Reilly :
Removed file: http://bugs.python.org/file36065/test_mult.py
___
Python tracker
<http://bugs.python.org/issue11271>
___
___
Python-bugs-l
Dan O'Reilly added the comment:
I've attached an updated patch based on your review comments; there's now a
unit test with a non-default chunksize, the chunking algorithm is more
readable, and the same code path is used no matter what chunksize is provided.
I've also upd
Dan O'Reilly added the comment:
re: Diverging ThreadPoolExecutor and ProcessPoolExecutor APIs. I thought about
this as well. It would of course be possible to make ThreadPoolExecutor's API
match, but it would serve no useful purpose that I can think of. I guess we
coul
Dan O'Reilly added the comment:
Hmm, I'm not sure what you mean. What check in the policy would prevent this
issue you described in #21998?:
import asyncio, os
loop = asyncio.get_event_loop()
pid = os.fork()
if pid:
print("parent", loop._csock.fileno(), loop._
Dan O'Reilly added the comment:
re: #21998, perhaps it's time to revive #16500? Without that, I'm not sure what
can be done aside from documenting the need to call "loop =
asyncio.get_event_loop()" in the child
Dan O'Reilly added the comment:
I've added a unit test that spawns a new forked process via multiprocessing,
and verifies that the loop returned by get_event_loop is not the same as the
one we have in the parent.
--
Added file: http://bugs.python.org
Dan O'Reilly added the comment:
Yep, agreed on both points. The latter suggestion also has the benefit of not
requiring any test changes. Here's an updated patch.
--
Added file: http://bugs.python.org/file36119/map_chunksize2.patch
Changes by Dan O'Reilly :
Added file: http://bugs.python.org/file36118/handle_mp_unix.diff
___
Python tracker
<http://bugs.python.org/issue22087>
___
___
Pytho
Changes by Dan O'Reilly :
Removed file: http://bugs.python.org/file36116/handle_mp_unix.diff
___
Python tracker
<http://bugs.python.org/issue22087>
___
___
Pytho
Changes by Dan O'Reilly :
--
title: _UnixDefaultEventLoop policy should either create a new loop or
explicilty fail when get_event_loop() is called from a multiprocessing child
process -> _UnixDefaultEventLoopPolicy should either create a new loop or
explicilty f
Changes by Dan O'Reilly :
--
type: -> behavior
___
Python tracker
<http://bugs.python.org/issue22087>
___
___
Python-bugs-list mailing list
Unsub
Changes by Dan O'Reilly :
Added file: http://bugs.python.org/file36117/test_loop.py
___
Python tracker
<http://bugs.python.org/issue22087>
___
___
Python-bugs-list m
New submission from Dan O'Reilly:
On non-Windows platforms, if a user attempts to use asyncio.get_event_loop() in
a child process created by multiprocessing.Process using the fork context, and
an asyncio event loop is also being used in the main process, the same
_UnixSelectorEventLoop o
Changes by Dan O'Reilly :
--
components: +Library (Lib) -Interpreter Core
type: compile error -> behavior
___
Python tracker
<http://bugs.python.org
Dan O'Reilly added the comment:
How are you importing JoinableQueue? You'll see this error if you import it
from multiprocessing.queues instead of directly from multiprocessing. This is
because multiprocessing.JoinableQueue is now a function:
def JoinableQueue(self,
Dan O'Reilly added the comment:
I've added new versions of the patch/demonstration that ensures we actually
submit all the futures before trying to yield from the returned iterator.
Previously we were just returning a generator object when map was called,
without actually subm
Changes by Dan O'Reilly :
Added file: http://bugs.python.org/file36067/map_chunksize.patch
___
Python tracker
<http://bugs.python.org/issue11271>
___
___
Pytho
Changes by Dan O'Reilly :
Removed file: http://bugs.python.org/file36064/map_chunksize.patch
___
Python tracker
<http://bugs.python.org/issue11271>
___
___
Pytho
Changes by Dan O'Reilly :
Added file: http://bugs.python.org/file36065/test_mult.py
___
Python tracker
<http://bugs.python.org/issue11271>
___
___
Python-bugs-list m
Changes by Dan O'Reilly :
Removed file: http://bugs.python.org/file36058/test_mult.py
___
Python tracker
<http://bugs.python.org/issue11271>
___
___
Python-bugs-l
Changes by Dan O'Reilly :
Added file: http://bugs.python.org/file36064/map_chunksize.patch
___
Python tracker
<http://bugs.python.org/issue11271>
___
___
Pytho
Changes by Dan O'Reilly :
Removed file: http://bugs.python.org/file36059/map_chunksize.patch
___
Python tracker
<http://bugs.python.org/issue11271>
___
___
Pytho
Dan O'Reilly added the comment:
Here's a patch that adds the new map implementation from the benchmark script
to concurrent.futures.process.
--
keywords: +patch
Added file: http://bugs.python.org/file36059/map_chunksize.patch
___
Pyth
Dan O'Reilly added the comment:
I'm seeing an even larger difference between multiprocessing.Pool and
ProcessPoolExecutor on my machine, with Python 3.4:
Starting multiproc...done in 2.160644769668579 s.
Starting futures...done in 67.953957319259644 s.
Starting futures "f
Dan O'Reilly added the comment:
It seems like everyone agrees that this functionality is useful, so I'm
reviving this in hopes of getting a patch pushed through. I've updated Andreas'
patch so that it applies cleanly against the latest tree, and tweaked the
handl
Dan O'Reilly added the comment:
Attached is a patch for the latter approach.
--
keywords: +patch
Added file: http://bugs.python.org/file36014/web.patch
___
Python tracker
<http://bugs.python.org/is
New submission from Dan O'Reilly:
Currently, when webbrowser.get() is passed a "using" argument that consists of
a command line string like
"C:\Users\dan\AppData\Local\Google\Chrome\Application\chrome.exe %s", it will
use shlex.split(command_line) to tokenize the st
Changes by Dan O'Reilly :
--
title: thread module documentation erroneously(?) states not all built-in
functions release the GIL -> thread module documentation erroneously(?) states
not all built-in functions that do blocking I/O release
New submission from Dan O'Reilly:
Currently, the caveats section of the thread/_thread module has this statement
in it:
"Not all built-in functions that may block waiting for I/O allow other threads
to run. (The most popular ones (time.sleep(), file.read(), select.select())
work a
51 matches
Mail list logo
