Pablo Galindo Salgado <pablog...@gmail.com> added the comment:
All the pending work items are added to a call queue that the processes consume
(_add_call_item_to_queue) and the status is set before adding it to said queue
(by calling set_running_or_notify_cancel()). Notice that the fact that
future.running() == True does not mean that a worker has picked up the work
item. The worker function (_process_worker for the ProcessPoolExecutor) gets
items for this queue and then sends back results in another queue (result
queue). If you add extra debugging you will see that only two items are
dequeued from the call_queue:
--- a/Lib/concurrent/futures/process.py
+++ b/Lib/concurrent/futures/process.py
@@ -231,6 +231,7 @@ def _process_worker(call_queue, result_queue, initializer,
initargs):
return
while True:
call_item = call_queue.get(block=True)
+ print("Worker: Getting a new item")
if call_item is None:
# Wake up queue management thread
result_queue.put(os.getpid())
@@ -277,6 +278,7 @@ def _add_call_item_to_queue(pending_work_items,
work_item = pending_work_items[work_id]
if work_item.future.set_running_or_notify_cancel():
+ print("Adding a new item to the call_queue")
call_queue.put(_CallItem(work_id,
work_item.fn,
work_item.args,
(END)
Executing your script:
import time, concurrent.futures
def call():
while True:
time.sleep(1)
if __name__ == "__main__":
with concurrent.futures.ProcessPoolExecutor(max_workers=2) as executor:
futures = [executor.submit(call) for _ in range(8)]
time.sleep(2)
for future in futures:
print(future.running())
Prints:
Adding a new item to the call_queue
Adding a new item to the call_queue
Adding a new item to the call_queue
Worker: Getting a new item
Worker: Getting a new item
True
True
True
False
False
False
False
False
----------
nosy: +pablogsal
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue37276>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
