New submission from Glenn Langford: In asyncio, tasks.py as_completed() appears to trigger adding and removing callbacks multiple times for the pending set of futures, each time a single future completes.
For example, to wait for 5 futures which complete at different times: - as_completed() uses _wait() - _wait() will add 5 callbacks, one for each future - when one future completes, a callback is triggered and all 5 callbacks are removed; this is because _wait() was called with FIRST_COMPLETED - for the 4 remaining futures - 4 callbacks have to be added back again - when the next future completes, the 4 callbacks are removed etc… The worst case is if as_completed() is called to wait for all of a larger number of futures, which complete at different times. For example, with 100 futures worst case, ~10,000 callback adds and removes would be performed. (I am very new to the asyncio code, so I don't have a patch to offer at this point). ---------- components: Library (Lib) messages: 210679 nosy: glangford, gvanrossum, haypo, pitrou priority: normal severity: normal status: open title: asyncio as_completed() thrashes adding and removing callbacks type: performance versions: Python 3.4, Python 3.5 _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue20566> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
