Kyle Stanley <aeros...@gmail.com> added the comment:
> No, that would be too much work. Writing a thread pool or process pool from > scratch is an extremely tedious task and it will take us years to stabilize > the code. It's not simple. > We should design *our* API correctly though. And that means that we can't > initialize our pools in __init__. I can see that it would be a lot of additional work, that's why I was using ThreadPoolExecutor in my earlier prototype. The main issue that I'm not seeing though is how exactly we're going to actually spawn the threads or processes asynchronously upon *startup* in ThreadPool using ThreadPoolExecutor's existing public API, with only the submit(), map(), and shutdown() methods. Unless I'm misunderstanding something, the executor classes were not designed with that intention in mind. With Executor, a new thread/process (worker) is spawned *as needed* when submit() is called throughout the lifespan of the Executor up to max_workers, rather than upon startup as you're wanting ThreadPool to do. Thus, it seemed to make more sense to me to actually build up a new Pool class from scratch that was largely based on Executor, but with significantly differing functionality. Otherwise, it seems like we would have to make some modifications to ThreadPoolExecutor, or inherit from it and then redesign the internals of some of the methods to change the way the threads/processes are spawned. > It shouldn't be much harder than run_in_executor() as we continue to rely on > concurrent.future (at least for the first version). I think that run_in_executor() was far more simple compared to this. The functionality of run_in_executor() almost maps directly to executor.submit(), other than a few conditional checks and converting the concurrent.futures.Future returned to an asyncio.Future through wrap_future(). > We need to start the discussion about this API on discourse. Please give me > a few days to organize that. I agree that we should probably continue this discussion on discourse, as it probably goes beyond the scope of a single issue. No problem. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32309> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
