feature requests

[macker]

Hi, hope this is the right group for this:

I miss two basic (IMO) features in parallel processing:

1. make `threading.Thread.start()` return `self`

I'd like to be able to `workers = [Thread(params).start() for params in 
whatever]`. Right now, it's 5 ugly, menial lines:

workers = []
for params in whatever:
thread = threading.Thread(params)
thread.start()
workers.append(thread)

2. make multiprocessing pools (incl. ThreadPool) limit the size of their 
internal queues

As it is now, the queue will greedily consume its entire input, and if the 
input is large and the pool workers are slow in consuming it, this blows up 
RAM. I'd like to be able to `pool = Pool(4, max_qsize=1000)`. Same with the 
output queue (finished tasks).

Or does anyone know of a way to achieve this?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: feature requests

[macker]

> 
> Ugly, menial lines are a clue that a function to hide it could be useful.

Or a clue to add a trivial change elsewhere (hint for Ethan: `return self` at 
the end of `Thread.start()`).

> Have you verified that this is a problem in Python?

?

> You could try subclassing.

I could try many things. What this thread is about is trying to fix it on 
stdlib level, so that people don't have to reinvent the wheel every time.

Thanks to Chris for his suggestion. Ethan, please stay away from this thread.

-macker



> 
> 
> 
> --
> 
> ~Ethan~

-- 
https://mail.python.org/mailman/listinfo/python-list