Dan Upton wrote:
This might be more information than necessary, but it's the best way I
can think of to describe the question without being too vague.
The task:
I have a list of processes (well, strings to execute said processes)
and I want to, roughly, keep some number N running at a time. If one
terminates, I want to start the next one in the list, or otherwise,
just wait.
The attempted solution:
Using subprocess, I Popen the next executable in the list, and store
it in a dictionary, with keyed on the pid:
(outside the loop)
procs_dict={}
(inside a while loop)
process = Popen(benchmark_exstring[num_started], shell=true)
procs_dict[process.pid]=process
Then I sleep for a while, then loop through the dictionary to see
what's terminated. For each one that has terminated, I decrement a
counter so I know how many to start next time, and then try to remove
the record from the dictionary (since there's no reason to keep
polling it since I know it's terminated). Roughly:
for pid in procs_dict:
if procs_dict[pid].poll() != None
# do the counter updates
del procs_dict[pid]
The problem:
RuntimeError: dictionary changed size during iteration
So, the question is: is there a way around this?
Yes. Create a list of keys, and loop through it:
pids = procs_dict.keys()
for pid in pids:
if procs_dict[pid].poll() != None
# do the counter updates
del procs_dict[pid]
Then the diction delete operation won't trip up the loop and its
internal counter.
OR: Create a list of things to delete while you are in the loop, and do
the delete afterwards
deleteme = []
for pid in procs_dict:
if procs_dict[pid].poll() != None
# do the counter updates
deleteme.append(pid)
for pid in deleteme:
del procs_dict[pid]
OR: shred and rebuild the dictionary each time:
new_dict = {}
for pid,value in procs_dict.items():
if value.poll() != None:
# do the counter updates
pass
else:
new_dict[pid] = value
procs_dict = new_dict
Gary Herron
I know that I can
just /not/ delete from the dictionary and keep polling each time
around, but that seems sloppy and like it could keep lots of memory
around that I don't need, since presumably the dictionary holding a
reference to the Popen object means the garbage collector could never
reclaim it. Is the only reasonable solution to do something like
append all of those pids to a list, and then after I've iterated over
the dictionary, iterate over the list of pids to delete?
(Also, from the implementation side, is there a reason the dictionary
iterator can't deal with that? If I was deleting from in front of the
iterator, maybe, but since I'm deleting from behind it...)
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
