On 31/05/11 01:28, James Youngman wrote: > [ CC += bug-findutils, += Paolo, -= bug-coreutils ] > > 2009/11/3 Pádraig Brady <p...@draigbrady.com>: >> Pádraig Brady wrote: >>> Paolo Bonzini wrote: >>>> Maybe we want a --parallel option (too bad -p is taken) for xargs that >>>> forces the creation of the number of processes passed with -P or taken >>>> from nproc (for example by starting "md5sum $1 $5 $9 ...", "md5sum $2 $6 >>>> $10 ...", etc.)? >>>> That would be an interesting alternative to this core-count proposal... >>> >>> I'm not sure what you mean here. >>> I already suggested to the xargs maintainer that `xargs -P` >>> should be equivalent to xargs -P$(nproc). >>> `nproc` as an external command would still be useful though. >> >> Here's a patch for that. >> It needs to be updated to reference the new gnulib >> when Bruno's nproc update hits gnulib. > > I'm sorry, I totally missed this email because although it was sent to > me it got buried under a ton of gnulib stuff. > > As I understand things the intent of the patch is to (optionally I > suppose) make xargs keener to keep the CPUs busy (or whatever else > we're parallelising over I guess), even if it needs to launch > processes with short command lines to do so? That is With -P N, if > there are less than N running children and xargs reads an argument it > should launch a new child process? If that's the approximate idea, I > think that there could be use cases for this. > > Could we discuss this aspect on bug-findutils instead? Hence I > changed the subject and the CC list...
The patch presented was flawed due to optional parameter processing. It was just to do support `xargs -P $(nproc)` without needing `nproc`. Also discussed was changing distribution of input to children. I.E. by adding a --parallel option which on a machine with 4 available cores would: $ seq 1 13 | xargs --parallel 1 5 9 13 2 6 10 3 7 11 4 8 12 Note -P $(nproc) is implicit in the above. I also discussed a mechanism using non blocking reading of input, to more efficiently split the input parameters across children. memory definitely hazy on this one :) cheers, Pádraig.
