Hi Mike,
On Mon, Nov 21, 2011 at 11:00 AM, Mike Frysinger <vap...@gentoo.org> wrote:
> On Monday 21 November 2011 00:15:21 Simon Glass wrote:
>> On Sun, Nov 20, 2011 at 1:23 PM, Mike Frysinger wrote:
>> > On Thursday 03 November 2011 03:28:29 Andy Fleming wrote:
>> >> + # Limit number of parallel builds
>> >> + if [ ${CURRENT_COUNT} -gt ${BUILD_NBUILDS} ] ; then
>> >> + CURRENT_COUNT=0;
>> >> + wait;
>> >> fi
>> >
>> > you don't need those semicolons. also, this is not as good as it should
>> > be: if you're running 10 jobs in parallel, you fork 10, and then you
>> > wait for all of them to finish before forking another set of 10.
>> >
>> > what you could do is something like:
>> > JOB_IDX=0
>> > JOB_IDX_FIRST=0
>> > BUILD_NBUILDS=1
>> >
>> > ... foreach target ... ; do
>> > build_target &
>> > pids[$(( JOB_IDX++ ))]=$!
>> > if [ $(( JOB_IDX - JOB_IDX_FIRST )) -ge ${BUILD_NBUILDS} ]
>> > ; then wait ${pids[$(( JOB_IDX_FIRST++ ))]}
>> > fi
>> > done
>> > wait
>> >
>> > this isn't perfect as it assumes the first job launched will always
>> > finish first, but it's a lot closer than the current code.
>>
>> Since all the jobs are launched at the same time this is not really a
>> valid assumption is it? IMO on this point it's good enough as it is
>> for now...
>
> my point was, it's about as valid as you can get in bash. you can't do non-
> blocking wait's in bash, nor can you wait on a list of pids and have it return
> only one at a time. the launching-jobs-at-the-same-time isn't really the
> worse part: builds can vary greatly in terms of how much code they actually
> compile. some boards are very small while others are quite fat.
>
> so your choices are:
> - make a general "good enough" assumption (which i did here)
> - have the main thread manually poll every child through some IPC mech
> such as looking for files the children touch when they're done, and *then* do
> the wait explicitly on that child
You missed one:
- write it in Python :-)
(which I might try if patman is accepted)
>
> the problem with the latter is that the code tends to get much more
> complicated, and the main thread ends up doing quite a bit of busy work which
> kills CPU cycles. my original assumption ends up keeping the CPU's fairly
> loaded i think, and is fairly simplistic code.
> -mike
>
It should be easy enough in Python to just maintain 20 threads (or
whatever) and start the next job when a thread becomes idle.
Regards,
Simon
_______________________________________________
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot
