On Sat, Apr 30, 2011 at 2:49 PM, Wolfgang Denk <w...@denx.de> wrote:
> Dear Andy Fleming,
>
> In message <1302687759-1649-1-git-send-email-aflem...@freescale.com> you
> wrote:
>> The MAKEALL script cleverly runs make with the appropriate options
>> to use all of the cores on the system, but your average U-Boot build
>> can't make much use of more than a few cores. If you happen to have
>> a many-core server, your builds will leave most of the system idle.
> ...
>> ${output_prefix}/$target. We launch up to 8 builds in parallel (with
>> each build still being told to use n+1 cores), and wait for all 8 to
>> finish before launching 8 more. It's not ideal, but it is faster.
>
> How did you figure that 8 * (n+1) would be an efficient number of
> tasks to use? Does this not depend on a lot of factors, like number
> of cores, relative speed of CPUs versus I/O subsystem, etc. ?
Heh, yes, this is still quite hacky. I'm curious if anyone has ideas
on this. I ran this on a 24-core system, and if I didn't limit the
number of concurrent builds to something (I don't recall how many I
tried), then make quickly consumed so many resources that fork refused
to work until some things had finished, and I ended up hitting ctrl-c
repeatedly. I'm not much of a Makefile guru, so I'm hoping someone
might know of better ways to do such resource management. My thought
was that 8 was decently high, and shouldn't consume more than a
smaller desktop system can handle.
>
> ...
>> +# Output
>> +# It is now possible to save the output to a build directory. This serves
>> +# two purposes. It allows you to keep the images for testing, and it
>> +# allows you to launch the makes in tandem. Pass in -o <dir>, and the build
>> +# will happen in <dir>/$target/
>
> Hm... this conflicts with / duplicates the function of BUILD_DIR,
> doesn't it?
Yes, the two usage models are slightly different in meaning. In order
to do concurrent builds, we have to build each board in a separate
directory. It should be possible to honor both, but we'd have to come
up with an accepted behavior. I also believe I had some issues with
using BUILD_DIR in the environment. But that may have been a temporary
issue brought on by syntax problems I was having. When I have some
free time again, I'll investigate.
>
>> [ -d ${LOG_DIR} ] || mkdir ${LOG_DIR} || exit 1
>>
>> +if [ "${output_prefix}" ] ; then
>> + [ -d ${output_prefix} ] || mkdir -p ${output_prefix} || exit 1
>> + [ -d "${output_prefix}/ERR" ] && rm -rf "${output_prefix}/ERR"
>> + mkdir "${output_prefix}/ERR"
>> +fi
>
> Should LOG_DIR not be adjusted, too?
It's not necessary, but it seems like a good idea.
>
>> + [ "$output_prefix" ] && export BUILD_DIR="${output_prefix}/${target}"
>
> Ouch. This means you are messing with user settings without warning
> or any indication. I don't like this.
>
> If we have a conflict (and here we have one), there should be at least
> errot / warning messages.
Ok.
>
>> + if [ "$output_prefix" ] ; then
>> + ${MAKE} clean
>> + find "${output_prefix}/$target" -type f -name '*.depend' |
>> xargs rm
>
> Why remove the .depend files and not any of the other temporary files?
Ignorance/laziness. :) This was a first pass at keeping the build dir
at a manageable size. When I did a "find" on the build directory after
make clean, the ".depend" files were the most prominent ones left. I
could add more in the next rev.
>
>> - TOTAL_CNT=$((TOTAL_CNT + 1))
>
> Um....
>
>> ${CROSS_COMPILE}size ${BUILD_DIR}/u-boot \
>> | tee -a ${LOG_DIR}/$target.MAKELOG
>> }
>> @@ -650,7 +681,20 @@ build_targets() {
>> if [ -n "${list}" ] ; then
>> build_targets ${list}
>> else
>> - build_target ${t}
>> + if [ "$output_prefix" ] ; then
>> + build_target ${t} &
>> + else
>> + build_target ${t}
>> + fi
>> +
>> + TOTAL_CNT=$((TOTAL_CNT + 1))
>> + CURRENT_COUNT=$((CURRENT_COUNT + 1))
>> + fi
>
> So TOTAL_CNT and CURRENT_COUNT get only set in the "else" case?
Right. Unless I misread the code, this is a recursive function. I
only want to count "leaf" builds. It's the same as before, but if the
counter were incremented in build_target, it would get concurrently
set 8 times to the same value.
>
>> + # Only run 8 builds concurrently
>> + if [ ${CURRENT_COUNT} -gt 8 ]; then
>
> Magic number...
>
>> echo "Boards compiled: ${TOTAL_CNT}"
>
> Is this report correct for all use cases, now?
It should be the same as it was before. I believe I checked it, since
I had a bug where it was clearly wrong in an earlier draft. I did a
build of all ppc, and the number looked right. I will double-check,
though.
Andy
_______________________________________________
U-Boot mailing list
U-Boot@lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot
