On 12/9/2017 3:15 PM, Mark Millard wrote:
> I've been experimenting with using devel/poudriere-devel
> in the contexts:
>
> rpi2, bpi-m3 (Cortex-A7's, 4 core effective, 1 GiByte and 2 GiByte RAM)
> rpi3, Pine64+ 2GB (Cortex-A53's, 4 core, 1 GiByte and 2 GiByte RAM)
> powerpc64 (32-bit) powerpc (so-called Quad-Core PowerMacs, 16GiBYte and 2
> GiByte effective RAM)
> amd64 (Ryzen Threadripper 16-core/32-thread-total, 128 GiByte RAM)
>
> As part of this even the large configurations are
> currently set up for UFS so that aspect is uniform.
>
> One thing that I've noted (using the amd64 context
> as an example) is a non-linear relationship between
> builders and starting/cloning time:
>
>
> [00:00:26] Building 47 packages using 4 builders
> [00:00:26] Starting/Cloning builders
> [00:00:32] Hit CTRL+t at any time to see build progress and stats
>
> [00:00:22] Building 49 packages using 14 builders
> [00:00:22] Starting/Cloning builders
> [00:05:07] Hit CTRL+t at any time to see build progress and stats
>
> [00:00:33] Building 38 packages using 29 builders
> [00:00:33] Starting/Cloning builders
> [00:17:22] Hit CTRL+t at any time to see build progress and stats
>
> Monitoring the longer running ones I see lots of:
>
> getblk
> (Waiting to access file data that is being accessed by another thread)
>
> So I tried a crude form of staggering the cloning
> by adding a 1 second sleep in start_builders
> ( /usr/local/share/poudriere/common.sh ):
>
> start_builders() {
> local arch=$(injail uname -p)
>
> msg "Starting/Cloning builders"
> bset status "starting_jobs:"
> run_hook start_builders start
>
> bset builders "${JOBS}"
> bset status "starting_builders:"
> parallel_start
> for j in ${JOBS}; do
> parallel_run start_builder ${j} ${arch}
> sleep 1
> done
> parallel_stop
>
> run_hook start_builders stop
> }
>
> The result for 14 was:
>
> [00:00:14] Building 23 packages using 14 builders
> [00:00:14] Starting/Cloning builders
> [00:00:53] Hit CTRL+t at any time to see build progress and stats
>
> For 23 (I no longer had more to build) the result was:
>
> [00:00:12] Building 23 packages using 23 builders
> [00:00:12] Starting/Cloning builders
> [00:04:49] Hit CTRL+t at any time to see build progress and stats
>
> Trying 5 seconds instead of 1 second for the
> sleep time:
>
> The result for 14 was:
>
> [00:00:12] Building 23 packages using 14 builders
> [00:00:12] Starting/Cloning builders
> [00:01:25] Hit CTRL+t at any time to see build progress and stats
>
> For 23 the result was:
>
> [00:00:12] Building 23 packages using 23 builders
> [00:00:12] Starting/Cloning builders
> [00:02:11] Hit CTRL+t at any time to see build progress and stats
>
> It appears that a few second sleep is of significant
> help for having lots of builders.
> Interesting. I wonder why exactly... > === > Mark Millard > markmi at dsl-only.net > -- Regards, Bryan Drewery
signature.asc
Description: OpenPGP digital signature
