On 2021-Jul-3, at 11:25, bob prohaska <f...@www.zefox.net> wrote:
>>>> On 2021-Jul-2, at 19:23, Mark Millard <marklmi at yahoo.com> wrote:
>>>
>>>>> Side note:
>>>>>
>>>>> It llooks like http://www.zefox.org/~bob/swaplogs/poudrierellvm10.log
>>>>> shows that you tried with:
>>>>>
>>>>> Device 1K-blocks Used Avail Capacity
>>>>> /dev/da0s2b 1048576 25784 1022792 2%
>>>>> /dev/mmcsd0s2b 1048576 25124 1023452 2%
>>>>> Total 2097152 50908 2046244 2%
>>>>>
> [hope the quotes are right!]
>
> That's correct. The sequence of experiments ran something like this:
>
> The Pi3 was configured with a a pair of ~3 GB swap partitions, one on
> microSD, the other on the 1 TB mechanical hard disk. Make was not limited
> in the number of jobs it could parallel. OOMA was restrained by putting
> vm.pageout_oom_seq="4096"
> vm.pfault_oom_attempts="20"
> in /boot/loader.conf The usual "excessive swap" warnings were presented
> during boot and ignored by me.
>
> Worlds and kernels built wtihout trouble, so I tried building www/chromium
> using poudriere. It stopped in /devel/llvm10 with the "expected expression"
> error and continued to stop there despite updating /usr/ports several times.
> At no time were there any hints of swap problems. Resorting to a GENERIC
> self-hosted kernel made no difference. /usr/src was not tampered with.
So you still have not tried an artifacts or snapshot kernel+world?
> Eventually I resorted to running make in devel/llvm10, to my surprise it
> ran to completion.
Interesting.
Was this -j4? -j1? -j2? Any other interesting characteristics
for how it was run?
It would be interesting to see if building in a chroot
in that make style also worked (or a non-poudriere jail).
> It also ran make package successfully. Again I tried to
> build just devel/llvm10 using poudriere, again getting "expected expression".
>
> At that point I resized the swap partitions to 1 GB each and tried poudriere
> on devel/llvm10. That got rid of the excessive swap warnings, but didn't help.
> Finally I placed
> MAKE_JOBS_NUMBER=2
> in /usr/local/etc/poudriere.d/make.conf and tried again. That still failed,
> still with "expected expression".
I'll note that the running build build shows Load Averages
of under 3. So the MAKE_JOBS_NUMBER=2 seems to be working.
> Since devel/llvm10 had created a package successfully, I tried slipping a copy
> into poudriere's package directory, hoping it would find and use the package
> to make further progress. Unfortunately, poudriere seems to remember the
> failure
> and won't use the proffered package.
After things build correctly, things tend to look something like
(using an example):
2# ls -FTla /usr/local/poudriere/data/packages/main-CA53-default/
total 12
drwxr-xr-x 3 root wheel 512 Jul 3 07:19:32 2021 ./
drwxr-xr-x 4 root wheel 512 Jul 1 19:25:44 2021 ../
lrwxr-xr-x 1 root wheel 18 Jun 28 04:32:43 2021 .buildname@ ->
.latest/.buildname
lrwxr-xr-x 1 root wheel 20 Jun 28 04:32:43 2021 .jailversion@ ->
.latest/.jailversion
lrwxr-xr-x 1 root wheel 16 Jul 3 07:19:32 2021 .latest@ -> .real_1625321972
drwxr-xr-x 4 root wheel 512 Jul 3 07:19:32 2021 .real_1625321972/
lrwxr-xr-x 1 root wheel 11 Jun 28 04:32:43 2021 All@ -> .latest/All
lrwxr-xr-x 1 root wheel 14 Jun 28 04:32:43 2021 Latest@ -> .latest/Latest
lrwxr-xr-x 1 root wheel 17 Jun 28 04:32:43 2021 meta.conf@ ->
.latest/meta.conf
lrwxr-xr-x 1 root wheel 16 Jun 28 04:32:43 2021 meta.txz@ ->
.latest/meta.txz
lrwxr-xr-x 1 root wheel 23 Jun 28 04:32:43 2021 packagesite.txz@ ->
.latest/packagesite.txz
But, if a bulk is in process or has finished after some package
had a build failure, there is also a:
.building/
in there. That is what the message:
Using packages from previously failed build: ${PACKAGES}/.building
is about when starting poudriere bulk again. This is how
poudriere avoids rebuilding what successfully built --but
without adjusting the prior successful bulk build (if any).
So poudriere would have expected the file for devel/llvm10 's
build to be in that .building/ directory instead of down under
the .real_*/ directory.
(I've not checked if there is other record keeping in .building/
about the materials as well.)
Going in a different direction, one way to force a build to
start over after a failure is to: rm -fr PATH/.building
before starting a new bulk build. This might be appropriate
if one suspects a problem of a kind that did not stop a
build but produced something for a build that fails to operate
correctly.
> It's still running, on lang/spidermoneky78.
So lang/rust finished. That is interesting because it includes an
llvm build internally.
Also: had you updated to pick up the workaround for the rust
build failures on aarch64? I doubt it because they were
commited on 2021-July-02. See,
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256864#c18
So that you did not get the process crash/core-dump during
lang/rust 's build is interesting.
> There were no reboots between experiments.
>
> My first suspicion is that I've somehow screwed up the poudriere setup,
> perhaps
> by a fumbled execution of poudriere jail -u, which I mistakenly thought was
> needed after updating /usr/ports.
Again, poudriere does not control memory initialization in
the processes in the builders.
> The fact that the stoppage reported looks like
> a syntax error specific to devel/llmv10 which is unaffected by swap pressure
> makes it seem unrelated to kernel or swap constraints.
The files with the syntax errors are ones generated by llvm-tblgen
during the build and it is the output of llvm-tblgen that is corrupt,
showing evidence of having used memory not initialized like it should
have been.
> AIUI, the hardware of the Pi4 is considerably different from the Pi3 in terms
> of memory management, noted from an interview with Eben Upton on YouTube.
Why would Eben Upton be talking about FreeBSD's memory management?
I suspect that the talk is not about what you think it is about,
but some narrower aspects than the overall memory managment.
> He
> didn't go into any detail. Whether that's relevant is unclear to me, but it
> does suggest the Pi4, even with restricted memory, won't behave like a Pi3.
Various reserved memory areas and such will vary but FreeBSD
uses the same general memory management code, not completely
separate code.
> Is there any sort of sanity test for the poudriere system? If I delete and
> re-create the existing jail can the existing package library be preserved
> and re-used? If not, that's OK, I'd just like to know beforehand.
>
# poudriere jail -jNAME -d
# poudriere jail -c -jNAME -m null -M /WORLDPATH -S /SRCPATH -v 14.0-CURRENT
should work fine. But really all that you are
doing is (using an example from my environment)
is deleting and rewriting a few very small files
in a directory with the jail's name:
# ls -FTla /usr/local/etc/poudriere.d/jails/main-CA53/
total 36
drwxr-xr-x 2 root wheel 512 Jul 2 21:03:23 2021 ./
drwxr-xr-x 3 root wheel 512 Jul 2 21:03:23 2021 ../
-rw-r--r-- 1 root wheel 14 Jul 2 21:03:23 2021 arch
-rw-r--r-- 1 root wheel 5 Jul 2 21:03:23 2021 method
-rw-r--r-- 1 root wheel 33 Jul 2 21:03:23 2021 mnt
-rw-r--r-- 1 root wheel 2 Jul 2 21:03:23 2021 pkgbase
-rw-r--r-- 1 root wheel 14 Jul 2 21:03:23 2021 srcpath
-rw-r--r-- 1 root wheel 11 Jul 2 21:03:23 2021 timestamp
-rw-r--r-- 1 root wheel 13 Jul 2 21:03:23 2021 version
# cat /usr/local/etc/poudriere.d/jails/main-CA53/arch
arm64.aarch64
# cat /usr/local/etc/poudriere.d/jails/main-CA53/method
null
# cat /usr/local/etc/poudriere.d/jails/main-CA53/mnt
/usr/obj/DESTDIRs/main-CA53-poud
# cat /usr/local/etc/poudriere.d/jails/main-CA53/pkgbase
0
# cat /usr/local/etc/poudriere.d/jails/main-CA53/srcpath
/usr/main-src
# cat /usr/local/etc/poudriere.d/jails/main-CA53/timestamp
1625285003
# cat /usr/local/etc/poudriere.d/jails/main-CA53/version
14.0-CURRENT
The deletion/replacement of timestamp may have rebuild
consequences from appearing to have changed (or just
being missing).
Nothing about any of those is going to change how memory
initialization is working in llvm-tblgen's operation
for generating any *GenGlobalISel.inc files, other than
if the timestamp forces some sort of rebuild from scratch
of some build dependencies first.
===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
