Re: WITH_META_MODE: any effect? Tree built twice!

2018-12-12 Thread Simon J. Gerraty
O. Hartmann  wrote:
> delete-old|-libs afterwards, I started again a build (filemon loaded!). And, 
> surprise,
> surprise, compilation of all the long-haul taking LLVM/CLANG stuff starts 
> again! That is
> not funny.

If you have META_MODE enabled -dM will tell you if meta_oodate
decided the target needs update - and if so exactly why.
Eg. a command changed, a file is updated, missing etc.

If it says nothing, then the target was out-of-date per normal make
rules.
 
> Why do I have to rebuild world twice to get WITH_META_MODE in effect?

I don't follow; why do you think that is the case?

> These setting dind't change over the past time, except some WITHOUT_ tags.
> 
> Are there any unrevealed secrets?

If changing any of those knobs impacts CFLAGS etc - then pretty much
everything will be out-of-date.

Adding -dM to your build command should be very instructive.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: WITH_META_MODE: any effect? Tree built twice!

2018-12-13 Thread Simon J. Gerraty
O. Hartmann  wrote:
> I'm able to reproduce this behaviour easily: make cleanworld; kldload filemon 
> (if not
> already loaded); make buildworld buildkernel

Why are you doing cleanworld?

> After the build has finished, install everything accordingly and reboot. Then 
> kldload
> filemon and make buildworld buildkernel in /usr/src - and be surprised.

Not surprised at all - all of your toolchain etc will be newer
and hence cause everything to be out of date.

I believe Bryan added some knobs to supress this particular check,
though I don't recall what they are.

It's not something we deal with here since we never actually install
what we are building on the build host.
I have trees I haven't cleaned in years - no need to.

> > Adding -dM to your build command should be very instructive.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: New vm-image size is much smaller than previos

2019-05-04 Thread Simon J. Gerraty
Warner Losh  wrote:
> > At the risk of being branded a wishful thinker, a firstboot script that
> > asked the user for some configuration information would be a great help
> > to both new and experienced foot-shooters. I'm thinking of Raspberry Pi,
> > but perhaps it applies to non-embedded platforms also.
> >
> 
> That's not a bad idea... we could press bsdinstall into service for that
> perhaps... we already expand the partition / filesystem to match the media
> size...

We run Junos in vm's under lots of different circumstances.

An etc/rc.d/vmguest is used to tailor the vm to the environment it finds
itself in at firstboot.  A raft of etc/rc.conf.d/vmguest.* help tailor
the basic logic to different platforms and vmhost environments.
It generally does nothing at subsequent boots.

We handle dynamic configuration, by providing a separate device to be
used for swap, var etc.  Dynamically sized swap is very handy.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Reducing UFS corruption from unclean shutdowns?

2019-06-21 Thread Simon J. Gerraty
Alan Somers  wrote:
> I would've thought that immediately following a sync(8), the
> filesystem would be consistent.  Why do I still see errors after a

sync(8) does little more than tell the kernel to start flushing some
pages - which the kernel would do anyway in about 30s
So, it does not really ensure a clean filesystem if you are about to
reboot - and certainly not if a panic is imminent.

FWIW to minimize fs problems after doing package install on junos I run
fsync on files and dirs I know are likely to have been updated and which
need to be flushed to disk before reboot.

Unlike sync(8), fsync(1) will not return until the I/O is done.

You may still lose data after a sudden power cycle or panic, but less
likely to be something critical.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: head -r351056 self-hosted amd64 installworld fails with: "don't know how to make _testsDATA_FILESINS1_data/md5test-in. Stop"

2019-08-14 Thread Simon J. Gerraty
Cy Schubert  wrote:
> > installworld is failing with:
> >
> > ===> lib/libc/tests/hash (install)
> > install  -o root  -g wheel -m 555  hash_test  
> > /usr/tests/lib/libc/hash/hash_t
> > est
> > make[7]: don't know how to make _testsDATA_FILESINS1_data/md5test-in. Stop

Sorry about that.  Looks like more extensive fix is needed.

> Revert r351055 and installworld complete for now.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: AMD Secure Encrypted Virtualization - FreeBSD Status?

2019-10-14 Thread Simon J. Gerraty
Tomasz CEDRO  wrote:

> would be really nice also to get UEFI BOOT compatible with SECURE BOOT :-)

Unless you are using your own BIOS, the above means getting Microsoft
to sign boot1.efi or similar. Shims that simply work around lack of
acceptible signature don't help.

That would need to then verify loader.efi - which can be built to
to verify all the modules and kernel.

In my implementation (uses the non efi loader) trust anchors are
embedded in loader but there is code in current to lookup trust anchors
in /efi I think which would be more generally useful - I've not looked
at the attack vectors that introduces though.

--sjg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: AMD Secure Encrypted Virtualization - FreeBSD Status?

2019-10-14 Thread Simon J. Gerraty
Clay Daniels Jr.  wrote:

> Simon, please do elaborate more on your implementation. I suspect you are
> talking about libsecureboot? I have played with the generation of certs
> with OpenSSL & LibreSSL, but libsecureboot seems to take a different
> approach. Please tell us more.

Yes I meant libsecureboot.

You should be able to create keys and certs with OpenSSL.
That's all we use, but we keep all the private keys etc isolated in
signing servers.  The local.trust.mk in libsecureboot leverages the
sign.py etc described at
http://www.crufty.net/sjg/blog/signing-server.htm
(which also contains a link to the src)

But that does not alter the fact that the certs are simply those created
by an OpenSSL based CA - there are a number of good tutorials on the net
on how to setup such things.

With all that said; you may find it more useful to use OpenPGP for
signing we again use sign.py to retrieve OpenPGP public key, but you can
do all you need using nothing more than gpg

For an embedded vendor like Juniper X.509 makes a lot of sense.
For an individual or small scale, OpenPGP is likely simpler.
libsecureboot supports both, but you need to tailor local.trust.mk to
suit.

IIRC you can have local.trust.mk simply set TA_PEM_LIST etc to paths of
pre-prepared pem files containing your trust anchors and ta.h
and/or TA_ASC_LIST to a list of .asc files containing ascii armored
openpgp trust anchors.

BTW in current boot1.efi is no more, loader.efi is used.
[I'm still mucking about trying to get a VM image booting using efi...]


> 
> Clay
> 
> On Mon, Oct 14, 2019 at 1:52 PM Simon J. Gerraty via freebsd-security <
> freebsd-secur...@freebsd.org> wrote:
> 
> > Tomasz CEDRO  wrote:
> >
> > > would be really nice also to get UEFI BOOT compatible with SECURE BOOT
> > :-)
> >
> > Unless you are using your own BIOS, the above means getting Microsoft
> > to sign boot1.efi or similar. Shims that simply work around lack of
> > acceptible signature don't help.
> >
> > That would need to then verify loader.efi - which can be built to
> > to verify all the modules and kernel.
> >
> > In my implementation (uses the non efi loader) trust anchors are
> > embedded in loader but there is code in current to lookup trust anchors
> > in /efi I think which would be more generally useful - I've not looked
> > at the attack vectors that introduces though.
> >
> > --sjg
> > ___
> > freebsd-secur...@freebsd.org mailing list
> > https://urldefense.com/v3/__https://lists.freebsd.org/mailman/listinfo/freebsd-security__;!8WoA6RjC81c!TLaVmP78NH0BviSHHV_3_V0-ispe2o0I7E59vmxZ_8XnbmOYxeHxemscoWsaXA$
> >  
> > To unsubscribe, send any mail to "freebsd-security-unsubscr...@freebsd.org
> > "
> >
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: bmake and .USEBEFORE

2015-01-31 Thread Simon J. Gerraty
Julian Elischer  wrote:

> On 1/28/15 1:41 PM, Julian Elischer wrote:
> > If I try the following:
> >
> > bar: .USE
> > @echo @ = $(@)
> > all: bar
> > @echo here is all
> oops
>  the failing example should be .USEBEFORE.. I pasted the wrong clip.
> >
> > I always get "bar is up to date"

If you put all: as the first target or add
.MAIN: all
or explicitly do make all

you will get the output you expect.

As is; 'bar' is the default target but it's .USEBEFORE 
which doesn't make a lot of sense (sort of being applied to itself ;-)
It would probably make sense for .USE* to imply .NOTMAIN.

Anyway, to illustrate the purpose of .USEBEFORE consider:

8<
.MAIN: all

u1: .USE
@echo u1 $@

u2: .USE
@echo u2 $@

ub: .USEBEFORE
@echo; echo ub $@

all: foo1 foo2 foo3

foo1: u1 u2 ub
foo2: u2 ub u1
foo3: u2 u1 ub
8<

when this makefile is run the output is

ub foo1
u1 foo1
u2 foo1

ub foo2
u2 foo2
u1 foo2

ub foo3
u2 foo3
u1 foo3

note that u1 and u2 are applied in the order given, but ub is always
done first.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [RFC] Removin the old make

2015-02-12 Thread Simon J. Gerraty
Ian Lepore  wrote:
> under bmake.  It's especially annoying because :L is really common in
> fmake and its meaning in bmake is all but useless.

Actually it is very useful. eg.

.if defined(NO_POSIX_SHELL) || ${type printf:L:sh:Mbuiltin} == ""
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: FreeBSD_HEAD_amd64_gcc4.9 - Build #85 - Failure

2015-06-14 Thread Simon J. Gerraty
Craig Rodrigues  wrote:
> On Sun, Jun 14, 2015 at 8:09 AM, Adrian Chadd  wrote:
> 
> + make -j 4 CROSS_TOOLCHAIN=amd64-gcc buildworld
> __MAKE_CONF=/builds/FreeBSD_HEAD_amd64_gcc4.9/make.conf
> make: "/builds/FreeBSD_HEAD_amd64_gcc4.9/Makefile" line 102: Malformed
> conditional (${MK_META_MODE} == "yes")

That would imply that src/share/mk isn't being used?

I'm not sure when Warner changed the default MAKESYSPATH to be
.../share/mk
but that would be the right thing to do:

either -m .../share/mk or 
export MAKESYSPATH=.../share/mk

of course you can also substiute the actual path for .../

> See:
> 
> https://lists.freebsd.org/pipermail/svn-src-all/2015-June/105777.html
> 
> I'm waiting for Simon to respond since I'm not sure if that is the
> correct fix.

You could make that just

.if ${MK_META_MODE:Uno} == "yes"

but using the right sys.mk would be better.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: FreeBSD_HEAD_amd64_gcc4.9 - Build #85 - Failure

2015-06-14 Thread Simon J. Gerraty
Garrett Cooper  wrote:

> On Jun 14, 2015, at 14:18, Simon J. Gerraty  wrote:
> 
> > Craig Rodrigues  wrote:
> >> On Sun, Jun 14, 2015 at 8:09 AM, Adrian Chadd  wrote:
> >> 
> >>+ make -j 4 CROSS_TOOLCHAIN=amd64-gcc buildworld
> >>__MAKE_CONF=/builds/FreeBSD_HEAD_amd64_gcc4.9/make.conf
> >>make: "/builds/FreeBSD_HEAD_amd64_gcc4.9/Makefile" line 102: Malformed
> >>conditional (${MK_META_MODE} == "yes")
> > 
> > That would imply that src/share/mk isn't being used?
> > 
> > I'm not sure when Warner changed the default MAKESYSPATH to be
> > .../share/mk
> > but that would be the right thing to do:
> 
> That’s only present on head.

The default? yes I think it went back and forth a bit.
But you can use

make -m .../share/mk

on 10.0
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: FreeBSD_HEAD_amd64_gcc4.9 - Build #85 - Failure

2015-06-15 Thread Simon J. Gerraty
Konstantin Belousov  wrote:
> I see the same problem on the up-to-date stable/10 host, trying to build

I'm building HEAD on stable/10, I just updated the tree and did a clean
tree buildworld ok.

> HEAD.  This is completely unacceptable, we have documented and always
> supported just
>   make buildworld buildkernel
> on latest stable to get buildable and bootable HEAD.

I think that proposition became shaking as soon as the current options
model was added to the build - any new option will break building 
unless you use the share/mk from head.

Hmm ok I've introduced the concept of options being set during sys.mk
which makes the above issue slightly worse.
Craig as already committed a fix for src/Makefile, 
hmm I thought it had logic to switch to the tree's share/mk
in which case the change to src/Makefile should be all that's needed.

> There is yet another issue with the build system, I have
>   INSTALL+=-CS
> in make.conf for around 15 years.  Apparently it is broken now.

Ah! make.conf is getting included earlier.
So that {local,src}.sys.mk can be included earlier so that any of them
can provide pointer to external toolchain before sys.mk sets CC etc.

That sucks, though that implies you are getting sys.mk from src/ at
least after the build gets going.

> The installkernel target results in
> ph/vjc (install)
> --- _kmodinstall ---
> CS -o root -g wheel -m 555   ng_vjc.ko 
> /usr/home/kostik/build/bsd/DEV/p/boot/kernel
> CS: not found
> etc.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Error building x11/nvidia-driver kernel module @r284408

2015-06-15 Thread Simon J. Gerraty
Garrett Cooper  wrote:
> > Now that "vanilla" head @284408 builds (& boots):


I fixed this the other day - just realized I haven't committed it.

> > make[6]: don't know how to make 
> > /common/S3/obj/usr/src/sys/CANARY/common/ports/x11/nvidia-driver/work/NVIDIA-FreeBSD-x86_64-346.47/src/machine.
> >  Stop
> > 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: geom classe's geom_*.so installed /usr/lib/ instead of /lib/geom (crochet build)

2015-06-15 Thread Simon J. Gerraty
Garrett Cooper  wrote:

> More breakage from projects/bmake. This should be fixed in theory, but
> bapt/sjg can confirm if it’s fixed.  Cheers,

Yes, sorry everyone - took a bit to identify the root cause
(ie. specific line)
*Should* be fixed now.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: FreeBSD_HEAD_amd64_gcc4.9 - Build #85 - Failure

2015-06-15 Thread Simon J. Gerraty
Garrett Cooper  wrote:
> >> There is yet another issue with the build system, I have
> >>INSTALL+=-CS
> >> in make.conf for around 15 years.  Apparently it is broken now.
> > 
> > Ah! make.conf is getting included earlier.
> > So that {local,src}.sys.mk can be included earlier so that any of them
> > can provide pointer to external toolchain before sys.mk sets CC etc.

> This is going to break a whole lot of stuff — especially because I’ve
> discovered people either try and be overly clever with make, or like
> .include’ing things more than once >:(.

I hope not, since the earlier inclusion is potentially very valuable.
 
Konstantin, sorry for the pain, putting

INSTALL?= install
INSTALL+=-CS

in your make.conf should restore your expected behavior - without losing
the ability to select toolchains etc.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Error building x11/nvidia-driver kernel module @r284408

2015-06-15 Thread Simon J. Gerraty
Thanks very much for the test David

David Wolfskill  wrote:
> OK; following up: I see Simon committed r284420; after hand-appling that
> (1-line fix); I performed:
...
> Each was successful:
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Error building x11/nvidia-driver kernel module @r284408

2015-06-15 Thread Simon J. Gerraty
Rainer Hurling  wrote:
> I just tried r284421 and get another error. My '/etc/src.conf' includes
> 'WITH_LLDB=1':

A couple of folk have issue with WITH_LLDB 
seeing if I can reproduce 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: toolchain target

2015-06-17 Thread Simon J. Gerraty
Andriy Gapon  wrote:
> > Seems like there is some problem with 'toolchain' target in the latest head.
> > Output of running `make toolchain TARGET=i386` on amd64 system can be found
> > here: http://dpaste.com/3RD3C4V
> > 
> 
> AFAICS, it still worked as of r283188.

There has been a clang update since then.

I just did make -j12 toolchain TARGET=i386 ok
do you have anything interesting in /etc/make.conf?
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: toolchain target

2015-06-18 Thread Simon J. Gerraty
Andriy Gapon  wrote:
> > do you have anything interesting in /etc/make.conf?
> 
> Thank you for the hint -- __MAKE_CONF=/dev/null SRC_CONF=/dev/null fix the 
> problem.
> 
> Now I am trying to figure out what the problem is.

The problem will be that I shifted the include of make.conf and etc
to earlier in sys.mk so that they can provide pointer to external
toolchains.

> 
> My make.conf:
> .if defined(CC)
> .if ${CC} == gcc

which will break all this - see the UPDATING entry.

> CPUTYPE?=k8-sse3
> .else
> CPUTYPE?=amdfam10
> .endif
> .endif
> CFLAGS+= -O2 -fno-strict-aliasing -pipe
> CFLAGS+= -fno-omit-frame-pointer
> CXXFLAGS+= -O2 -fno-strict-aliasing -pipe

The root problem is that a global file like /etc/make.conf is a very
fragile concept - makes it almost impossible to evolve the the build.
Especially if you put anything in there more complex that setting global
knobs.

Any use of += etc, makes it sentitive to the point of inclusion.

It is useful to include local.sys.mk (hence src.sys.mk) very early.
So that they can do CC?= blah etc. before sys.mk does

But the current semantic is that make.conf should be included first
which means it needs to be included early .. which breaks everyone who
has "interesting" stuff there.

> And src.conf:
> WITH_DEBUG_FILES=yes
> WITH_CTF=yes

What is perhaps really needed is yet another include that can happen
early and another later during sys.mk processing
We could then move include of make.conf back 

Will investigate...

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: toolchain target

2015-06-18 Thread Simon J. Gerraty
Dimitry Andric  wrote:
> Hmm, is this still not fixed?  This was broken by Simon's large
> "meta-mode" commit r284345. but I would assume Baptiste's fixes after
> that might have fixed it. I can't test this myself at the moment, due to
> ENOTIME...

I think https://reviews.freebsd.org/D2860 will fix this.
It puts include of make.conf etc back where they were and
introduces new early include.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: amd64 base/head r284673 fails to build on amd64 base/head r284639, pilot error?

2015-06-21 Thread Simon J. Gerraty
Garrett Cooper  wrote:
> > Am I the only one who fails to build recent base/head (r284673) on
> > pretty recent base/head (r284639)? This is on amd64 with ZFS and BEs.
> 
> ...
> 
> > CC=clang
> > CXX=clang++
> > CPP=clang-cpp

>   You need to remove these lines. They shouldn’t have been set before or 
> after the commits from projects/bmake .

Note: both the grn's specified above are  than r284598 which put the
inlcude of make.conf back to its original spot, so the meta mode related
changes should not be relevant.



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: powerpc and powerpc64 builds broken

2015-06-30 Thread Simon J. Gerraty
Justin Hibbits  wrote:
> > Yeah, crossbuilds work fine. It's the actual run-on-real-hardware bit
> > that doesn't.
> >
> > (powerpc64 runs fine in qemu-devel; people should try it!)
> 
> r284345 (introduction of metamode) is the problem.  I bisected it on
> the power8 and it reliably fails this way.  Andreas tested it on his
> G4 (32-bit hardware) and it worked fine, so there's something
> introduced that just doesn't like powerpc64.

Hmm, didn't touch usr.bin/mkesdb so must be something indirect.
Is this with[out] NO_CLEAN ?
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: "proper way" or "unworkable idea" ?

2015-06-30 Thread Simon J. Gerraty
Jeffrey Bouquet  wrote:

> If I've a spare /mnt/usr/src , it seems buildworld quite soon fails,
> where it otherwise may succeed in /usr/src. Any CLI parameters or> the
> build system is hardcoded enough so that there will always be
> problems?

The only thing hard coded is the default MAKEOBJDIRPREFIX (it isn't
called that but it works the same way), but even that should work for any
location. I always have MAKEOBJDIRPREFIX when doing buildworld etc,
and have never used /usr/src.

Is there perhaps something interesting about /mnt/usr/src (like
ancient?)


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Parallel release failed on pwd_mkdb

2015-07-06 Thread Simon J. Gerraty
Oliver Pinter  wrote:
> We got this build failure, when two release make release running in parallel:

Can you elaborate what you mean by "two release make release" ?
Do you mean two separate builds running in the same tree at the same
time using the same DESTDIR?

> >>> stage 4.4: building everything
> --
> pwd_mkdb: /var/tmp/temproot/etc/pwd.db.tmp to
> /var/tmp/temproot/etc/pwd.db: No such file or directory
> *** Error code 1

The pwd_mkdb command in etc uses $DESTDIR/etc - it obviously assumes
that is unique.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [PATCH] uart_core: start countdown for non-interrupt mode

2015-07-09 Thread Simon J. Gerraty
Aleksey Kuleshov  wrote:

> The uart_intr will never be called if interrupts are not available.
> Start counter with callout_reset call.

FWIW this fixed an issue we were investgating.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: -current broken when src is on NFS

2015-07-18 Thread Simon J. Gerraty
O'Connor, Daniel  wrote:
> However, Crochet _does_ build on the NFS client _and_ when the source
> tree isn't in /usr/src which makes this issue very strange :-/

I've seen similar errors in rescue... (no NFS) though I cannot 
quite recall the cause other than it seems very sensitive
to MAKEOBJDIRPREFIX value.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: -current broken when src is on NFS

2015-07-19 Thread Simon J. Gerraty
> Given that we have (or at least had at one time) some of those magical
> "..." paths that cause bmake to search up the hierarchy for its .mk
> files, I wonder if an odd relationship between src and obj dir confuses
> it, or if it somehow wanders into a wrong src tree while searching?

Based on Daniel's latest mail, the issue would appear to be the way he
was setting MAKEOBJDIRPREFIX.

As for .../ src.sys.env.mk resolves that to absolute path
since otherwise things like kernel builds wouldn't work - make run from
within obj tree.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: -current broken when MAKEOBJDIRPREFIX is set (was: src is on NFS)

2015-07-19 Thread Simon J. Gerraty
O'Connor, Daniel  wrote:
> Yeah the subject is wrong (I just updated it).
> 
> I just did a build like so and it worked..
> env MAKEOBJDIRPREFIX=/src/obj-amd64 make -j 8 buildworld

That's the right way to use it.

> But this did not..
> make -j 8 buildworld MAKEOBJDIRPREFIX=/src/obj-amd64

Nor should it.
There are several makefiles in the tree that expect to be able to 
change MAKEOBJDIRPREFIX in the environment of a sub-make.
When you set it on the command line like that you prevent such changes
from working.

> So, it seems MAKEOBJDIRPREFIX only works as an environmental variable

Yes, it has always been documented that way.

> - I wonder if there is a way the make system can be changed to warn
> about that?

I thought there was a check in src/Makefile for that.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: -current broken when MAKEOBJDIRPREFIX is set (was: src is on NFS)

2015-07-19 Thread Simon J. Gerraty
O'Connor, Daniel  wrote:
> >> So, it seems MAKEOBJDIRPREFIX only works as an environmental variable

> Weird, I could have sworn I have set it on the command line and had it
> work, but..

In most "normal" usage you will likely not notice a difference.
It is only when a makefile is "being clever" that things go south when
you break its expectations.

> > I thought there was a check in src/Makefile for that.
> 
> Not so far as I can tell - it certainly gets quite far before blowing
> up with a non useful error message :)

The check in src/Makefile is only guarding against MAKEOBJDIRPREFIX set
in say make.conf, it explicitly discards the possibility of setting
MAKEOBJDIRPREFIX on command line - which seems wrong.

Also with bmake, you *can* usefully set MAKEOBJDIRPREFIX in a makefile
since the choice of .OBJDIR can be made after make starts reading
makefiles.

So the current test is perhaps out dated on that score too - though
only if something like auto.obj.mk is being used (-DWITH_AUTO_OBJ)
which I don't think I've tested with buildworld.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: amd64 base/head r284673 fails to build on amd64 base/head r284639, pilot error?

2015-08-01 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> 1: subdir make
>   src.conf: STRIP=
>   rescue/rescue% make all
>   -> make -f OBJDIR/rescue.mk
> 
>  STRIP= is not passed down into rescue.mk, resulting in 'strip rescue'.

unless src.conf does .export STRIP, or submake reads src.conf for itself
this isn't surprising.
The result probably depends on whether /usr/share/mk or src/share/mk is
used.

> 
> 2. subdir make STRIP env override
>   rescue/rescue% make all STRIP=

Setting STRIP= (or anything) like that causes vastly different behavior
compared to 

env STRIP= make
or setting STRIP= in a makefile (eg src.conf)

When you do 'make STRIP=' the variable is handled differently,
it is put into a special context CMD which has higest priority
and is explicityl passed on to submakes via MAKEFLAGS.
The goal is to treat the setting as immutable (to the extent possible)
as required by posix.


>  STRIP= is passed down resulting in ' rescue'.
> 
> 3: buildworld
>  STRIP= from src.conf is passed down, resulting in ' rescue'.

buildworld tries to ensure that the tree's share/mk is used
so src.sys.mk is used and /etc/src.conf likely to be read.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: amd64 base/head r284673 fails to build on amd64 base/head r284639, pilot error?

2015-08-01 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> I think the problem here is the use of -m for SUB_MAKE in /Makefile.
> Specifying -m share/mk causes all of the issues I've seen (expected
> including of /etc/src.conf), while not using -m does not include
> /etc/src.conf even though the build is being done in a src dir.

Sorry, the behavior here will vary depending on when /usr/bin/make was built.

IIRC when I first put bmake in I made it use .../share/mk by default
(during the original discussion at BSDCan about 4 years ago
ensuring that "correct" src/share/mk used by default was stated as a
goal - without need for env settings).

Of course .../share/mk breaks POLA for those who expect /usr/share/mk to
always be used, so that was backed out, but was more recently put back
in ;-)

Sooo unless you explicitly use -m or set MAKESYSPATH, we cannot say for
sure whether you will get /usr/share/mk/sys.mk or
$SRCTOP/share/mk/sys.mk when you run 'make' in a subdir, and the result
wrt to src.conf depends on using $SRCTOP/share/mk/sys.mk

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [CFT] Buildworld ccache support

2015-10-27 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> https://people.freebsd.org/~bdrewery/patches/world-ccache.diff

In the Junos build - where we used ccache for quite some time
I did:

_CC := ${CC}
CC = ${CCACHE_ENV} ${_CC}

Since sometimes you want the compiler without ccache - eg when linking.

That needs to happen after CC is determined of course.

If the include of bsd.mkopt.mk were moved to after the inlcude of
local.sys.env.mk, then you could simply:

__DEFAULT_NO_OPTIONS+= CCACHE_BUILD

there or in src.sys.env.mk

> 
> To use just set WITH_CCACHE_BUILD= in src.conf or make.conf.  I
> purposely matched it to the same as the ports build.
> 
> Thanks!
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Compilation failure with WITH_DTRACE_TESTS on mips/mips and powerpc/powerpc; it's trying to compile a host object on mips/powerpc because MAKEOBJDIRPREFIX is incorrect

2015-10-30 Thread Simon J. Gerraty
NGie Cooper  wrote:
>   I tried doing buildworld on powerpc/powerpc with -DWITH_DTRACE_TESTS 
> and I ran into this linker issue below. I have no idea (yet) why it’s trying 
> to compile an x64 object when I specify powerpc/powerpc — and more 
> importantly, why is the object not being put in obj.powerpc?
>   I ran into the same issue on ref11-amd64.freebsd.org when I ran “make 
> tinderbox".

Is it possible that the file is left over from a previous build (of amd64?)

Does your log show it being built?


> dtrace -C -x nolibs -G -o usdt.o -s 
> /usr/src/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d 
> tst.usdt.o
> dtrace: failed to link script 
> /usr/src/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d: 
> incorrect ELF machine type for object file: tst.usdt.o
> *** Error code 1
> $ find /usr/obj/usr/src/svn/ -name tst.usdt.o
> /usr/obj/usr/src/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o
> $ file /usr/obj/usr/src/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o
> /usr/obj/usr/src/svn/cddl/usr.sbin/dtrace/tests/common/json/tst.usdt.o: ELF 
> 64-bit LSB relocatable, x86-64, version 1 (FreeBSD), not stripped
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Compilation failure with WITH_DTRACE_TESTS on mips/mips and powerpc/powerpc; it's trying to compile a host object on mips/powerpc because MAKEOBJDIRPREFIX is incorrect

2015-11-01 Thread Simon J. Gerraty
NGie Cooper  wrote:

> And here’s the real issue — .PATH is completely broken when
> TARGET/TARGET_ARCH are specified as explicit values:

It would help if you could indicate what you think the right value
should be.
 
> $ env MAKEOBJDIRPREFIX=/scratch/tmp/ngie/obj/ make buildenv TARGET=powerpc 
> TARGET_ARCH=powerpc
> Entering world for powerpc:powerpc
> $ cd cddl/usr.sbin/dtrace/tests/common/json
> $ make -V.OBJDIR
> /scratch/tmp/ngie/obj//powerpc.powerpc/scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json
> $ make -VMAKEOBJDIRPREFIX
> /scratch/tmp/ngie/obj//powerpc.powerpc
> $ make depend
> (cd /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json &&  
> DEPENDFILE=.depend.tst.usdt.exe  NO_SUBDIR=1 make -f 
> /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/Makefile 
> _RECURSING_PROGS=  PROG=tst.usdt.exe  depend)

If you are doing this on current (ie MAKE_VERSION==20151020), adding -w
would be useful, since will report entering src and obj dirs.

> $ make all
> (cd /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json &&  
> DEPENDFILE=.depend.tst.usdt.exe  NO_SUBDIR=1 make -f 
> /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json/Makefile 
> _RECURSING_PROGS=  PROG=tst.usdt.exe )
> dtrace -C -x nolibs -G -o usdt.o -s 
> /scratch/tmp/ngie/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d
>  tst.usdt.o
> dtrace: failed to link script 
> /scratch/tmp/ngie/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d:
>  incorrect ELF machine type for object file: tst.usdt.o

> Stop.
> make[2]: stopped in 
> /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json
> $ make -V.PATH

what dir do you execute that in?
I'm *guessing* cddl/usr.sbin/dtrace/tests/common/json.

It's actually quite useful when reporting/describing problems to do
everything from src eg.

make -w -C cddl/usr.sbin/dtrace/tests/common/json

leaves very little room for confusion.


> . /scratch/tmp/ngie/svn/cddl/usr.sbin/dtrace/tests/common/json 
> /scratch/tmp/ngie/svn/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json

What else do you expect in .PATH?
I didn't see anything in the Makefile or ../../Makefile.inc1 to add
anything else

You may also find it useful to set MAKE_PRINT_VAR_ON_ERROR
to a list of variables - that will be reported when make dies.
eg.

MAKE_PRINT_VAR_ON_ERROR=".CURDIR .OBJDIR MACHINE MACHINE_ARCH .PATH"

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Failing buildword due to execution permission (with fix)

2015-11-09 Thread Simon J. Gerraty
> >> include/Makefile:MK_OSRELDATE_SH=   ${.CURDIR}/mk-osreldate.sh
> >> include/Makefile:   sh ${MK_OSRELDATE_SH}
> 
> I actually wrote up a patch recently to use ${SH} in all places of 'sh'
> and '/bin/sh', and noted on SHELL?= that was not useful to use, but did
> not commit it (yet).

We (junos) use HOST_SH I think for this, since if building on a non-native
host, /bin/sh cannot be relied on.
FreeBSD sys.mk already has

__MAKE_SHELL?=/bin/sh

so I guess you could use that.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: libXO-ification - Why - and is it a symptom of deeper issues?

2015-11-15 Thread Simon J. Gerraty
Hi Dan

> Meaning, is that simple to push things in head , if somone does the
> work, even with with no proper review of the problem at hand , and the
> proposed solutions ?

Not sure what sort of review you are looking for.
But I can speak to some of the history behind this.

FreeBSD holds regular "vendor summit" meetings where folk using FreeBSD
in products can get together - sort of a swap meet (I have this, I need
this ...).

At one such meeting at BSDCan 2011, I metioned that we (Juniper) had
tweaked many of the startard tools to output XML, and asked if anyone
else was interested in the facility.
I was frankly surprised at the number of hands raised.
There was clearly demand from a segment of the FreeBSD community.

The ability to get machine parsable output from OS components is a big
part of the success of Junos CLI, netconf etc.

It took a few years to get some time from phil@ to design a solution
that we could consider upstreaming - libxo was the result.
BTW this is a "problem space" that phil has been deeply involved in for
over 15 years, so yes I think we can say the problem has been studied.

That demand I mentioned? also resulted in a GSoC project to do the same
thing - though it was much like approach that Juniper had done over a
decade ago that we had considered unsuitable for upstreaming.
But it rather clearly demonstrates that there was demand beyond the whim
of a small group of folk or just one company pushing  something unwanted
into the project.
The number of developers who have jumped in to XO'ify apps also speaks
to that.

The original proposal was for XML only (that's what we'd used and found
useful), but others wanted JSON as well.
Libxo can also output very rich HTML - I forget which HTML or JSON, is
used but this allows some seriously slick UI's to be implemented using a
modern web browser.

Hope that helps
--sjg

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Need help fixing failing locale tests

2015-11-15 Thread Simon J. Gerraty
Craig Rodrigues  wrote:
> I don't know much about locales, so don't know what to do.

I find LANG=C LC_ALL=C solves most locale induced issues.
I suspect the tests in question assume the above anyway.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: libXO-ification - Why - and is it a symptom of deeper issues?

2015-11-15 Thread Simon J. Gerraty
Garrett Cooper  wrote:
> We lack a [dtd/json] spec for tools, so programming for xo'ification
> doesn't seems like the best idea in the world to me from a end-user
> sysadmin/developer perspective.

A dtd etc is good for sure, and we (Juniper) do have them, as well as
ui-police to help ensure things go smoothly - even that doesn't catch
everything.
But all that is a layer above something like libxo - which is just the
mechanism.
All that is fundamentally required to obtain a reasonable result is
consistent use of verbs and nouns.

> I could just as easily use standard tools like awk, grep, sed, and
> more advanced languages like perl or Python to parse output, and

If you are parsing plain text output - that is all the data you have
and it is a small subset of what the app knows.

XML and HTML allow the app to provide lots of ancilliary/meta data
that is invaluable in doing clever things with the data.
Eg. by simply marking something as hostname/ipaddr the ui can hook in
pulldown menus to let you do things with that info.
I don't know if we've released anything I can point you at easily
but 5 minutes would suffice to make a believer of you ;-)

BTW libifying apps is a nice thing too, but does not eliminate the need
to do all the exact same UI work - just changes where you do it, since
the libraries themselves would need to be XO'ified to be useful.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: libXO-ification - Why - and is it a symptom of deeper issues?

2015-11-16 Thread Simon J. Gerraty
Dan Partelly  wrote:
> >> The ability to get machine parsable output from OS components is a big 
> >> part of the success of Junos CLI, netconf etc.
> 
> Once you get machine parsable output, and feed it to your GUIs , WEB,
> other tools, and modify it, how do you feed it back to your underlying
> OS ?

We didn't make any changes to the way tools are run as there was no
need.

All requests - whether from CLI, NETCONF etc, are channeled into the MGD
(management daemon) to verify input, permissions etc (Junos implements
very fine grained permission system), and act accordingly.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: libXO-ification - Why - and is it a symptom of deeper issues?

2015-11-17 Thread Simon J. Gerraty
Dan Partelly  wrote:
>   Juniper can further help FreeBSD by donating the code of their
>   system management daemon and their fine granularity permissions

At the cost of i18n etc?
The Junos UI is totally data driven, syntax is verified term by term
(since depending on your permissions some terms simply do not exist for
you).   Such a model cannot be successfully translated to other
languages where the order of verbs and nouns differ for example.

Everything I've read on the topic suggests that messages must be
translated on at least phrase if not sentence granularity for reasonable
results, and that just doesn't fit our UI.
Thus enhancement requests for i18n are politely rejected.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: libXO-ification - Why - and is it a symptom of deeper issues?

2015-11-17 Thread Simon J. Gerraty
Dan Partelly  wrote:
> Management daemon with fine grained permission, extremely
> useful. Would Juniper consider donating to FreeBSD under a BSD license
> portions of this code, the MGD, which could be reused in FreeBSD ? A

Right now I suspect the answer to that might be "no".
We know we have competitors who would dearly like that ;-)
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: sys/modules "make clean" seems broken again

2015-12-01 Thread Simon J. Gerraty
John Baldwin  wrote:

> +CLEANFILES+= ${_MFILES:R:S/$/.c/} ${_MFILES:R:S/$/.h/}

Since CLEANFILES is given to rm, you can use globs

CLEANFILES+= ${_MFILES:R:S/$/.[ch]/} or .?

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0-CURRENT SRC_ENV_CONF file vs. MAKEOBJDIRPREFIX in the file: they do not mix

2015-12-07 Thread Simon J. Gerraty
Mark Millard  wrote:
> My guess is that it is picking up the
> 
> MAKEOBJDIRPREFIX=/usr/obj/xtoolchain

You should use ?= if you want this to work.
There are many places in Makefile.inc1 where MAKEOBJDIRPREFIX is tweaked
in the environment of a sub-make.

By using = above, you break that.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0-CURRENT SRC_ENV_CONF file vs. MAKEOBJDIRPREFIX in the file: they do not mix

2015-12-07 Thread Simon J. Gerraty
Mark Millard  wrote:
> >> MAKEOBJDIRPREFIX=/usr/obj/xtoolchain
> > 
> > You should use ?= if you want this to work.
> > There are many places in Makefile.inc1 where MAKEOBJDIRPREFIX is tweaked
> > in the environment of a sub-make.
> > 
> > By using = above, you break that.
> 
> That presumes that MAKEOBJDIRPREFIX has not been assigned a default
> value before the SRC_ENV_CONF file has been included the first
> time.

Yes. If that's a concern:

.if ${.MAKE.LEVEL} == 0
MAKEOBJDIRPREFIX= /usr/obj/xtoolchain
.export MAKEOBJDIRPREFIX
.endif

will do what you want.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: My build work and goals

2015-12-10 Thread Simon J. Gerraty
Firstly, nice writup - a few extra blank lines might have helped my eyes
though.

Bryan Drewery  wrote:
> This mail is to outline my recent work, goals and motivations.  This is
> long.  This is not really a architectural review or discussion mail.  I
...

> Some problems with the combined DIRDEPS_BUILD build:
> 
> It is very hard to maintain, prone to massive churn in directories you
> did and did not touch, requires checking in fickle generated files, has
> a chicken-and-egg problem for adding new things, does not respect
> WITH/WITHOUT options or ARCH-dependent SUBDIR, has subtle problems with
> a static dependencies list around options, and has no 'installworld'
> support.

Dealing with all the optional cases is definitely a problem to be
solved.
I've a few ideas on that front, but haven't had time to experiment.

FWIW we (Juniper) see zero "churn" in our dependencies
a/ we use a fixed set of options
b/ we've fixed the bugs in makefiles that were causing churn.

Ignoring the WITH/WITHOUT options stuff for a sec, dependency churn is
usually a result of bugs in the leaf makefiles.
These are often easy to spot, sometimes quite hard.

The bootstrap issue and even just the need to checkin "generated" files
is a head scratcher for a surprising number of people - even after 4+
years of building this way.

> The lack of 'installworld' is a deal breaker, albeit probably trivial to
> fix.  The reason for this is that the system is used to generate
> packages (not pkgng style) at Juniper.  We have our own packaging effort
> occurring in projects/release-pkg though and won't likely use the same
> methods that the DIRDEPS_BUILD does (of which none of the support is
> really in the src tree).

Though AFAIK all the tools we use are, so its really just a matter of
adding some suitable makefiles and manifest - to say build a bootable vm
image - I probably would have done that by now if bhyve supported vmdk
or qcow2 ;-)

But that's likely orthogonal to your point.
Some form of "installworld" is probably needed.

> This build does not use recursive SUBDIR walking (by design).  It uses a
> static list of directories to build in the
> targets/pseudo/userland/*/Makefile.depend files.

That's mostly just for the purpose of demonstration/example.

If/when FreeBSD has some form of manifest/list for what goes into its
"packages" (of whatever form they might be,) then the necessary data
to drive what needs to be built could be obtained from them - as we do
in the Junos build.

> The checked-in Makefile.depend files have a flaw (feature for static
> builds perhaps) that the invoking the linker will cause non-direct
> dependencies to show up in Makefile.depend.  This leads to situations
> such as https://svnweb.freebsd.org/changeset/base/291558.  There is also
> a situation where a local build may set MK_SSP=no and yet need to build
> a dependency that wants MK_SSP=yes (libssp dependency) but because of

Yes, options are a pain.
You can either hard code them via local.dirdeps.mk but it is probably
better to do something a bit like what the current intermediate
makefiles do (one of the idea I mentioned above).
Eg. if the captured dependencies had something like ${lib_libssp} rather than
lib/libssp. If MK_SSP=no it expands to nothing.
I can elaborate in separate thread if you prefer.

> than upstream, is an exhaustive chicken-and-egg problem.  These problems
> make checked-in Makefile.depend infeasible.  "But wait, can you just not
> check them in?"  Not really.

You can, but you lose virtually all the benefits.

> recently added support to local.dirdeps.mk to make an educated guess on
> what DIRDEPS should be even if there is no Makefile.depend.  This is
> based on what is being built (C and C++ objects have common
> dependencies) and what DPADD/LIBADD there are.  In most cases this is
> enough.  It has greatly helped when bootstrapping Makefile.depend.

You still have the issue of local depenenencies.
Sure, only a small percentage of makefiles have problems in this regard,
but humans are generally bad at expressing the dependencies that result
from using things like yacc - get it wrong about 80% of the time.

> Going further though I have written a script that just recursively walks
> the tree to generate the DIRDEPS graph from the first make process
> rather than including the Makefile.depend files.  This removes the need

That's what we used to do, but was deemed too expensive.
Using that to generate Makefile.depend though for your tree, could save
you some overhead on subsequent builds, while allowing tailorying tree
to your set of options.

> P.S. Working on this stuff can be exhausting.  Mistakes are easy.

Thanks for your efforts btw.

--sjg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: My build work and goals

2015-12-11 Thread Simon J. Gerraty
Julian Elischer  wrote:
> > Some improvements I have made recently:
> > - WITH_FAST_DEPEND: Replacing the antiquated 'make depend'/'mkdep' with
> > compiler dependency flags to generate the .depend files as a side-effect
> > of compiling.  This saves tremendous time in buildworld and buildkernel.
> 
> Mach used to have this through their entire (bsd 4.3) tree. I forget

Junos built this way for over 10 years
but it only helps for the C/C++ code.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: make .SUFFIXES bug?

2015-12-15 Thread Simon J. Gerraty
Carsten Kunze  wrote:

> current groff doesn't build on FreeBSD.  I had noticed the same issue
> some months ago on NetBSD and cross checked on FreeBSD and it had
> worked on FreeBSD.  There must have somethig changed since then.  How
> to reproduce:

FreeBSD now uses same make as NetBSD ;-)

> When there is a file "test.1.man" and a makefile:
> 
> .SUFFIXES:
> .SUFFIXES: .roff .in .ps .mom .pdf .me .ms .ps .html .txt .texi .dvi .pdf 
> .xhtml .man .c .cpp .log .o .obj .sed .sin .test .test$(EXEEXT) .trs .ypp

What is the value of EXEEXT at this point?

> .man:
> @echo Making $@ from $<
> rm -f $@
> @LC_ALL=C \
>  sed -e "s|foo|bar|g" \
>  $< >$@
> 
> "make test.1" results in "make: don't know how to make test.1. Stop".
> 
> When ".man" is put to the start of the list it works.  It also works
> when the first .SUFFIXES line is removed.
> 
> The answer from NetBSD is that this is very likely a bug in make.  May
> this also be the case for FreeBSD?

Since the make is the same, the answer would be yes.
But it may not be a bug in make itself:

$ echo hi > test.1.man
$ make -r -f /homes/sjg/make-tests/suffix2 test.1
Making test.1 from test.1.man
$ make  -f /homes/sjg/make-tests/suffix2 test.1
make: don't know how to make test.1. Stop

make: stopped in /tmp
$

Will look deeper...
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: make .SUFFIXES bug?

2015-12-15 Thread Simon J. Gerraty
Simon J. Gerraty  wrote:
> Carsten Kunze  wrote:
> > current groff doesn't build on FreeBSD.  I had noticed the same issue
> > some months ago on NetBSD and cross checked on FreeBSD and it had
> > worked on FreeBSD.  There must have somethig changed since then.  How
> > to reproduce:

> > The answer from NetBSD is that this is very likely a bug in make.  May
> > this also be the case for FreeBSD?

Yes it is.  Fix is simple enough.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Aw: Re: make .SUFFIXES bug?

2015-12-28 Thread Simon J. Gerraty
Carsten Kunze  wrote:

> Thomas Dickey  wrote:
> > On Tue, Dec 15, 2015 at 04:01:41PM +0100, Carsten Kunze wrote:
> > > current groff doesn't build on FreeBSD.  I had noticed the same issue some
> > > months ago on NetBSD and cross checked on FreeBSD and it had worked on

This should be fixed in head, will MFC to stable/10 soon.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: xo_config.h missing?

2016-03-19 Thread Simon J. Gerraty
Jung-uk Kim  wrote:
> The attached patch fixed the build issues for me.

Since xo_config.h does not look like part of public api, this seems
appropriate.

I've committed this, while I check for other fallout.

Thanks

> Jung-uk Kim
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: /usr/bin/make segmentation fault

2016-04-02 Thread Simon J. Gerraty
Roger Marquis  wrote:

> Don't know how to debug this and cannot post the Makefile in question but it

Can you provide something similar that triggers the issue?
It's rather hard to tell what's wrong without knowing what *should* be
happening.

> last worked in 8.4.  In 11-CURRENT Var_Value appears to return NULL at
> /usr/src/contrib/bmake/compat.c:621
> 
>   Var_Set(IMPSRC, Var_Value(TARGET, gn, &p1), pgn, 0);
> 
> which passes the NULL to Var_Set as the second argument (val) where,
> eventually, /usr/src/contrib/bmake/var.c:973

Which means TARGET hasn't been set, and there's probably something
rather interesting about your makefile ;-)
This should of course never happen.

Even if Var_Set checked the failure and returned harmlessly you have a
serious problem.

> Any and all pointers appreciated,

What is content of gn at

> #2  0x00402040 in Compat_Make (gnp=0x800a1c340,
> #pgnp=0x800a1df00)
> at /usr/src/contrib/bmake/compat.c:621
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [CFT] WITH_META_MODE: Working incremental build [only on i386 and amd64]

2016-05-29 Thread Simon J. Gerraty
Mark Millard  wrote:
> > .if ${MACHINE_CPUARCH} == "i386" || ${MACHINE_CPUARCH} == "amd64"
> . . .
> > _filemon=   filemon
> . . .
> 

> Thus, for example, arm variants (32 bit and 64 bit) and powerpc
> variants (32bit and 64 bit) do not have WITH_META_MODE as an option as
> things are set up.

FWIW I'm not aware of any reason that filemon(4) wouldn't work on any
architecture.  I expect the above restriction is mostly just a
reflection of expected build hosts.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [CFT] WITH_META_MODE: Working incremental build

2016-05-31 Thread Simon J. Gerraty
> Another reported issue just now is that right after an installworld,
> everything rebuilds due to changed /bin/sh (-dM flag to make tells you
> why things rebuild).  I'll look into some mitigations for this.

It is probably sufficient to just add

.MAKE.META.IGNORE_PATHS += ${__MAKE_SHELL}
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [CFT] WITH_META_MODE: Working incremental build

2016-06-02 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> Yup, it's not really simple to fix.  This problem defeats the goal of
> the feature too. I had not ran into this case in all of my testing since
> I wasn't installing to /.

I never do that either (except for bmake).
I'm guessing that installworld it is a rare event - compared to building.

Touching everything in objtree as part of that might be a crude
workaround.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301139: WITH_META_MODE=yes vs. "sh: ./make_keys: Exec format error"?

2016-06-02 Thread Simon J. Gerraty
Mark Millard  wrote:

>  Building /usr/obj/clang/arm.armv6/usr/src/lib/ncurses/ncursesw/make_keys
>  sh: ./make_keys: Exec format error

This is an arm host or cross-building?

The error suggests HOST_CC got the wrong value.
You should be able to look at

/usr/obj/clang/arm.armv6/usr/src/lib/ncurses/ncursesw/make_keys.meta

to confirm whether the right toolchain was used.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301139: WITH_META_MODE=yes vs. "sh: ./make_keys: Exec format error"?

2016-06-02 Thread Simon J. Gerraty
BTW Mark, thanks very much for testing this.

> > # grep make_keys 
> > ~/sys_typescripts/typescript_make_rpi2_nodebug_clang_bootstrap-amd64-host-2016-06-01:15:17:28
> > Building /usr/obj/clang/arm.armv6/usr/src/lib/ncurses/ncursesw/make_keys
> > Building /usr/obj/clang/arm.armv6/usr/src/lib/ncurses/ncurses/make_keys
> > Building /usr/obj/clang/arm.armv6/usr/src/lib/ncurses/ncursesw/make_keys
> > sh: ./make_keys: Exec format error
> 
> Note that ncursesw has two Building lines above with the same path listed.

This build is still using the normal orchestration (tree walks etc)
so it it not at all out of the question for directories to be visited
more than once.

If curious; you can add -dM to have make explain why it built it again
the 2nd time.
The output can be copious, so you might want to only enable it in
ncursesw
eg.

.MAKEFLAGS: -dM

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: mergemaster internally using make [for example] vs. WITH_META_MODE?

2016-06-11 Thread Simon J. Gerraty
Mark Millard  wrote:
> > # grep -i make /usr/sbin/mergemaster | more
> . . .
> > MM_MAKE="make ${ARCHSTRING} -m ${SOURCEDIR}/share/mk"
> > ${MM_MAKE} DESTDIR=${DESTDIR} distrib-dirs >/dev/null
> >   ${MM_MAKE} DESTDIR=${TEMPROOT} distrib-dirs >/dev/null &&
> >   ${MM_MAKE} _obj SUBDIR_OVERRIDE=etc >/dev/null &&
> >   ${MM_MAKE} everything SUBDIR_OVERRIDE=etc >/dev/null &&
> >   ${MM_MAKE} DESTDIR=${TEMPROOT} distribution >/dev/null;} ||
> . . .
> 
> If one is using WITH_META_MODE= for buildworld, buidlkernel,
> installkernel, installworld what is appropriate for scripts or other
> uses of make for other makefile-targets?

I cannot speak to your specific example, but yes in general there are
cases where you want a sub-make to not run in meta mode.

Using 'make -B' is usually sufficient.
Eg.

$ make -C bin/cat -V .MAKE.MODE
[Creating objdir /tank/home/sjg/work/FreeBSD/current/obj/i386.i386/bin/cat...]
missing-meta=yes missing-filemon=yes meta verbose silent=yes
$
$ make -C bin/cat -B -V .MAKE.MODE
compat
$

If you have a makefile that should *never* run in meta mode but you
don't necessarily want to run in compat mode you can put:

.MAKE.MODE= normal

in the makefile
 
> Are there explicit mixes of using WITH_META_MODE= for some makefile
> targets and not using WITH_META_MODE= for other makefile targets that
> need to be avoided? Does one need to force some scripts to use [or not
> use] WITH_META_MODE= for their "internal" make usage?

The most common case where you want to use -B or otherwise supress meta
mode, is when you are using 'make -V' to print the value of some
variable.

--sjg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301139: WITH_META_MODE=yes vs. "sh: ./make_keys: Exec format error"? [still true of -r301815]

2016-06-12 Thread Simon J. Gerraty
Mark Millard  wrote:
> > --- build-tools_lib/ncurses/ncursesw ---
> > Building /usr/obj/clang/arm.armv6/usr/src/lib/ncurses/ncursesw/make_keys

I must have been looking at on of our internal FreeBSD trees last
time...

In FreeBSD lib/ncurses/ncursesw/Makefile and other places I checked
just uses ${CC} for building make_keys - I don't see how that could work
for cross-building. 

If you want cross-building to work, the simple solution is to ensure
that you use HOST_CC rather than CC when building things that need to
run on the build host.

eg. in our internal tree - which cross builds fine:

make_keys: make_keys.c names.c ncurses_def.h ${HEADERS}
${HOST_CC} -o $@ ${HOST_CFLAGS} ${NCURSES_DIR}/ncurses/tinfo/make_keys.c

make_hash: make_hash.c hashsize.h ncurses_def.h ${HEADERS}
${HOST_CC} -o $@ ${HOST_CFLAGS} -DMAIN_PROGRAM \
${NCURSES_DIR}/ncurses/tinfo/make_hash.c

and in share/mk/local.init.mk

HOST_CC?= /usr/bin/cc

should do.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301139: WITH_META_MODE=yes vs. "sh: ./make_keys: Exec format error"? [still true of -r301815]

2016-06-12 Thread Simon J. Gerraty
Simon J. Gerraty  wrote:
> If you want cross-building to work, the simple solution is to ensure
> that you use HOST_CC rather than CC when building things that need to
> run on the build host.

FWIW there are not many makefiles to fix:

bin/csh/Makefile
gnu/usr.bin/cc/cc_tools/Makefile
lib/libmagic/Makefile
lib/ncurses/ncurses/Makefile
share/syscons/scrnmaps/Makefile
usr.bin/awk/Makefile

and bin/sh/Makefile which is a bit different for some reason.
also in Makefile.inc1

CC=${HOST_CC} is in the BMAKE command line:

BMAKE=  MAKEOBJDIRPREFIX=${WORLDTMP} \
${BMAKEENV} ${MAKE} ${WORLD_FLAGS} -f Makefile.inc1 \
DESTDIR= \
BOOTSTRAPPING=${OSRELDATE} \
SSP_CFLAGS= \
HOSTPROG=yes \
CC=${HOST_CC} \
MK_HTML=no NO_LINT=yes MK_MAN=no \
-DNO_PIC MK_PROFILE=no -DNO_SHARED \
-DNO_CPU_CFLAGS MK_WARNS=no MK_CTF=no \
MK_CLANG_EXTRAS=no MK_CLANG_FULL=no \
MK_LLDB=no MK_TESTS=no \
MK_INCLUDES=yes 

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301139: WITH_META_MODE=yes vs. "sh: ./make_keys: Exec format error"? [still true of -r301815]

2016-06-12 Thread Simon J. Gerraty
Mark Millard  wrote:
> Cross builds work just fine based on the FreeBSD tree when omitting 
> WITH_META_MODE=.
> 

Hmm must do something odd then.

> As of -r301825 there is almost no use of HOST_CC at the upper levels or in 
> share/mk/*:

Yes, which means if cross-building works it must be requring a separate
walk of the tree or similar dance.
It cannot work by building tools like make_keys in the same pass as the
library, since that would require use of something other than CC.


> > # grep HOST_CC /usr/src/Makefile*
> > # grep HOST_CC /usr/src/share/mk/*
> > /usr/src/share/mk/bsd.compiler.mk:.for var in CC CXX HOST_CC HOST_CXX
> 
> where that last does:

Yes, that's for ccache, but I don't see HOST_CC used anywhere.

Sorry, guess I have no clue how cross-building currently works in
FreeBSD, but as is it won't work with DIRDEPS - which pretty much
expects to be able to build the tree in a single pass.

META_MODE should be orgthogonal, but that appears not to be so.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301139: WITH_META_MODE=yes vs. "sh: ./make_keys: Exec format error"? [still true of -r301815]

2016-06-13 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> > eg. in our internal tree - which cross builds fine:
> > 
> > make_keys: make_keys.c names.c ncurses_def.h ${HEADERS}
> > ${HOST_CC} -o $@ ${HOST_CFLAGS} 
> > ${NCURSES_DIR}/ncurses/tinfo/make_keys.c
> 
> I like this method but am going to put it off for a while.  The

The above is a reasonably non-intrusive way of dealing with targets like
that.

For tools which are more expensive to build, having a separate dir is
best, since (with DIRDEPS at least) it is then easier to build them
only once and at the right time.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301815 to -r310873 using WITH_META_MODE=yes : an empty filename failure

2016-06-13 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> >> ${_FIRM}: ${.CURDIR}/../../../../contrib/dev/drm2/radeonkmsfw/${_FIRM}.uu
> >> uudecode -p $? > ${.TARGET}

Targets like this that use $? or ${.OODATE} are a bad fit with META mode.

If the normal make rules think the target is up to date, .OODATE will be
empty, thus if meta_oodate says the target is out-of-date, the script
will run with no args - because $? expands to nothing.

So either the use of $? should be replaced with ${.ALLSRC} or something
else that will be consistent, or the target should be marked .NOMETA

FYI we can [ab]use ${.OODATE} as a token to prevent a line of a script
from being compared - meta mode knows that the value of .OODATE is
inconsistent so should not be compared... thus putting
${.OODATE:MNO_META_CMP} on a line expands to nothing - no impact on the
command but still tells meta_oodate to ignore that line for comparison
purposes. 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301815 to -r310873 using WITH_META_MODE=yes : an empty filename failure

2016-06-14 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> The problem is missing-meta requiring a .meta file here.  The $?/.OODATE
> comparison exception is only used meta_oodate() if there is already a
> .meta file, not for the new missing .meta logic.

Even if there is already a .meta file, if meta_oodate ever returns TRUE
the target will not work correctly.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301815 to -r310873 using WITH_META_MODE=yes : an empty filename failure

2016-06-14 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> Actually it does seem to be meta-missing bug since $? (.OODATE) is empty
> and yet it is requiring a .meta file.

As I said; .meta files and targets that use $? (.OODATE)
are fundamentally incompatible.

Such a target will not work correctly, if meta_oodate returns TRUE
so add .NOMETA or change the target to not use $?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 11.0 -r301815 to -r310873 using WITH_META_MODE=yes : an empty filename failure

2016-06-14 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> I think my point is getting lost.  With the new missing-meta feature, if

> Yet, via meta_oodate, if there is already a .meta file and .OODATE is
> used and has an empty source list then $.OODATE is forced to be .ALLSRC:

Ah yes forgot about that.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Possible race condition building libraries: head/amd64 r303329 -> r303379

2016-07-27 Thread Simon J. Gerraty
David Wolfskill  wrote:
> The build failed (initially -- a restart worked):

That's usually a good indicator of a race.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 1st build stops when WITH_AUTO_OBJ=yes

2016-11-09 Thread Simon J. Gerraty
Renato Botelho  wrote:

> I decided to give a try to WITH_AUTO_OBJ and noted the first time I ran 
> buildworld it failed with following message:
> 
> /u/src # ❯❯❯ make WITH_AUTO_OBJ=yes buildworld
> [Creating objdir obj...]
> make: "/usr/src/share/mk/auto.obj.mk" line 61: could not use obj: 
> .OBJDIR=/usr/src/obj
> 
> After that I noted it created a directory /usr/src/obj and if I call
> it again it runs without issues. If I remove /usr/src/obj directory
> error happens again

Interesting; what .OBJDIR do you end up with for say bin/cat ?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: 1st build stops when WITH_AUTO_OBJ=yes

2016-11-11 Thread Simon J. Gerraty
Renato Botelho  wrote:
> > Interesting; what .OBJDIR do you end up with for say bin/cat ?
> 
> 
> In this case it fails the first time pointing to expected .OBJDIR, then 
> second time I run it builds
> 
> /u/s/b/cat # ❯❯❯ make -DWITH_AUTO_OBJ
> [Creating objdir obj...]
> make: "/usr/src/share/mk/auto.obj.mk" line 61: could not use obj: 
> .OBJDIR=/usr/obj/usr/src/bin/cat

The creating line is the clue, it is just obj
rather than say /usr/obj/usr/src/bin/cat

Do you have MAKEOBJDIRPREFIX set in env?
Or are you relying on bsd.obj.mk to set
CANONICALOBJDIR:=/usr/obj${.CURDIR}

?

Since we need to do auto.obj.mk *very* early (so .PATH is correct),
it is probably relying on MAKEOBJDIRPREFIX or MAKEOBJDIR
(I always have MAKEOBJDIR set) since bsd.obj.mk will not be included
yet.

If that's all the case though I wouldn't expect it to work any better on
subsequent runs.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: MANPATH not handled correctly

2017-01-09 Thread Simon J. Gerraty
Steve Kargl  wrote:

> MANPATH is not handled correctly.  According to the documentation
> in apropos(1) and whatis(1):

Can you clarify where you are seeing this documentation?
I don't see it in 7.x 10.x or current.

Eg, in 10.2 apropos(1) says only:

ENVIRONMENT
 The following environment variables affect the execution of
 apropos:

 MANLOCALES, MANPATH, PATH
 Used to find the location of the whatis database
 files. See manpath(1) for additional information.

 PAGER  Program used to display files.  If unset, more -s is used.


> 
>   MANPATH   The standard search path used by man(1) may be changed by
> specifying a path in the MANPATH environment variable.  Invalid
> paths, or paths without manual databases, are ignored.
> Overridden by -M.  If MANPATH begins with a colon, it is
> appended to the default list; if it ends with a colon, it is
> prepended to the default list; or if it contains two adjacent
> colons, the standard search path is inserted between the
> colons.  If none of these conditions are met, it overrides the
> standard search path.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: MANPATH not handled correctly

2017-01-09 Thread Simon J. Gerraty
Steve Kargl  wrote:

> On Mon, Jan 09, 2017 at 05:19:01PM -0800, Simon J. Gerraty wrote:
> > Steve Kargl  wrote:
> > 
> > > MANPATH is not handled correctly.  According to the documentation
> > > in apropos(1) and whatis(1):
> > 
> > Can you clarify where you are seeing this documentation?
> > I don't see it in 7.x 10.x or current.
> > 
> 
> This an unaltered FreeBSD-current at r311832.
> 
> The text I quoted can be seen at 
> 
> https://svnweb.freebsd.org/base/head/contrib/mdocml/apropos.1?revision=294113&view=markup
> 

But that isn't the man page that gets installed.

usr.bin/man/apropos.1

is the one that matches apropos(1)

Note, not saying the behavior you describe shouldn't be implemented,
but it does not appear to be the claimed behavior.

> starting at line 346 and ending at 362.
> 
> The Makefile in /usr/src/usr.bin at line 93 points at the 
> directory mandoc.  The Makefile in /usr/src/usr.bin/mandoc
> points at ../../contrib/mdocml.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: MANPATH not handled correctly

2017-01-10 Thread Simon J. Gerraty
Steve Kargl  wrote:
> Well, yes, it is the manpage that gets installed.
> 
> > usr.bin/man/apropos.1
> > 
> > is the one that matches apropos(1)

I should have said "in 10.x" ;-)
In current, MK_MANDOCDB defaults yes, so you are right.

So the script needs an update.
Is there a PR filed?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Secure Boot

2017-01-14 Thread Simon J. Gerraty
Johannes Lundberg  wrote:
> https://wiki.freebsd.org/SecureBoot
> 

Interested in this too - though for proprietary systems where we have
control over BIOS.  The design should hopefully accommodate both.

In particular any plan for how the loader would verify kernel and any
pre-loaded modules, and kernel verify init.
Hopefully allowing for regular update of sining keys.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Fix /etc/rc.d/random umask handling (/entropy permissions)

2017-01-23 Thread Simon J. Gerraty
Jilles Tjoelker  wrote:
> Index: etc/rc.d/random
> ===
> --- etc/rc.d/random   (revision 311446)
> +++ etc/rc.d/random   (working copy)
> @@ -20,12 +20,14 @@
>  
>  save_dev_random()
>  {
> + oumask=`umask`

why not simply use a sub-shell to tighten umask

(umask 077; what-ever)

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: bmake core dump

2017-02-27 Thread Simon J. Gerraty
Hi Iblis

> I encounted core dump with `make -f - ...`

can you share the content of stdin? or a relevant snippet?

Also what version of bmake? (make -r -f /dev/null -V MAKE_VERSION)

Your patch risks overflow, so would like to reproduce first...

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: bmake core dump

2017-02-27 Thread Simon J. Gerraty
Iblis Lin  wrote:
> Accutally, I made it core dump via a julia script.
> Please checkout this code

I'm not familiar with juila, in most scripting languages
cmd = `/usr/bin/make -f - -V MAKE_ENV`
would run that command and assign the output to cmd.
The make instance would be reading from stdin which is rather unknown at
that point.

Could you perhaps run that with ktrace?

Eg.

ktrace -i -t cnis -f /var/tmp/make.kt whatever command you ran
kdump -m 128 -f /var/tmp/make.kt > /var/tmp/make.kd

that would show what make is actually reading

> └─[iblis@abeing ]% cat test.jl
> makefile="""
> CATEGORIES= devel
> 
> USES=
> .include """
> 
> cmd = `/usr/bin/make -f - -V MAKE_ENV`
> in_ = Pipe()
> out, proc = open(cmd, "r", in_)
> 
> print(in_, makefile)
> close(in_)
> 
> @assert success(proc)
> close(out)
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: bmake core dump

2017-02-27 Thread Simon J. Gerraty
Iblis Lin  wrote:
> > Could you perhaps run that with ktrace?
> > 
> > Eg.
> > 
> > ktrace -i -t cnis -f /var/tmp/make.kt whatever command you ran
> > kdump -m 128 -f /var/tmp/make.kt > /var/tmp/make.kd
> > 
> > that would show what make is actually reading
> > 
> 
> I uploaded the ktrace to the gist:
> https://gist.github.com/iblis17/0beff3e1b4fcb7eac9df5889cd3ddde7#file-make-kd-L2733-L2853

Thanks, so the last line of your makefile as no newline.
Not that that upsets any bmake binary I have handy.

Thanks for the input...
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Apparent build race(s), r315238 -> r315298

2017-03-15 Thread Simon J. Gerraty
Ngie Cooper (yaneurabeya)  wrote:
> Alternatively, could you please revert just r313650 in another 
> branch/workspace (I wonder if changing from .OBJDIR to OBJTOP had some 
> unintended consequences that unrooted issues with how bmake evaluates paths)?

The only change to dir handling in recent bmake is handling of -C where
arg involves symlinks - provided that refers to same dir as getcwd, the
logical value is used.



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: buildkernel broken for META_MODE

2017-04-18 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> Aha /usr/obj/usr/obj.
> 
> That was in Renato's report as well.
> 
> The bug is WITH_AUTO_OBJ. I just confirmed that. A bunch of errors occur
> when doing the first build and the opt_*.h files are not generated in
> the "proper" place by config(8).
> 
> WITH_AUTO_OBJ is not really workable yet in buildworld/buildkernel. I
> have patches that I will commit soon to add support for them (meaning to
> work as it all does not but without a 'make obj' tree walk).

What is the issue above? diff?

FWIW the latest version of auto.obj.mk deals with __objdir=obj which is
a corner case compared to any use of MAKEOJDIRPREFIX etc.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: buildkernel broken for META_MODE

2017-04-18 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> > What is the issue above? diff?
> 
> I don't know what the issue is with buildkernel specifically, I never
> looked into it. Buildworld had other issues like rescue/rescue not being
> AUTO_OBJ safe. That's fixed. I forget the details of buildworld as well.
> One of the simpler changes is to remove the 'make obj' treewalk and
> prefer .OBJDIR/OBJTOP over other ${CANONICALOBJDIR} ${OBJTREE} type
> hacks. My patches are quite mangled with the idea of moving to
> /usr/obj/CURDIR/TARGET.TARGET_ARCH/RELDIR and I need to split it all up
> still.

Ok

> 
> > 
> > FWIW the latest version of auto.obj.mk deals with __objdir=obj which is
> > a corner case compared to any use of MAKEOJDIRPREFIX etc.
> > 
> 
> It does not fix the issues.

Not this lot no.  Its for folk wanting to use the traditional
${.CURDIR}/obj/ which used to throw an error.
I haven't used ${.CURDIR}/obj/ in almost 20 years ;-)


> >  SIZE="size"  INSTALL="sh /root/git/freebsd/tools/install.sh"  
> > PATH=/usr/obj/root/git/freebsd/tmp/legacy/usr/sbin:/usr/obj/root/git/freebsd/tmp/legacy/usr/bin:/usr/obj/root/git/freebsd/tmp/legacy/bin:/usr/obj/root/git/freebsd/tmp/usr/sbin:/usr/obj/root/git/freebsd/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin
> >  make  -j 1
> > 5 -D NO_FILEMON -d M -J 15,16 -m /root/git/freebsd/share/mk  KERNEL=kernel 
> > obj
> > [Creating objdir /usr/obj/usr/obj/root/git/freebsd/sys/GENERIC...]
> Wrong^
> 
> Note we have 'cd /usr/obj/' and 'MAKEOBJDIRPREFIX=/usr/obj' in
> there, so we get a nested /usr/obj/.CURDIR problem of /usr/obj/usr/obj.

Yes, ugly.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: buildkernel broken for META_MODE

2017-04-18 Thread Simon J. Gerraty
> > [Creating objdir /usr/obj/usr/obj/root/git/freebsd/sys/GENERIC...]
> Wrong^
> 
> Note we have 'cd /usr/obj/' and 'MAKEOBJDIRPREFIX=/usr/obj' in
> there, so we get a nested /usr/obj/.CURDIR problem of /usr/obj/usr/obj.

The following would probably help that case:

Index: auto.obj.mk
===
RCS file: /volume/cvs/sjg/mk/auto.obj.mk,v
retrieving revision 1.13
diff -u -p -r1.13 auto.obj.mk
--- auto.obj.mk 24 Mar 2017 20:53:22 -  1.13
+++ auto.obj.mk 18 Apr 2017 22:25:08 -
@@ -41,6 +41,10 @@ MKOBJDIRS= auto
 # Use __objdir here so it is easier to tweak without impacting
 # the logic.
 .if !empty(MAKEOBJDIRPREFIX)
+.if ${.CURDIR:M${MAKEOBJDIRPREFIX}/*} != ""
+# we are already in obj tree!
+__objdir?= ${.CURDIR}
+.endif
 __objdir?= ${MAKEOBJDIRPREFIX}${.CURDIR}
 .endif
 __objdir?= ${MAKEOBJDIR:Uobj}
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: filemon: weird full-time build although filemon enabled

2017-05-07 Thread Simon J. Gerraty
Hi,

> I build CURRENT on two technically similar systems on a almost daily basis. 
> Therefore, it
> was a great relief having WITH_META_MODE=yes set in /etc/src-env.conf for 
> incremental
> builds. To make my understanding of this clear (just in case I'm wrong): 
> setting
> WITH_META_MODE builds only portions that does not need to be build in the 
> make context.

I'm not sure I parsed that correctly, so let me just say;
With MK_META_MODE=yes, bmake should be put into meta mode.

This simply means that most targets get a .meta file, that records the
expanded command line, any command output and a syscall trace.
These can be very useful for debugging.

More important; on an update build bmake can use them to make targets
out-of-date that the normal rules miss.
Any change to the expanded command line, makes the target out-of-date.
Any change to any file read or executed makes the target out-of-date.
 
> Well, the reason writing this email is: on one system, I run almost every 
> reboot into a
> "full build" and this puzzles me a bit. The long-lasting and time exhausting 
> builds are

You can add -dM to your build flags to have meta_oodate() - the function
that does the above, explain why it decided the target is out-of-date.
The results can be enlightening.

Note, if a target rebuilds without -dM saying why - then the normal make
rules decided it was out-of-date.

> within the LLVM/CLANG tree. They consume a lot of time. The box in
> question does have a

Yes, building clang takes a while...

> Well, having said this, I need to mention that both systems use almost
> identical /etc/src.conf setting - except the order of appearance of
> the WITH_ tags.

That should not matter, bsd.mkopt.mk processes them in a specified
order.

> The problem: to make my point clear: the "weak" box starts compiling almost 
> everytime now
> the LLVM/CLANG tree while the XEON box does not. This is spooky.

Add -dM, and see if that tells you why.
Also confirm that filemon is getting loaded after each reboot.
If that comes and goes, that could cause the behavior you see.

> I think there is something missing an I'd like to ask what is the suggested 
> way to
> initially restart a full build to ensure that WITH_META_MODE gets initialised 
> correctly.

I think Bryan has set that up so that it should take care of itself
pretty well - even toggling bewteen non-meta and meta builds should
DTRT - which is why it matters if filemon was used that it is there for
next build.

> Well, I'm not a developer, so please be patient with my naive report.

No problem.  Adding -dM to your build (will add a fair bit of output),
will hopefully tell you what's up.

>From what you describe, I'd not be surprised if you see some
"a build command has changed" and it only takes a few to trigger a
cascade of targets out-of-date.

--sjg

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: filemon: weird full-time build although filemon enabled

2017-05-08 Thread Simon J. Gerraty
O. Hartmann  wrote:
> It is weird!
> 
> Today, after yesterday's built, I face the same 90 minutes build horror 
> again, this time
> I switched on "-dM" with the make command.
> 
> This happens:
> 
> [...]
> /usr/obj/usr/src/lib/clang/libllvm/_usr_obj_usr_src_lib_clang_libllvm_CodeGen_SelectionDAG_LegalizeDAG.o.meta:
> 15: file '/usr/local/etc/libmap.d/nvidia.conf' is newer than the target...
> 

That does seem odd why anything involved in building clang should care
about that file...

You can use the pid field in the syscall trace to show what process was
looking at that file.

> This box has the following lines in /etc/src.conf to rebuild the nvidia 
> kernel module
> every time:
> 
> PORTS_MODULES+= x11/nvidia-driver
> 
> I do not know what is going on here ... :-(

well that might explain why nvidia.conf is updated, but not why clang
build cares.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: filemon: weird full-time build although filemon enabled

2017-05-08 Thread Simon J. Gerraty
Konstantin Belousov  wrote:
> If I understand the motto of meta-mode, any file change is detected for any
> file accessed during the build.  All dynamically-linked binary includes
> the rtld into the process image, and rtld reads all config files in the
> libmap.d subdirectories.  The end result is that everything must be rebuild
> if any config file changed.

Ah.

> Then, after the world build, according to OP, the nvidia driver port is
> reinstalled, which installs the nvidia.conf anew, which triggers the
> behaviour on the next build.

Yes, that would account for it.

There are always exceptions - things we expect to be volatile and don't
care - or for any other reason just want to ignore.

bmake has a set of knobs for telling it to ignore things.
OP try

.MAKE.META.IGNORE_PATHS += /usr/local/etc/libmap.d

--sjg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: filemon: weird full-time build although filemon enabled

2017-05-08 Thread Simon J. Gerraty
O. Hartmann  wrote:
> > .MAKE.META.IGNORE_PATHS += /usr/local/etc/libmap.d
> > 
> > --sjg
> 
> I suppose I have to set this flag in 
> 
> /etc/src-env.conf

That should work.
Let us know how it goes
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: make warning: ?: No such file or directory.

2017-05-10 Thread Simon J. Gerraty
Ngie Cooper (yaneurabeya)  wrote:
> I see similar oddness when running some commands. It seems to be happening as 
> of the last month or two.
> 
> $ make buildenv TARGET_ARCH=armv6
> make warning: I: No such file or directory.
> make warning: I: No such file or directory.
> Entering world for armv6:arm
> $

That looks to be from Main_SetObjdir()
what is make -V MAKE_VERSION ?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: filemon: weird full-time build although filemon enabled

2017-05-10 Thread Simon J. Gerraty
David Wolfskill  wrote:
> I placed it in /etc/src.conf; thus:
> 
> g1-252(11.0-S)[1] cat /etc/src.conf 
> KERNCONF=CANARY
> PORTS_MODULES=x11/nvidia-driver-340
> .MAKE.META.IGNORE_PATHS += /usr/local/etc/libmap.d
> WITHOUT_DEBUG_FILES=1
> IWN_DEBUG=1
> IEEE80211_DEBUG=1
> WITH_ELFCOPY_AS_OBJCOPY=1
> g1-252(11.0-S)[2] 
> 
> My morning updates (stable/11 & head, as well as installed ports (while
> running stavble/11), which had been getting to around 2 hours, were
> finished in 30 minutes this morning.  Thank you! :-)

Good to know, we should probably add this to share/mk/local.meta.sys.mk
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: make warning: ?: No such file or directory.

2017-05-10 Thread Simon J. Gerraty
Bryan Drewery  wrote:
> Oh now I get it too after updating system from head from r317177 to
> r318116. So it seems to be a bug in bmake-20170420.

What's in your env?
Eg.

env | grep MAKE
ls


> > ~/git/freebsd # make check-old
> > make warning: E No such file or directory.
> > make warning: E No such file or directory.
>  Checking for old files
> > make warning: E No such file or directory.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: buildworld not working with MAKEOBJDIRPREFIX

2017-05-16 Thread Simon J. Gerraty
Roger Pau Monné  wrote:
> $ cd /home/royger/buildjob/freebsd
> $ make -j30 buildworld MAKEOBJDIRPREFIX=/home/royger/buildjob/obj/

That will not work as desired.

When you set VAR=val as an argument to make,
it overrides anything the makefiles want to do
and there are a number of points where the top level makefiles want to
play with MAKEOBJDIRPREFIX

By contrast;

MAKEOBJDIRPREFIX=/home/royger/buildjob/obj/ make -j30 buildworld 

or for csh users;

env MAKEOBJDIRPREFIX=/home/royger/buildjob/obj/ make -j30 buildworld

provides the same value via the environment, this leaves the makefiles
able to do as they will.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: build src with colored output?

2017-05-16 Thread Simon J. Gerraty
David Chisnall  wrote:

> Ideally, we’d solve this by fixing bmake to behave more like a modern
> build tool and:

It's called meta mode ;-)
and makes the OP's real issue much easier - when build fails, the
failing .meta file is saved in src/../error/ providing no doubt and no
need to search.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Bug in make setting wrong MAKESYSPATH

2017-05-22 Thread Simon J. Gerraty
Thomas Mueller  wrote:

> I tried building ports, starting with ports-mgmt/synth, on HEAD
> (12-current) and ran into difficulties with syntax error in
> bsd.compiler.mk .
> 
> With PORTSDIR on another partition, mounted as /BETA1, I got these
> errors, but not when I null-mounted /BETA1/usr/ports as /usr/ports.
> 
> I shouldn't have to resort to this kludge, didn't have to in the
> recent past.

The default value for MAKESYSPATH is ".../share/mk:/usr/share/mk" which
is geared to building src/, there was some to and fro over that value
when we first started using bmake, but that's what we settled on.

Presumably /BETA1/usr/share/mk exists?
If so, it will be found via .../share/mk if you start somewhere under
/BETA1/usr/ports

> This bug shows in both 11.0-STABLE and 12.0-CURRENT.

Not a bug - working as intended, which doesn't mean it is doing what you
want.

You can of course set MAKESYSPATH as you wish, but of course
/usr/share/mk would not be correct for building src/
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Bug in make setting wrong MAKESYSPATH

2017-05-23 Thread Simon J. Gerraty
Thomas Mueller  wrote:
> It seems to me that MAKESYSPATH should match the host building system
> FreeBSD version.

Which would only be correct if building the same version of FreeBSD as
is running on the host.
Many folk work on multiple branches on the same machine.

Thus for anyone working on src/ there is no guarantee that /usr/share/mk
is even remotely correct.

So you either buy into the idea of using a wrapper script and canned
.env file for each tree (that's what I do;
http://www.crufty.net/sjg/docs/sb-tools.htm) to ensure correct 
settings per tree (whether FreeBSD, NetBSD, ...), or you compromise and
have 'make' itself try to find the "correct" share/mk for whereever it is.
This is why the default MAKESYSPATH starts with .../share/mk

> Now I believe the correct syntax for setting MAKESYSPATH in .cshrc and 
> .profile would be
> export MAKESYSPATH=/usr/share/mk  # for .profile, or
> setenv MAKESYSPATH /usr/share/mk  # for .cshrc

Yes, but requires you to keep /usr/share/mk in sync with whatever you a
building, tricky if you build head, stable/10 and stable/11 on the
same box.

--sjg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Bug in make setting wrong MAKESYSPATH

2017-05-24 Thread Simon J. Gerraty
Thomas Mueller  wrote:
> For building the system, MAKESYSPATH should be $SRCDIR/share/mk , to be in 
> sync.
> 
> I tried "make -V MAKESYSPATH" from several SRCDIRs, and that's what happened.

Yes.  If you look at share/mk/src.sys.env.mk
it detects that it was found via a .../ path, and replaces it in
MAKESYSPATH with the actual location - otherwise some makefiles break.

> So maybe I have to set MAKESYSPATH every time I am in ports, don't
> know if this would work from /etc/make.conf (need to experiment).

Probably not, because sys.mk will have already been found before that
ie. the damage may already have been done
 
> I wonder what would happen if there is no FreeBSD installation at all
> on partition where ports tree is located.

An error like:

make: no system rules (sys.mk).

> 
> > Yes, but requires you to keep /usr/share/mk in sync with whatever you a
> > building, tricky if you build head, stable/10 and stable/11 on the
> > same box.
> 
> I don't think you have to do that, since I find that MAKESYSPATH
> becomes $SRCDIR/share/mk .

Per above; that's due to the default .../share/mk
I was speaking about alternatives to that magic value.

Thanks
--sjg
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: ino64? r318606 -> r318739 OK; r318739 -> r318781 fails SIGSEGV

2017-05-24 Thread Simon J. Gerraty
David Wolfskill  wrote:
> As far as I can tell, the "ld" command was:

Since you have the .meta file, it should tell you exactly what the
command was and what path was actually exec'd

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: ino64? r318606 -> r318739 OK; r318739 -> r318781 fails SIGSEGV

2017-05-24 Thread Simon J. Gerraty
Ngie Cooper  wrote:

> There was another report on the list about a stale
> MAKEOBJDIRPREFIX causing someone grief.

Pointer?
What does stale MAKEOBJDIRPREFIX mean?

>I think it's safe to say that
> meta mode and -DNO_CLEAN might not work across this transition--in
> particular meta mode tends to err on the side of not to rebuilding
> things.

I would have expected the opposite.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: ino64? r318606 -> r318739 OK; r318739 -> r318781 fails SIGSEGV

2017-05-24 Thread Simon J. Gerraty
Peter Jeremy  wrote:
> as follows.  My suspicion is that meta mode isn't seeing enough of the
> differences between the bootstrap and main build steps and so causing make
> to incorrectly skip steps.

I see a number of places in src/Makefile* where BUILD_TOOLS_META=.NOMETA
is added to env of things like CROSSENV, CD2MAKE, LIBCOMPATWMAKEENV

Use of .NOMETA could be leading to problems - but I'm not familiar with
where BUILD_TOOLS_META is used.


> 
> --
> >>> stage 2.3: build tools
> --
> cd /usr/src; MAKEOBJDIRPREFIX=/usr/obj  INSTALL="sh 
> /usr/src/tools/install.sh"  TOOLS_PREFIX=/usr/obj/usr/src/tmp  
> PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/bin:/sbin:/bin:/usr/sbin:/usr/bin
>   WORLDTMP=/usr/obj/usr/src/tmp  MAKEFLAGS="-m /usr/src/tools/build/mk  -m 
> /usr/src/share/mk" /usr/obj/usr/src/make.amd64/bmake  -f Makefile.inc1  
> TARGET=amd64 TARGET_ARCH=amd64  DESTDIR=  BOOTSTRAPPING=1200031  SSP_CFLAGS=  
> -DNO_LINT  -DNO_CPU_CFLAGS MK_WARNS=no MK_CTF=no  MK_CLANG_EXTRAS=no 
> MK_CLANG_FULL=no  MK_LLDB=no MK_TESTS=no build-tools
> ...
> ===> usr.bin/mkesdb_static (obj,build-tools)
> Building /usr/obj/usr/src/usr.bin/mkesdb_static/citrus_bcs.o
> Building /usr/obj/usr/src/usr.bin/mkesdb_static/citrus_db_factory.o
> Building /usr/obj/usr/src/usr.bin/mkesdb_static/citrus_db_hash.o
> Building /usr/obj/usr/src/usr.bin/mkesdb_static/citrus_lookup_factory.o
> Building /usr/obj/usr/src/usr.bin/mkesdb_static/lex.c
> Building /usr/obj/usr/src/usr.bin/mkesdb_static/lex.o
> /usr/src/usr.bin/mkesdb/lex.l:44:10: fatal error: 'yacc.h' file not found
> #include "yacc.h"
>  ^~~~
>1 error generated.
>*** Error code 1
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Bug in make setting wrong MAKESYSPATH

2017-05-24 Thread Simon J. Gerraty
Thomas Mueller  wrote:

> I go into /BETA1/usr/ports/ports-mgmt/synth , run
> env MAKESYSPATH make all-depends-list

I assume you mean MAKESYSPATH=something? otherwise env itself should
vomit

> and then it seems to work correctly with no syntax error in
> /BETA1/usr/share/mk/bsd.compiler.mk
> 
> Maybe I need to file a bug.

For what?
 
> What happens if src, ports and doc trees are installed on an NFS
> share, where there would be no FreeBSD installation?

Not sure what you mean. make doesn't care what the filesystem is
if there is a share/mk found in . or somewhere above it, the default
MAKESYSPATH will find it.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: ino64? r318606 -> r318739 OK; r318739 -> r318781 fails SIGSEGV

2017-05-24 Thread Simon J. Gerraty
Peter Jeremy  wrote:
> In my case, I have "WITH_META_MODE=yes" in /etc/src-env.conf and was
> using "make buildworld" - which failed.  The upgrade worked cleanly
> when I manually deleted all the .meta files.  If I get a round tuit,

sys.mk is setup such that missing .meta file makes the target
out-of-date.

FWIW I just did a buildworld -DWITH_META_MODE followed by the same with
-DNO_CLEAN - but no change to the tree.
That at least worked fine (and quickly) tree was last updated to r318755

Wonder if it is safe to update...
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: ino64? r318606 -> r318739 OK; r318739 -> r318781 fails SIGSEGV

2017-05-25 Thread Simon J. Gerraty
Simon J. Gerraty  wrote:

> Peter Jeremy  wrote:
> > In my case, I have "WITH_META_MODE=yes" in /etc/src-env.conf and was
> > using "make buildworld" - which failed.  The upgrade worked cleanly
> > when I manually deleted all the .meta files.  If I get a round tuit,
> 
> sys.mk is setup such that missing .meta file makes the target
> out-of-date.
> 
> FWIW I just did a buildworld -DWITH_META_MODE followed by the same with
> -DNO_CLEAN - but no change to the tree.
> That at least worked fine (and quickly) tree was last updated to r318755
> 
> Wonder if it is safe to update...

FWIW I updated to r318860
and again

make -j12  -DWITHOUT_TESTS buildworld -DWITH_META_MODE -DNO_CLEAN

completed happily.

I guess need to update to/from the specific grns people had issue with
to reproduce.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Bug in make setting wrong MAKESYSPATH

2017-05-25 Thread Simon J. Gerraty
Thomas Mueller  wrote:
> When I did those last examples, that last line was
> env MAKESYSPATH=/usr/share/mk make all-depends-list

ok, that makes sense.

> > > and then it seems to work correctly with no syntax error in
> > > /BETA1/usr/share/mk/bsd.compiler.mk
> >
> > > Maybe I need to file a bug.
> 
> > For what?
> 
> Bug occurs when building or configuring ports, syntax error in 
> /BETA1/usr/share/mk/bsd.compiler.mk  line 52

This is of course specific to your particular arrangement
if you'd mounted /BETA1/usr/ports on /usr/ports, it would function as
you wish, or if /BETA1/usr/share/mk happend to match /usr/share/mk
it would work fine.

So, anoying in this case, but not a bug.

> I don't know about other situations such as building doc.
> 
> I could avoid this error either by setting (setenv or export, depending on 
> shell) MAKESYSPATH or
> by null-mounting /BETA1/usr/ports at /usr/ports .

Yes.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Build failure in sys/modules/drm2/radeonkmsfw/ARUBA_pfp

2017-05-25 Thread Simon J. Gerraty
David Wolfskill  wrote:

> On Thu, May 25, 2017 at 05:07:19AM -0700, David Wolfskill wrote:
> > This is on my "build machine"; running GENERIC/amd64 built yesterday:
> > 
> > FreeBSD freebeast.catwhisker.org 12.0-CURRENT FreeBSD 12.0-CURRENT #353  
> > r318739M/318739:1200031: Wed May 24 10:00:20 PDT 2017 
> > r...@freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC  amd64
> > 
> 
> Well, the laptop, running:

Compare the meta files for ARUBA_pfp.bin
in the failure case it looks very much like a input file was not
supplied (eg empty variable)

A possibility is .OODATE being used; which will be empty if none of the
srcs are out-of-date.


A quick scan didn't identify where the target for that comes from
so guessing...
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


  1   2   >