Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-07 Thread Roland Illig
I fixed the SIGPIPE problem in usr.bin/make/job.c 1.488 from a few minutes ago. The test cases you provided run successfully for me now. Thank you for preparing the tests, I'll add a fast variant of them to the test suite later. Roland

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-06 Thread Greg A. Woods
At Wed, 05 Mar 2025 19:17:06 -0800, "Greg A. Woods" wrote: Subject: Re: parallel build failure with .c.o rule interrupted mid-step! > > > > > H maybe there's a clue to this problem in the fact the child > > shell's output is to a pipe which must be

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-06 Thread Mouse
> There's a race between the shell writing the final command-line to > the pipe and the SIGPIPE being delivered, [...] Is there? My understanding is that the SIGPIPE is generated within the write() call, and it is noticed and delivered in the syscall exit codepath. If so, the signal-generating w

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-06 Thread Greg A. Woods
At Thu, 06 Mar 2025 14:05:33 +0700, Robert Elz wrote: Subject: Re: parallel build failure with .c.o rule interrupted mid-step! > > Hmm - that's a dangerous way to operate, it reverts sh back to having > Thompson shell style semantics - the shell's command input is being > s

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-05 Thread Robert Elz
Date:Wed, 05 Mar 2025 17:40:11 -0800 From:"Greg A. Woods" Message-ID: | Yes, I think of -v as showing what the shell has read, which is | basically true in how it works if I understand correctly, You do. Personally I find it to be mostly a waste of space, but

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-05 Thread Roland Illig
Am 06.03.2025 um 04:17 schrieb Greg A. Woods: > At Wed, 05 Mar 2025 17:40:11 -0800, "Greg A. Woods" wrote: > Subject: Re: parallel build failure with .c.o rule interrupted mid-step! >> >> H maybe there's a clue to this problem in the fact the child >

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-05 Thread Greg A. Woods
At Wed, 05 Mar 2025 17:40:11 -0800, "Greg A. Woods" wrote: Subject: Re: parallel build failure with .c.o rule interrupted mid-step! > > H maybe there's a clue to this problem in the fact the child > shell's output is to a pipe which must be read by make. May

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-05 Thread Greg A. Woods
At Thu, 06 Mar 2025 06:02:47 +0700, Robert Elz wrote: Subject: Re: parallel build failure with .c.o rule interrupted mid-step! > > > It certainly isn't undocumented, from man sh ... Ah, yes, sorry -- I meant to remove that part of the comment about -q -- I'm not quite sur

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-05 Thread Robert Elz
Date:Wed, 05 Mar 2025 13:09:38 -0800 From:"Greg A. Woods" Message-ID: | # Note by default on NetBSD the default shell used by make is /bin/sh and by | # default it is passed the options "-q" (which show up in "$-" as "eLqs") as the | # .echoFlag is set to thi

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-05 Thread Roland Illig
Am 05.03.2025 um 22:09 schrieb Greg A. Woods: > Ah ha! > > I can reproduce the problem frequently now with the makefile below, both > with the patched make on NetBSD and as nbmake on macOS. > > I cannot reproduce any problem, not even left-over intermediate files, > with bmake-20200524nb1 from pkgs

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-05 Thread Greg A. Woods
Ah ha! I can reproduce the problem frequently now with the makefile below, both with the patched make on NetBSD and as nbmake on macOS. I cannot reproduce any problem, not even left-over intermediate files, with bmake-20200524nb1 from pkgsrc on macOS. However the problem does exist with bmake-20

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-03 Thread Greg A. Woods
So, some more rather odd results that I can't explain: - on NetBSD, both older versions, and with yesterday's -current (which includes Roland's make patch), I get left-over intermediate files, but never any empty object files. - with Roland's patch on macOS (i.e. in nbmake) I

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-03 Thread Greg A. Woods
I couldn't see the forest for the trees it seems. Here's a working version: # # tfail.mk: # # Demo two-commands-in-one-script failure # # Run with: # # rm -rf tfail; mkdir -p tfail; touch tfail/tfail.trace; make MAKEOBJDIR=tfail -T tfail.trace -f tfail.mk -j 20; rc=$?; ls -l tfail/*.

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-02 Thread Greg A. Woods
At Mon, 24 Feb 2025 23:09:00 +0100, Roland Illig wrote: Subject: Re: parallel build failure with .c.o rule interrupted mid-step! > > Am 24.02.2025 um 21:04 schrieb Greg A. Woods: > > In jobs mode make is supposed to feed the whole rule's script to one > > invocation of a sh

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-03-01 Thread Greg A. Woods
At Tue, 25 Feb 2025 12:58:30 -0800, "Greg A. Woods" wrote: Subject: Re: parallel build failure with .c.o rule interrupted mid-step! > > At Tue, 25 Feb 2025 00:11:06 +0100, Roland Illig wrote: > Subject: Re: parallel build failure with .c.o rule interrupted mid-step! > >

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-02-25 Thread Greg A. Woods
At Tue, 25 Feb 2025 00:11:06 +0100, Roland Illig wrote: Subject: Re: parallel build failure with .c.o rule interrupted mid-step! > > Starting with usr.bin/make/job.c 1.486 from a few minutes ago, it is now > much harder for a child process to generate a partial target file. That'

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-02-25 Thread RVP
On Mon, 24 Feb 2025, Greg A. Woods wrote: Somehow when a failure occurred during a parallel build the rule for .c.o was interrupted mid-step and left an empty .o: I missed that crucial "interrupted" when I read your email, and went off-course in my answer (rillig

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-02-24 Thread Roland Illig
Am 24.02.2025 um 21:04 schrieb Greg A. Woods: > Somehow when a failure occurred during a parallel build the rule for > .c.o was interrupted mid-step and left an empty .o: > > $ ll hist* > 16 -rw-r--r-- 1 woods staff - 7671 Feb 23 18:59 hist.d > 0 -rw-r--r-- 1 woods staff - 0 Feb 23

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-02-24 Thread Roland Illig
Am 24.02.2025 um 21:04 schrieb Greg A. Woods: > In jobs mode make is supposed to feed the whole rule's script to one > invocation of a shell, and in theory that invocation should run to > completion even if some other parallel branch of make encounters an > error. That's how it _should_ work. I m

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-02-24 Thread RVP
On Mon, 24 Feb 2025, Greg A. Woods wrote: The rule where this happened is of course this one: .c.o: ${COMPILE.c} ${.IMPSRC} ${OBJECT_TARGET} ${CTFCONVERT_RUN} Which basically expands to the following script cc host.c -o host.o.o ctfconve

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-02-24 Thread Greg A. Woods
At Mon, 24 Feb 2025 12:04:48 -0800, "Greg A. Woods" wrote: Subject: parallel build failure with .c.o rule interrupted mid-step! > > In jobs mode make is supposed to feed the whole rule's script to one > invocation of a shell, and in theory that invocation should run to > completion even if some ot

Re: parallel build failure with .c.o rule interrupted mid-step!

2025-02-24 Thread Greg A. Woods
At Mon, 24 Feb 2025 12:04:48 -0800, "Greg A. Woods" wrote: Subject: parallel build failure with .c.o rule interrupted mid-step! > > Somehow when a failure occurred during a parallel build the rule for > .c.o was interrupted mid-step and left an empty .o: > > $ ll hist* > 16 -rw-r--r-- 1 woods s