Regression on Cygwin: Problems with parallel make in 4.4

2023-02-14 Thread Ken Brown
Several packages that used to build fine on Cygwin with parallel make now fail to build. I either get strange errors or a hang. The problems started with version 4.4 and seem to stem from the fact that make now uses a FIFO by default instead of a pipe. If I use the flag '--jobserver-style=pi

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-15 Thread Ken Brown
On 2/15/2023 1:34 PM, Bruno Haible wrote: Paul Smith wrote: And possibly also on GNU/Hurd. Cf. https://lists.gnu.org/archive/html/bug-make/2023-01/msg00107.html I may be misremembering but I thought that you had tried forcing the pipe jobserver option on GNU/Hurd and it didn't help. This mus

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-19 Thread Ken Brown
On 2/19/2023 8:49 AM, Paul Smith wrote: On Wed, 2023-02-15 at 13:57 -0500, Ken Brown wrote: One thing to keep in mind here is that your tests on Cygwin were done on Cygwin 2.9.0, in which FIFOs were very poorly supported.  For example, a FIFO couldn't have multiple readers or writers.  I

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-19 Thread Ken Brown
On 2/19/2023 9:29 AM, Paul Smith wrote: On Sun, 2023-02-19 at 09:17 -0500, Ken Brown wrote: So I'm not sure where to go from here on Cygwin.  Should I force Cygwin builds to use the "pipe" jobserver, as I've done for GNU/Hurd? My preference would be for you to provid

Re: GNU Make 4.4.0.91 on Cygwin

2023-02-19 Thread Ken Brown
On 2/19/2023 2:59 PM, Paul Smith wrote: On Sun, 2023-02-19 at 20:32 +0100, Bruno Haible wrote: All "Device or resource busy" failures are gone. Only the 1 failure in category 'misc/general4' is still present. Hm. This is a test of this: https://savannah.gnu.org/bugs/index.php?57674 which exp

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-20 Thread Ken Brown
On 2/19/2023 9:29 AM, Paul Smith wrote: I will change the default setting of the jobserver to use "pipe" on Cygwin, at least for now. Parallel make is still not working reliably. I've just discovered that my TeX Live build logs have several occurrences of the following warning: jobserver

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Ken Brown
On 2/21/2023 12:27 AM, Paul Smith wrote: On Mon, 2023-02-20 at 20:21 -0500, Ken Brown wrote: Parallel make is still not working reliably.  I've just discovered that my TeX Live build logs have several occurrences of the following warning:    jobserver unavailable: using -j1.  Add &

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Ken Brown
On 2/21/2023 9:39 AM, Paul Smith wrote: On Tue, 2023-02-21 at 08:35 -0500, Ken Brown wrote: make  check-TESTS make[4]: Entering directory '/home/kbrown/src/texlive/test.x86_64/Work/texk/kpathsea' make[5]: Entering directory '/home/kbrown/src/texlive/test.x86_64/Work/texk/kpaths

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Ken Brown
On 2/21/2023 11:34 AM, Paul Smith wrote: On Tue, 2023-02-21 at 10:36 -0500, Paul Smith wrote: But, I think I might have found the bug.  If I'm right, it's a doozy! But, as you mentioned it's not widespread because it only affects pipe jobservers.  I'm glad 4.4.1 is almost ready to go.  I'll try

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-21 Thread Ken Brown
On 2/21/2023 1:26 PM, Paul Smith wrote: On Tue, 2023-02-21 at 13:11 -0500, Ken Brown wrote: I think you're on the right track.  I got through 'make -j13 check' without the jobserver warning. w00t! Thanks for the help. A full fix should be in 4.4.1 which I hope to relea

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-23 Thread Ken Brown
On 2/21/2023 5:59 PM, Ken Brown wrote: On 2/21/2023 1:26 PM, Paul Smith wrote: On Tue, 2023-02-21 at 13:11 -0500, Ken Brown wrote: I think you're on the right track.  I got through 'make -j13 check' without the jobserver warning. w00t!  Thanks for the help.  A full fix sho

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-24 Thread Ken Brown
On 2/24/2023 9:54 AM, Paul Smith wrote: On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote: I'm attaching that script so you can see exactly how "make" is invoked in a subshell.  I'm also attaching my build log up to the point of the warning and the Makefile in the ft-buil

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-02-24 Thread Ken Brown
On 2/24/2023 11:29 AM, Paul Smith wrote: On Thu, 2023-02-23 at 16:17 -0500, Ken Brown wrote: Sorry, I spoke too soon.  I just went back to my original use case, in which I used the TeX Live "Build" script instead of directly invoking make, and I again got the jobserver warning. Than

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-03-02 Thread Ken Brown
I'm returning to this thread because a surprising thing happened. I decided to try to debug the fifo problem I reported at the beginning of the thread (a hang building TeX Live on Cygwin when the jobserver used a fifo). So I installed make 4.4.1 built with fifos enabled (by setting CPPFLAGS=-

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-03-11 Thread Ken Brown
On 3/2/2023 5:39 PM, Ken Brown wrote: I'm returning to this thread because a surprising thing happened.  I decided to try to debug the fifo problem I reported at the beginning of the thread (a hang building TeX Live on Cygwin when the jobserver used a fifo).  So I installed make 4.4.1

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-03-12 Thread Ken Brown
On 3/11/2023 12:51 PM, Ken Brown wrote: Update: The hang occurred again.  It appears to be caused by an infinite loop starting with a call to pselect[*]. [...] If I'm right, the solution would seem to be to disable the use of pselect on Cygwin when the jobserver is using a fifo.  I&#

Re: Regression on Cygwin: Problems with parallel make in 4.4

2023-03-13 Thread Ken Brown
On 3/13/2023 9:15 AM, Paul Smith wrote: On Sun, 2023-03-12 at 11:25 -0400, Ken Brown wrote: If I'm right, the solution would seem to be to disable the use of pselect on Cygwin when the jobserver is using a fifo.  I'll try that on a local build of make and see if I can still rep

Re: [bug #64806] "invalid output sync mutex" on windows

2024-01-15 Thread Ken Brown
This is a long shot, but I had a problem a year ago with parallel make on Cygwin occasionally hanging. The solution turned out to be to force make's jobserver to use pipes instead of fifos. If you want to try this, pass make the option '--jobserver-style=pipe'. Ken