Re: make -t and directories

[Dmitry Goncharov]

On Fri, Mar 10, 2023 at 7:55 PM Alejandro Colomar
 wrote:
> How about using `mkdir -p` instead of touch(1) for nonexistent files?

mkdir -p would create a directory where a file is supposed to be.
Let's say you have hello.c, but no hello.o.If you run make -t it'll
create an empty hello.o. Then you can update hello.c and run make
again and it'll build a new hello.o from the new hello.c. If there
was a directory called 'hello.o' then a file 'hello.o' cannot be
created.

In your particular case, why don't you build your directories before
you run make -t?
$ make dir
$ make -t

regards, Dmitry

Re: make -t and directories

[Alejandro Colomar]

Hi Dmitry,

On 3/11/23 14:36, Dmitry Goncharov wrote:
> On Fri, Mar 10, 2023 at 7:55 PM Alejandro Colomar
>  wrote:
>> How about using `mkdir -p` instead of touch(1) for nonexistent files?
> 
> mkdir -p would create a directory where a file is supposed to be.
> Let's say you have hello.c, but no hello.o.If you run make -t it'll
> create an empty hello.o. Then you can update hello.c and run make
> again and it'll build a new hello.o from the new hello.c. If there
> was a directory called 'hello.o' then a file 'hello.o' cannot be
> created.

Hmm, that's right.  I didn't consider that.  How about running mkdir
for targets whose last byte is a '/'? [1]

Then, with a Makefile like:

$ cat Makefile 
.PHONY: all
all: dir/file

.PHONY: clean
clean:
rm -rf dir

dir/:
mkdir $@

dir/file: | dir/
echo foo >$@

$ make -t
touch dir/
make: touch: open: dir/: Is a directory

I would get the expected behavior.  Something ending in '/' can
only be a directory, so the correct behavior should be to mkdir(1)
it if empty.  touch(1) fails to do anything with a non-existing dir:

$ touch foo/
touch: setting times of 'foo/': No such file or directory

> 
> In your particular case, why don't you build your directories before
> you run make -t?

Because the actual Makefile is not as simple as this reproducer.  :-)

> $ make dir
> $ make -t

I removed the 'builddirs' target long ago because it wasn't easy to
maintain, and I didn't see any benefits.  It was something useful
for recursive make, but now I have a single Makefile, as recommended
by Paul, I'm much happier.  :)

> 
> regards, Dmitry

Kind regards,

Alex


[1]:

Which BTW reminds me we had some discussion about slashes, in which I
had something pending to answer but didn't have time for it, and then
forgot.  Maybe it would be interesting to resurrect it.

 

-- 

GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5


OpenPGP_signature
Description: OpenPGP digital signature

Re: make -t and directories

[Paul Smith]

On Sat, 2023-03-11 at 01:54 +0100, Alejandro Colomar wrote:
> Let's say I have a build tree that has directories which are created
> during build.  The build system has some linters in a `lint` target,
> which touches files when it succeeds.  When I want to lint only a
> specific file, it's interesting to convince make(1) to touch all
> files, and then touch the source file I want to lint; a subsequent
> make(1) invocation will only lint the file I want.

To me that seems like not a good solution.  TBH I've never really
understood the use-case for the -t flag.  But basically what you're
saying here is that you're willing to rebuild your entire source tree
from scratch (because you've touched all the sources) just to run lint
on a specific file you want.  That doesn't seem like a good trade-off
to me.

Why don't you just add a special target for linting a specific file:

lint-%:
 $*

and now if you want to run lint on foo.c you'd say "make lint-foo.c"?

> However, that only works if I've created all the directories, so I
> need to run `make -ij lint && make -t lint`.
> 
> How about using `mkdir -p` instead of touch(1) for nonexistent files?
> The effect would be the same, but it would allow such structures to
> work with this feature.  I don't foresee big issues with it (I don't
> imagine that some systems will depend on make -t creating regular
> empty files).

As Dmitry points out, that would be a disaster because lots of tools
will overwrite existing files without any problem, but no tools will
delete an existing directory.

The right way to handle this is to prefix your lines that you want to
be invoked even with -t, with a "+" operator.  That's one of the things
it's for.

>From the GNU Make manual:

https://www.gnu.org/software/make/manual/html_node/Instead-of-Execution.html

> The ‘-n’, ‘-t’, and ‘-q’ options do not affect recipe lines that
> begin with ‘+’ characters or contain the strings ‘$(MAKE)’ or
> ‘${MAKE}’. Note that only the line containing the ‘+’ character or
> the strings ‘$(MAKE)’ or ‘${MAKE}’ is run regardless of these
> options. Other lines in the same rule are not run unless they too
> begin with ‘+’ or contain ‘$(MAKE)’ or ‘${MAKE}’ (See How the MAKE
> Variable Works.)

Your example if you change your mkdir rule to:

  dir:
+mkdir $@

instead then you get:

  $ make -t
  mkdir dir
  touch dir/file

Re: make -t and directories

[Alejandro Colomar]

Hi Paul,

On 3/11/23 15:20, Paul Smith wrote:
> On Sat, 2023-03-11 at 01:54 +0100, Alejandro Colomar wrote:
>> Let's say I have a build tree that has directories which are created
>> during build.  The build system has some linters in a `lint` target,
>> which touches files when it succeeds.  When I want to lint only a
>> specific file, it's interesting to convince make(1) to touch all
>> files, and then touch the source file I want to lint; a subsequent
>> make(1) invocation will only lint the file I want.
> 
> To me that seems like not a good solution.  TBH I've never really
> understood the use-case for the -t flag.  But basically what you're
> saying here is that you're willing to rebuild your entire source tree
> from scratch (because you've touched all the sources) just to run lint
> on a specific file you want.  That doesn't seem like a good trade-off
> to me.

For my own stuff, I don't do that.  However, for contributors, it's
interesting.  Since the man-pages tree triggers a lot of warnings in
the recently-added linters, it would be very noisy for contributors to
run it on the entire tree.  However, I don't want to add new warnings,
so I want them to run the linters on the pages they touch.

For that, I have the following recommendation in the CONTRIBUTING file:

   Lint
   If you plan to patch a manual page, consider running the linters
   configured in the build system, to make sure your change doesn't add
   new warnings.  However, you might still get warnings that are not your
   fault.  To minimize that, do the following steps:

   (1)  First use make(1)'s -t option, so that make(1) knows that it only
needs to lint again pages that you will touch.

$ make -t lint

   (2)  Touch the page that you'll edit, and run make(1) again, to see
which warnings you'll still see from that page that are not your
fault.

$ touch man2/membarrier.2  # replace by the page you'll modify
$ make -k lint

   (3)  Apply your changes, and then run make(1) again.  You can ignore
warnings that you saw in step (2), but if you see any new ones,
please fix them if you know how, or at least note them in your
patch email.

$ vi man2/membarrier.2  # do your work
$ make -k lint


Since contributors usually clone the repo freshly, or update it very
infrequently, they have the exact situation you mentioned.

> 
> Why don't you just add a special target for linting a specific file:
> 
> lint-%:
>  $*
> 
> and now if you want to run lint on foo.c you'd say "make lint-foo.c"?

I considered doing it, but it would be a lot of extra work in the Makefile.
Since I found a workaround with make -ij, I thought I don't really need
to do that extra work.  Also, it would make it even slower (a simple
`make clean` already takes 1.5 seconds).

$ time make clean V=1
RM -rf  tmp
rm -rf tmp

real0m1.517s
user0m1.130s
sys 0m0.446s

> 
>> However, that only works if I've created all the directories, so I
>> need to run `make -ij lint && make -t lint`.
>>
>> How about using `mkdir -p` instead of touch(1) for nonexistent files?
>> The effect would be the same, but it would allow such structures to
>> work with this feature.  I don't foresee big issues with it (I don't
>> imagine that some systems will depend on make -t creating regular
>> empty files).
> 
> As Dmitry points out, that would be a disaster because lots of tools
> will overwrite existing files without any problem, but no tools will
> delete an existing directory.

Yep.

> 
> The right way to handle this is to prefix your lines that you want to
> be invoked even with -t, with a "+" operator.  That's one of the things
> it's for.

Didn't know about it :)

> 
> From the GNU Make manual:
> 
> https://www.gnu.org/software/make/manual/html_node/Instead-of-Execution.html
> 
>> The ‘-n’, ‘-t’, and ‘-q’ options do not affect recipe lines that
>> begin with ‘+’ characters or contain the strings ‘$(MAKE)’ or
>> ‘${MAKE}’. Note that only the line containing the ‘+’ character or
>> the strings ‘$(MAKE)’ or ‘${MAKE}’ is run regardless of these
>> options. Other lines in the same rule are not run unless they too
>> begin with ‘+’ or contain ‘$(MAKE)’ or ‘${MAKE}’ (See How the MAKE
>> Variable Works.)
> 
> Your example if you change your mkdir rule to:
> 
>   dir:
>   +mkdir $@
> 
> instead then you get:
> 
>   $ make -t
>   mkdir dir
>   touch dir/file
> 

Yup, it's exactly what I needed.




Thanks for your help!


Cheers,

Alex

-- 

GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5


OpenPGP_signature
Description: OpenPGP di

[bug #51200] Improvement suggestion: listen to signals to adjust number of jobs

[Henrik Carlqvist]

Follow-up Comment #2, bug #51200 (project make):

The latest version of the patch (signal_num_jobs5.patch) no longer immediately
increases or decreases the number of jobs when signals USR2 or USR1 are
received, instead, for signal safety, the number of running jobs will be
adjusted when a job is finished.

For those who do not want to apply the patch themselves, I have cloned the git
repo at https://github.com/henca/Henriks-make/tags and applied my patch to tag
4.4.1h.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/

Re: Regression on Cygwin: Problems with parallel make in 4.4

[Ken Brown]

On 3/2/2023 5:39 PM, Ken Brown wrote:
I'm returning to this thread because a surprising thing happened.  I 
decided to try to debug the fifo problem I reported at the beginning of 
the thread (a hang building TeX Live on Cygwin when the jobserver used a 
fifo).  So I installed make 4.4.1 built with fifos enabled (by setting 
CPPFLAGS=-DJOBSERVER_USE_FIFO=1).  And now I can no longer reproduce the 
hang.


Update: The hang occurred again.  It appears to be caused by an infinite 
loop starting with a call to pselect[*].  I looked briefly at the code 
that calls pselect, and I suspect that there is a timing issue.  Perhaps 
certain operations that are supposed to be atomic on Posix platforms are 
not atomic on Cygwin.  (Unfortunately, Cygwin's fifo implementation is 
extremely complicated in order to support multiple readers and writers, 
and atomicity had to be sacrificed.)


If I'm right, the solution would seem to be to disable the use of 
pselect on Cygwin when the jobserver is using a fifo.  I'll try that on 
a local build of make and see if I can still reproduce the problem.  It 
might be several weeks until I'm confident, since the hang occurs only 
sporadically and only after about 90 minutes of running the TeX Live build.


Ken

[*] I say "appears to be" because I was running an optimized build of 
make and an optimized build of the Cygwin DLL, so the gdb backtrace 
might not be reliable.

[bug #60811] Add long-form aliases for automatic variables

[Gwyneth Morgan]

Follow-up Comment #3, bug #60811 (project make):

Will this patch be considered? This would simplify writing understandable
Makefiles that work across implementations.

If long-form aliases like this are unwanted, adding $> as an alias to $^ would
work as well.

[comment #0 original submission:]
> BSD Make has several long-form aliases for automatic variables, such as
`${.TARGET}` for `$@`. These are helpful for readability.
> 
> GNU Make and BSD Make both have automatic variables for all prerequisites,
but they differ (`$^` in GNU, `$>` or `${.ALLSRC}` in BSD), which makes
writing portable makefiles tricky. Adding `${.ALLSRC}` as an alias for `$^`
makes portability easier.
> 
> Introduce these automatic variable aliases for easier readability and
portability:
> 
> 
> .IMPSRC = $<
> .PREFIX = $*
> .TARGET = $@
> .MEMBER = $%
> .ALLSRC = $^  # $> in BSD Make
> .OODATE = $?
> 
> 
> Pass --directory=src when applying patch with git.


___

Reply to this item at:

  

___
Message sent via Savannah
https://savannah.gnu.org/