Hi David, Thanks for your informative response, comments and references.
I do see your point (and in a way wanted to attack the separation of the DAG myself with this topic). However, the advice to place all make information into a top-level Makefile.am is not always practical. The point of subdirectories is isolation, this is why we put files into separate directories. The dependencies between such directories would be best placed besides them not in some central makefile. Henrik mentioned inclusion. I find that a more attractive solution as it seems to combine the best of both worlds: it has both isolation and results in a single DAG. Do you think autotools is possible to instrument to do that? Any advice on how to set it up? Zoltan -----Original Message----- From: David A. Wheeler <dwhee...@dwheeler.com> Sent: Tuesday, May 23, 2023 8:32 PM To: Henrik Carlqvist <h...@poolhem.se>; bug-make <bug-make@gnu.org> Cc: Zoltán Turányi <zoltan.tura...@ericsson.com> Subject: Re: new feature idea: ingesting processed rulesets > > On Tue, 23 May 2023 12:07:51 -0400 > "David A. Wheeler" <dwhee...@dwheeler.com> wrote: >> The solution is to *NOT* use recursive make. Have *ONE* process run >> the makefile, with the correct data. Now you can enable parallel >> jobs, and have it run really quickly, because the make process has the >> correct information. >> Using this approach you can routinely run large make jobs in a >> fraction of a second. > On May 23, 2023, at 1:16 PM, Henrik Carlqvist <h...@poolhem.se> wrote: > If you prefer to do it that way, you can have your top Makefile > include all sub.mk files in the directory structure that it is able to find. Yes, absolutely. The point is that you want a single process to have all the information, so it can perform the correct actions... multiple *files* is fine. As noted in "Recursive Make Considered Harmful" by Peter Miller <"Recursive Make Considered Harmful" by Peter Miller, e.g.: https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-01613997b4600749&q=1&e=93bda6e5-ed75-46ba-a224-c1c4e19d671a&u=https%3A%2F%2Faccu.org%2Fjournals%2Foverload%2F14%2F71%2Fmiller_2004%2F%2F>: "The above [long list of problems] is based on one simple action: the DAG was artificially separated into incomplete pieces. This separation resulted in all of the problems familiar to recursive make builds. Did make get it wrong? No. This is a case of the ancient GIGO principle: Garbage In, Garbage Out . Incomplete Makefiles are wrong Makefiles. To avoid these problems, don’t break the DAG into pieces; instead, use one Makefile [process] for the entire project... modern make implementations have include statements. . By including a relevant fragment from each module, the total size of the Makefile and its include files need be no larger than the total size of the Makefile s in the recursive case." I find that "whole project" Makefiles are *incredibly* fast. I auto-generate dependencies from source in larger projects, but store them in ".d" files that depend on their sources; thus, regeneration of dependencies only happens when it needs to happen. Modern filesystems (with modern caches) can load all those make fragments in a blink of an eye. And they completely eliminate the endless problems of recursive make approaches. Here are some things you might find useful for using make well: "A Super-Simple Makefile for Medium-Sized C/C++ Projects" https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-de62c1656fb0d189&q=1&e=93bda6e5-ed75-46ba-a224-c1c4e19d671a&u=https%3A%2F%2Fspin.atomicobject.com%2F2016%2F08%2F26%2Fmakefile-c-projects%2F https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-f93c4da8de9d600e&q=1&e=93bda6e5-ed75-46ba-a224-c1c4e19d671a&u=http%3A%2F%2Fmake.mad-scientist.net%2Fpapers%2Fadvanced-auto-dependency-generation%2F My make-booster: https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-908c634bcb7bf69a&q=1&e=93bda6e5-ed75-46ba-a224-c1c4e19d671a&u=https%3A%2F%2Fgithub.com%2Fdavid-a-wheeler%2Fmake-booster --- David A. Wheeler
