In this context it might be worth mentioning two projects of mine, one simple/lightweight and the other complex and semi-abandoned. I will not describe them here in detail because the links do so.
1. Poor Man's File Auditor (https://github.com/boyski/pmaudit) This is a lightweight, unambitious, but reliable way of figuring out which files were used during the course of a build. It works well in local disk and some NFS configurations; I'm currently battling some NFS caching issues which make its reliability dependent on things like server vendor and mount options. In other words the current version works for some NFS configurations and not yet others but it should work in any filesystem supporting the relatime mount option (which is the default in modern Linux). It's just a matter of figuring out how to force cache flushes which varies per server implementation. 2. Audited Objects (https://github.com/boyski/audited-objects) This is/was a large project aimed at recording not only the files used by a given build but the commands as well, thus a goal similar to the .COMMANDCHANGE and .CACHE features being explored here. Rather than relying on strace/ptrace technology it intercepts system calls using LD_PRELOAD semantics. The original goal was to replicate the ClearCase file auditing and "wink-in" capabilities, for those who remember that system, in pure Linux without any special filesystem. It was going quite well some years ago but then, as often happens, my day job and personal life had other ideas so I had to drop it though not for any technical reason. IIRC the client (recording) side was stable on Linux; I'd also written a server to handle wink-ins from distributed builds but the first attempt was a mess. I had a design for a server rewrite which is where things stopped when I had to drop the project. However, the client works standalone for build auditing and I still use it from time to time. The output format is a little mysterious but it comes with a few Python scripts to parse and interpret it. David B On Tue, Jun 11, 2019 at 7:10 PM David A. Wheeler <dwhee...@dwheeler.com> wrote: > Problem: > > Makefiles often have errors, and they lay dormant because they're > undetected. > > On Tue, 11 Jun 2019 07:44:31 -0700, <th...@vmware.com> wrote: > > Considering that many build systems created with Make do not properly > > perform incremental builds (because all the necessary rules are not > > written, or are written incorrectly), it's also a way to trivially > > make a build that is mysteriously broken because one or more files > > will not be properly rebuilt. > > Of course, that's a potential problem for *any* use of make. > Garbage in, garbage out. I use automated dependency generation > where I can, but the generators can have bugs & not > all situations have a generator. > > > Potential solution: > > Perhaps there should be a make flag that attempts to detect & warn > about missing dependency information during make execution, > to help people fix their makefiles. > > Basically, when running a rule, make could enable strace to see > what is *actually* getting checked/read & written, compare that to what the > rule *claims* are the prerequisites & targets, and then complain about > differences. > > I did some brief experiments using: > strace -f -e trace=%file gcc -o b.o b.c > > As expected, there are a huge number of special-case events. > But if you only consider file-related events in a specific directory or > down > (which could probably default to the current directory), including > relative paths, most of the events that would not normally go in a > makefile disappear. It's also helped if you limit the set of system calls > that > need to be considered (e.g., openat is really important on Ubuntu). > > It might be much easier if this required other tools (like strace) to work, > instead of directly implementing the tracing in make. > This mechanism could be slow, but that's okay; I envision doing this > checking occasionally, instead of doing it on every run. > This might not be implemented on all platforms, but it would be useful > on many platforms & help people fix their makefiles for all. > > --- David A. Wheeler > > _______________________________________________ > Bug-make mailing list > Bug-make@gnu.org > https://lists.gnu.org/mailman/listinfo/bug-make >
_______________________________________________ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
