On Sat, Aug 10, 2024 at 8:16 PM Navin P <navinp0...@gmail.com> wrote: > > On Sun, Aug 11, 2024, 06:29 Kaz Kylheku <k...@kylheku.com> wrote: > > > Your makefile is broken, or at least not suitable for parallel builds. > > > > You should add .NOTPARALLEL: to it. That's the minimal change that will > > fix it. A makefile with the .NOTPARALLEL: target will not be parallelized > > even if -j is requested on the command line or via the environment. > > > > Why it's broken is that you have multiple rules that update the same > > object. > > > > An archive should be built by exactly one rule which has all the > > prerequisites that are going into the archive. > > > > On August 10, 2024 4:04:13 AM PDT, Navin P <navinp0...@gmail.com> wrote: > > >Hi, > > > I ran multiple instances of ar during a parallel build and they were > > >updating different object files using ar rv sometimes the same object file > > >again. > > > > > >When i run make -j1 i find all the object files libx.a but when i do make > > >-j8 i find some missing object files inside libx.a. > > > > > >Then i looked at ar.c and found that there is no synchronization between 2 > > >instances of ar on same library. The problem is that makefile is generated > > >and there are lots of bash scripts doing that. > > > > > >Is there any way to fix the problem? > > >1. Without changing makefile > > >2. With changing the makefile in long term. > > > > > > > I was looking at > https://stackoverflow.com/questions/8824969/how-do-i-avoid-concurrency-problems-when-using-gnu-make-for-parallel-builds-of-a > > AR := flock make.lock $(AR) > > > What are the problems with using flock ?
I think the correctness concern is that this provides mutual exclusion for ar only. Other programs may be reading the .a, including make itself. If your system's ar does atomic updates (write new temp file, rename over existing .a) then the readers should have a self-consistent view and this should work Just Fine. If not, then depending on exactly how it does update then make may get confused about the .a contents. The performance concern is two-fold: 1) ar's normal operation includes rebuilding the symbol table after each change. Adding objects to a .a with separate ar invocations results in O(N^2) symbol operations. 2) make's parallel operation isn't aware of the "these operations will be serialized" behavior, so make may schedule multiple 'invoke rule using $(AR)' recipes concurrently despite them being serialized, resulting in reduced parallelism vs the "build all of the out-of-date objects then insert them all at once" rule setup. Philip Guenther
