Paul Smith wrote: > On Fri, 2021-08-13 at 14:14 +0100, Howard Chu via Bug reports and > discussion for GNU make wrote: >> I'm looking for a way to expose a job ID number to the individual >> jobs, in a consecutive range from 1-[number of jobs]. Not just >> unique numbers, for that we could just use $$ already. The purpose >> is to e.g. assign non-overlapping network port numbers when firing >> off a number of client/server test scripts in parallel. >> >> Is there anything that could do this now, exposed as a Make variable? > > I don't see any way make can provide something like this, at least not > using the model it does today. The current requirement is that each > read from the pipe is of exactly 1 byte and there's no way to guarantee > a unique value in a single byte of data if there are >256 possible > values of course. > > One possibility would be for make to write out bytes with values > 0,1,2,3... then when it gets to 255 start over again: this would give > unique values for any -j <= 256 and you'd only have to worry about > wrapping for very large values of -j.
We could change the tokens to be 2 bytes. Individual read/write syscalls are still guaranteed to be atomic, so there's no danger of interleaving. The only problem then is ensuring a large enough pipe buffer to avoid premature blocking. > The other downside of using a semaphore is I was thinking of > implementing a "fast fail" feature where as soon as any make in the > jobserver domain died all the other makes would stop quickly as well. > This is something people seem to really want. One way to implement > this is to write "error bytes" into the jobserver pipe when we've hit > "failure mode"; whenever a make instance reads the "error byte" instead > of the "normal byte" it would finish any outstanding jobs then stop. It'd be easier just to close the write-side of the pipe, and then all subsequent read attempts would fail. > This is pretty straightforward although it's not ideal: make can > potentially do a lot of work with its single token without trying to > read from the jobserver for example. > > Of course a single semaphore can't help here. If we go with "named X" > we'll need a directory to hold it; one idea would be to write a > "failed" file into that directory that all make's could check for. Yes, this is another advantage of using pipes, no special considerations about namespaces, filesystem or otherwise. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
