Pádraig Brady <p...@draigbrady.com> writes: > To get a better handle on the generality of this > I went back about 16 years on the list, > and even though there have been many tee functionality requests over the > years, > this is the first time this has been requested.
I'm glad not to have wasted time on a redundancy! I totally overlooked checking the MLs, so I only searched through the "rejected features" page; I'll remember to check archives too in the future. > Trying to understand your use case better, > I'm presuming the input generator is depending on the compiler runs > (written to by tee) to exit cleanly, before exiting / generating more? > Hence the hangs? > If that was the case then there still might be the potential for hangs > even if tee detected closed pipes. I.e. if the compiler runs hung rather > than exited, this not be distinguishable from tee as the pipe outputs would > remain. Yes, but these are fine. The problem here isn't that the build process can hang, the problem is that it can hang in a way different to the compilers it's emulating. The work tee is involved in consists of running the configure stage of a package build against two compilers, universally, by essentially tee'ing to both, storing and comparing the results, and then returning the result of the "normal" compiler, in order to detect packages that misconfigure due to compiler changes. The bug we observed is that on occasion, for instance when running with a tty, or with a script that (for some reason) has a pipe on stdin, the tee-based "compiler" would hang. To replicate this, try: tee >(gcc test.c -o a.out.1) >(gcc test.c -o a.out.2) in a tty (here, the stdin is meant to be irrelevant). In an applied version of this scenario, this would be cc-old/normal and cc-new of some sort, through a wrapper function so that results are stored/retrieved/compared/etc, but that complexity isn't necessary to see to understand the issue I think (but [1] is the script involved in the process, see the timeout line and the comment above it) The behavior we'd want out of tee here is to let it pass down as much data as possible to both compilers, and to be able to give up when the last of the compilers die, hence waiting for POLLHUP events. This is useful in other instances where stdin is not necessarily relevant to what tee is sending data to, and where you can't deduce whether it would be (at least, without fairly complex code to replicate argument parsing). > If that's the case this has become a bit less generally useful in my mind. > > To keep tee data driven, perhaps your input could periodically > send a "clock" input through to (say a newline), to check everything > is still running as expected. I.e. your periodic input generator > seems like it would be async already, so it would be better to > add any extra async logic there, and keep tee more simple and data driven. This is, sadly, not in our code. We control (distro) build scripts (ebuilds) and the compiler-diff-wrapper-thing, and the ability to run (normal program) build systems unmodified, with just a custom CC, in order to detect when they misbehave due to a compiler update is paramount here. > cheers, > Pádraig. > > p.s. while reading the tee docs in this area, they were a bit dispersed, > so the attached updates the texinfo for the -p option to summarise > it's operation and how it differs from the default operation. Thanks! This seems nice. Have a great night. [1] https://gist.github.com/thesamesam/4ddaa95f3f42c2be312b676476cbf505 -- Arsen Arsenović
signature.asc
Description: PGP signature
