On Fri, Sep 23, 2016 at 6:55 AM, Nelson H. F. Beebe <be...@math.utah.edu> wrote: > Thanks, Jim, for the patch to backref-multibyte-slow; I've applied it > to a new directory grep-2.25.92-f3e9.p1, and have builds in progress > now on the Fedora systems. > > I think that any kind of wallclock-time-dependent test in a package > test suite is seriously prone to failure.
Writing portable perf-measuring tests is hard, indeed. I agree they're prone to failure, *if* we're considering tests with naive absolute limits. This however was a carefully written test to ensure that the matcher no longer has O(N^2) behavior, using only relative duration comparisons. The problem is that I don't want to make the test duration unnecessarily long. If I didn't mind a longer-duration initial test (to get a reference time), then there'd be no problem requiring that the potentially much-longer-in-case-of-regression one terminates in < 10x that duration. Anyway, thanks for confirming that this simple fix is probably enough. > Although some developers > may have the luxury of a fast personal machine on which nothing else > is running when a package is built, many of us live in multiuser These tests are designed not just to work on my systems, but also on bottom-end molasses-slow m68k simulators. > environments where the load on the build machines is completely > unpredictable, and beyond our control. > > Also, for someone like me with numerous VMs running on a single > desktop, parallel builds will certainly drive up the load. With the > large number of test systems that I have, there is no reasonable way > to do the builds sequentially, because such a process could take hours > to days to complete. Hah. I would never suggest that you or anyone else run builds/tests sequentially. I have been militant about making things work in parallel ever since Roland added job support to GNU make.
