On Sun, Sep 24, 2017 at 11:30 PM, Fabien COELHO <coe...@cri.ensmp.fr> wrote: > Attached is an attempt at improving the situation: > > (1) there are added comments to explain the whys, which is to provide > coverage for pgbench time-related features... while still not being > time-sensitive, which is a challenge. > > (2) the test now only expects "progress: \d" from the output, so it is > enough that one progress is shown, whenever it is shown. > > (3) if the test is detected to have gone AWOL, detailed log checks are > coldly skipped. > > This would have passed on "skink" under the special conditions it > encountered. > > I cannot guaranty that it would pass under any circumstance, though. > > If it still encounters a failure, ISTM that it should only be a missing > "progress:" in the output, which has not been encountered so far. > > If it occurs, a few options would remain, none of them very convincing: > > - give the test some more time, eg 3 seconds (hmmm... could still fail > after any time...) > > - drop the "progress:" expectation (hmmm... but then it does not check > anything). > > - make the "progress:" output check conditional to the running time > (hmmm... it would require changing the command_checks_all function, > and there is still a chance that the bench was stuck for 2 seconds > then given time to realize that it has to stop right now...) > > - use an even simpler transaction, eg "SELECT;" which is less likely to > get stuck (but still could get stuck...). > > For the random-ness related test (--sample-rate), we could add a feature to > pgbench which allows to control the random seed, so that the number of > samples could be known in advance for testing purposes.
This didn't get any reviews, so bumped to CF 2018-01. -- Michael
