Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-07 Thread Ramsay Jones
On 07/06/17 11:24, Jeff King wrote: > On Mon, Jun 05, 2017 at 10:55:42AM +0900, Junio C Hamano wrote: > > If you use "prove", it already records this information, and it can > print it with "--timer". Yes I have DEFAULT_TEST_TARGET=prove GIT_PROVE_OPTS='--timer' in my config.mak and r

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-07 Thread Jeff King
On Mon, Jun 05, 2017 at 10:55:42AM +0900, Junio C Hamano wrote: > I do not expect any single person to tackle the splitting. I just > wished that a patch inspired by this patch (or better yet, a new > version of this patch) made the tail end of "make test" output to > read like this: > >...

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-05 Thread Ævar Arnfjörð Bjarmason
On Mon, Jun 5, 2017 at 9:03 PM, Stefan Beller wrote: >> That's never going to be a problem on a less beefy machine with >> --state=slow,save, since the 30s test is going to be long over by the >> time the rest of the tests run. >> >> Cutting down on these long tail tests allows me to e.g. replace

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-05 Thread Stefan Beller
> That's never going to be a problem on a less beefy machine with > --state=slow,save, since the 30s test is going to be long over by the > time the rest of the tests run. > > Cutting down on these long tail tests allows me to e.g. replace this: > > git rebase -i --exec '(make -j56 all && cd t

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-05 Thread Stefan Beller
On Sun, Jun 4, 2017 at 10:48 PM, Christian Couder wrote: >> when the hidden feature is _not_ used, so that wider set of people >> will be forced to see that some tests take inordinate amount of >> time, and entice at least some of them to look into it. > > I wonder if splitting tests would make a

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-05 Thread Ævar Arnfjörð Bjarmason
On Mon, Jun 5, 2017 at 3:17 PM, Lars Schneider wrote: > >> On 04 Jun 2017, at 09:29, Ævar Arnfjörð Bjarmason wrote: >> >> On Sun, Jun 4, 2017 at 2:31 AM, Junio C Hamano wrote: >>> Ævar Arnfjörð Bjarmason writes: >>> Speeding up the test suite by simply cataloging and skipping tests t

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-05 Thread Lars Schneider
> On 04 Jun 2017, at 09:29, Ævar Arnfjörð Bjarmason wrote: > > On Sun, Jun 4, 2017 at 2:31 AM, Junio C Hamano wrote: >> Ævar Arnfjörð Bjarmason writes: >> >>> Speeding up the test suite by simply cataloging and skipping tests >>> that take longer than N seconds is a hassle to maintain, and e

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-04 Thread Christian Couder
On Mon, Jun 5, 2017 at 3:55 AM, Junio C Hamano wrote: > Ævar Arnfjörð Bjarmason writes: > >> Realistically I'm going to submit this patch, I'm not going to take >> the much bigger project of refactoring the entire test suite so that >> no test runs under N second, and of course any such refactori

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-04 Thread Junio C Hamano
Ævar Arnfjörð Bjarmason writes: >> I certainly understand that but in the longer term, I'd prefer the >> approach to call out an overly large test. That will hopefully >> motivate us to split it (or speed up the thing) to help folks on >> many-core machines. > > The reason I didn't document this

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-04 Thread Ævar Arnfjörð Bjarmason
On Sun, Jun 4, 2017 at 2:31 AM, Junio C Hamano wrote: > Ævar Arnfjörð Bjarmason writes: > >> Speeding up the test suite by simply cataloging and skipping tests >> that take longer than N seconds is a hassle to maintain, and entirely >> skips some tests which would be nice to at least partially r

Re: [PATCH] test-lib: add ability to cap the runtime of tests

2017-06-03 Thread Junio C Hamano
Ævar Arnfjörð Bjarmason writes: > Speeding up the test suite by simply cataloging and skipping tests > that take longer than N seconds is a hassle to maintain, and entirely > skips some tests which would be nice to at least partially run, > e.g. instead of entirely skipping t3404-rebase-interact

[PATCH] test-lib: add ability to cap the runtime of tests

2017-06-03 Thread Ævar Arnfjörð Bjarmason
Add a GIT_TEST_TIMEOUT environment variable which optimistically sets an approximate upper limit on how long any one test is allowed to run. Once the timeout is exceeded all remaining tests are skipped, no attempt is made to stop a long running test in-progress, or deal with the edge case of the e