Re: reducing isolation tests runtime

2019-02-13 Thread Tom Lane
Andres Freund writes: > Right, but why don't we allow for more tests in a group, and then use a > default max_connections to limit concurrency? Having larger groups is > advantageous wrt test runtime - it reduces the number of artificial > serialization point where the slowest test slows things do

Re: reducing isolation tests runtime

2019-02-13 Thread Andres Freund
Hi, On 2019-02-13 12:41:41 -0500, Tom Lane wrote: > Andres Freund writes: > > Do you have an idea why we have both max_concurrent_tests *and* > > max_connections in pg_regress? ISTM the former isn't really useful given > > the latter? > > No, the former is a static restriction on what the schedu

Re: reducing isolation tests runtime

2019-02-13 Thread Tom Lane
Andres Freund writes: > Do you have an idea why we have both max_concurrent_tests *and* > max_connections in pg_regress? ISTM the former isn't really useful given > the latter? No, the former is a static restriction on what the schedule file is allowed to contain, the latter is a dynamic restrict

Re: reducing isolation tests runtime

2019-02-13 Thread Andres Freund
On 2019-02-13 10:58:50 -0500, Tom Lane wrote: > Alvaro Herrera writes: > > On 2019-Feb-13, Tom Lane wrote: > >> Some of the slower buildfarm critters use MAX_CONNECTIONS to limit > >> the load on their hosts. As long as the isolation tests honor that, > >> I don't see a real need for a separate s

Re: reducing isolation tests runtime

2019-02-13 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Feb-13, Tom Lane wrote: >> Some of the slower buildfarm critters use MAX_CONNECTIONS to limit >> the load on their hosts. As long as the isolation tests honor that, >> I don't see a real need for a separate serial schedule. > MAX_CONNECTIONS was the only reason I

Re: reducing isolation tests runtime

2019-02-13 Thread Alvaro Herrera
On 2019-Feb-13, Tom Lane wrote: > Andres Freund writes: > > I'm working an updated version of this. Adding the new tests is a bit > > painful because of conflicting names making it harder than necessary to > > schedule tests. While it's possible to work out a schedule that doesn't > > conflict, i

Re: reducing isolation tests runtime

2019-02-13 Thread Tom Lane
Andres Freund writes: > I'm working an updated version of this. Adding the new tests is a bit > painful because of conflicting names making it harder than necessary to > schedule tests. While it's possible to work out a schedule that doesn't > conflict, it's pretty annoying to do and more importan

Re: reducing isolation tests runtime

2019-02-12 Thread Andres Freund
Hi, On 2018-01-25 17:34:15 -0300, Alvaro Herrera wrote: > Tom Lane wrote: > > Alvaro Herrera writes: > > > > I think we could solve this by putting in the same parallel group only > > > slow tests that mostly sleeps, ie. nothing that would monopolize CPU for > > > long enough to cause a problem.

Re: reducing isolation tests runtime

2018-12-04 Thread Andres Freund
Hi, On 2018-12-04 15:45:39 -0500, Tom Lane wrote: > Andres Freund writes: > > I'd like to see this revived, getting a bit tired waiting longer and > > longer to see isolationtester complete. Is it really a problem that we > > require a certain number of connections? Something on the order of 30-

Re: reducing isolation tests runtime

2018-12-04 Thread Tom Lane
Andres Freund writes: > I'd like to see this revived, getting a bit tired waiting longer and > longer to see isolationtester complete. Is it really a problem that we > require a certain number of connections? Something on the order of 30-50 > connections ought not to be a real problem for realist

Re: reducing isolation tests runtime

2018-12-04 Thread Alvaro Herrera
On 2018-Dec-04, Andres Freund wrote: > Hi, > > I'd like to see this revived, getting a bit tired waiting longer and > longer to see isolationtester complete. Is it really a problem that we > require a certain number of connections? Something on the order of 30-50 > connections ought not to be a

Re: reducing isolation tests runtime

2018-12-04 Thread Andres Freund
Hi, On 2018-01-25 18:27:28 -0300, Alvaro Herrera wrote: > Tom Lane wrote: > > Alvaro Herrera writes: > > > Here's a concrete proposal. Runtime is 45.7 seconds on my laptop. It > > > can be further reduced, but not by more than a second or two unless you > > > get in the business of modifying ot

Re: reducing isolation tests runtime

2018-01-25 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Here's a concrete proposal. Runtime is 45.7 seconds on my laptop. It > > can be further reduced, but not by more than a second or two unless you > > get in the business of modifying other tests. (I only modified > > deadlock-soft-2 because it saves 5

Re: reducing isolation tests runtime

2018-01-25 Thread Tom Lane
Alvaro Herrera writes: > Here's a concrete proposal. Runtime is 45.7 seconds on my laptop. It > can be further reduced, but not by more than a second or two unless you > get in the business of modifying other tests. (I only modified > deadlock-soft-2 because it saves 5 seconds). Looks reasonab

Re: reducing isolation tests runtime

2018-01-25 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > I think we could solve this by putting in the same parallel group only > > slow tests that mostly sleeps, ie. nothing that would monopolize CPU for > > long enough to cause a problem. Concretely: > > test: timeouts tuplelock-update deadlock-hard deadl

Re: reducing isolation tests runtime

2018-01-25 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Alvaro Herrera writes: >>> On the subject of test total time, we could paralelize isolation tests. >> BTW, one small issue there is that the reason the timeouts test is so >> slow is that we have to use multi-second timeouts to be sure slower >> buildf

Re: reducing isolation tests runtime

2018-01-25 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > On the subject of test total time, we could paralelize isolation tests. > > Right now "make check" in src/test/isolation takes 1:16 on my machine. > > Test "timeouts" takes full 40s of that, with nothing running in parallel > > -- the machine is complet

Re: reducing isolation tests runtime

2018-01-24 Thread Tom Lane
Alvaro Herrera writes: > On the subject of test total time, we could paralelize isolation tests. > Right now "make check" in src/test/isolation takes 1:16 on my machine. > Test "timeouts" takes full 40s of that, with nothing running in parallel > -- the machine is completely idle. BTW, one small

Re: reducing isolation tests runtime

2018-01-24 Thread Tom Lane
Robert Haas writes: > On Wed, Jan 24, 2018 at 6:10 PM, Alvaro Herrera > wrote: >> On the subject of test total time, we could paralelize isolation tests. > Oh, cool. Yes, the time the isolation tests take to run is quite > annoying. I didn't realize it would be so easy to run it in parallel.

Re: reducing isolation tests runtime

2018-01-24 Thread Robert Haas
On Wed, Jan 24, 2018 at 6:10 PM, Alvaro Herrera wrote: > On the subject of test total time, we could paralelize isolation tests. > Right now "make check" in src/test/isolation takes 1:16 on my machine. > Test "timeouts" takes full 40s of that, with nothing running in parallel > -- the machine is c