On Mon, Mar 19, 2012 at 1:50 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > For an example in our own > problem space look at mysql, whose regression tests take well over an > hour to run on a fast box. So they must be damn near bug-free right? > Uh, not so much, and I think the fact that developers can't easily run > their test suite is not unrelated to that.
The other problem with this approach is that it's hard to keep a huge test suite 100% clean. Changes inevitably introduce behaviour changes that cause some of the tests to fail. If the test suite is huge then it's a lot of work to be continually fixing these tests and you're always behind. If it's always the case that some tests in this huge suite are failing then it's extra work whenever you make a change to dig through the results and determine whether any of the failures are caused by your changes and represent a real problem. Even if you do the work it's easy to get it wrong and miss a real failure. My suggestion would be to go ahead and check in the python or perl script but not make that the pg_regress tests that are run by mak check. Cherry pick just a good set of tests that test most of the tricky bits and check that in to run on make test. I tihnk there's even precedent for that in one of the other modules that has a make longcheck or make slowcheck or something like that. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
