On Sat, Aug 8, 2015 at 8:24 PM, Peter Geoghegan <p...@heroku.com> wrote: > I think that there needs to be a way of running an extended set of > regression tests. I could definitely respect the desire for minimalism
The larger expense in having extensive test suites is the cost to maintain them. With our current test framework tests have to be fairly carefully written to produce output that isn't very fragile and sensitive to minor changes in the code. In practice this is what really drives the push towards minimal tests. If we tried testing every code path right now -- which I tried to do once for the TOAST code -- what you'll find is that what you're really testing is not that the code is correct but that it does exactly what it does today. At that point the test failures become entirely noise meaning "something changed" rather than signal meaning "something's broken". A better test framework can go a long way towards fixing this. It would be much less of a problem if we had a unit test framework rather than only black box integration tests. That would allow us to test that every function in tuplestore.c meets its contract, not just that some SQL query produces output that's correct and we think happened to go through some code path we were interested in. There are many code paths that will be hard to arrange to reach from SQL and impossible to have any confidence will continue to be reached in the future when the behaviour is intentionally changed. That said, I don't think anyone would object to adding some regression tests that at least test basic correctness of the sorting code. That's a pretty embarrassing gap and iirc the only reason for it is that it would make the regression tests sensitive to collation locale settings. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
