On Sun, Aug 16, 2015 at 7:33 AM, Noah Misch <n...@leadboat.com> wrote: > When I've just spent awhile implementing a behavior change, the test diffs are > a comforting sight. They confirm that the test suite exercises the topic I > just changed. Furthermore, most tests today do not qualify under this > stringent metric you suggest. The nature of pg_regress opposes it.
It's not a comforting sight for me. It means I need to change the test results and then of course the tests will pass. So they didn't really test anything and I don't really know whether the changes broke anything. Any test I had to adjust for my change was effectively worthless. This is precisely my point about pg_regress and why it's the reason there's pressure not to have extensive tests. It's useful to have some end-to-end black box tests like this but it's self-limiting. You can't test many details without tying the tests to specific behaviour. I have worked with insanely extensive testing suites that didn't have this problem. But they were unit tests for code that was structured around testability. When the API is designed to be testable and you have good infrastructure for mocking up the environment and comparing function results in a much narrower way than just looking at text output you can test the correctness of each function without tying the test to the specific behaviour. *That* gives a real comfort. When you change a function to behave entirely differently and can still run all the tests and see that all the code that depends on it still operate correctly then you know you haven't broken anything. In that case it actually *speeds up* development rather than slowing it down. It lets newbie developers hack on code where they don't necessarily know all the downstream dependent code and not be paralyzed by fear that they'll break something. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
