On Mon, Mar 17, 2025 at 4:30 PM Nathan Bossart <nathandboss...@gmail.com> wrote: > On Mon, Mar 17, 2025 at 04:04:45PM -0400, Robert Haas wrote: > > On Mon, Mar 17, 2025 at 3:34 PM Nathan Bossart <nathandboss...@gmail.com> > > wrote: > >> * Once committed, I should update one of my buildfarm animals to use > >> PG_TEST_PG_UPGRADE_MODE=--swap. > > > > It would be better if we didn't need a separate buildfarm animal to > > test this, because that means you won't get notified by local testing > > OR by CI if you break this. Can we instead have one test that checks > > this which is part of the normal test run? > > That's what I set out to do before I discovered PG_TEST_PG_UPGRADE_MODE. > The commit message for b059a24 seemed to indicate that we don't want to > automatically test all supported modes, but I agree that it would be nice > to have some basic coverage for --swap on CI/buildfarm regardless of > PG_TEST_PG_UPGRADE_MODE. How about we add a simple TAP test (attached), > and I still plan on switching a buildfarm animal to --swap for more > in-depth testing?
The background here is that I'm kind of on the warpath against weird configurations that we only test on certain buildfarm animals at the moment, because the result of that is that CI is clean and then the buildfarm turns red when you commit. That's an unenjoyable experience for the committer and for everyone who looks at the buildfarm results. The way to fix it is to stop relying on "rerun all the tests with this weird mode flag" and rely more on tests that are designed to test that specific flag and, ideally, that get run by in local testing or at least by CI. I'm not quite sure what the best thing is to do is for the pg_upgrade tests in particular, and it may well be best to do as you propose for now and figure that out later. But I question whether just rerunning all of those tests with several different mode flags is the right thing to do. Why for example does 005_char_signedness.pl need to be checked under both --link and --clone? I would guess that there are one or maybe two tests in src/bin/pg_upgrade/t that needs to test --link and --clone and they should grow internal loops to do that (when supported by the local platform) and PG_UPGRADE_TEST_MODE should go in the garbage. -- Robert Haas EDB: http://www.enterprisedb.com
