On 2024-Jul-22, Melanie Plageman wrote: > On Sun, Jul 21, 2024 at 5:04 PM Tom Lane <t...@sss.pgh.pa.us> wrote:
> > I note also that the PG_TEST_EXTRA approach has caused xid_wraparound > > to get next-to-zero buildfarm coverage. If that test is actually > > capable of revealing problems, we're unlikely to find out under the > > status quo. > > What is the argument for PG_TEST_EXTRA if it is not running on almost > any buildfarm animals? Are some of those tests valuable for other > reasons than being consistently automatically run (e.g. developer > understanding of how a particular part of code works)? I think it's a bad idea to require buildfarm owners to edit their config files as we add tests that depend on PG_TEST_EXTRA. AFAIR we invented that setting so that tests that had security implications could be made opt-in instead of opt-out; I think this was a sensible thing to do, to avoid possibly compromising the machines in some way. But I think these new tests have a different problem, so we shouldn't use the same mechanism. What about some brainstorming to improve this? For example: have something in the tree that lets committers opt some tests out from specific BF machines without having to poke at the BF machines. I imagine two files: one that carries tags for buildfarm members, something like the /etc/groups file, src/test/tags.lst slow: gull,mamba,mereswine,copperhead and another file that lists tests to skip on members that have certain tags, src/tools/buildfarm/do_not_run.lst slow:src/test/modules/xid_wraparound slow:src/test/recovery/t/043_vacuum_horizon_floor.pl so that run_build.pl know that if the current member has tag slow, then these two tests are to be skipped. Then we can have xid_wraparound enabled generally (without requiring PG_TEST_EXTRA), and the BF client knows not to run it in the particular cases where it's not wanted. This proposal has a number of problems (a glaring one being the maintenance of the list of members per tag), but maybe it inspires better ideas. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "Siempre hay que alimentar a los dioses, aunque la tierra esté seca" (Orual)
