Hi Andrey, Thanks for your reply.
> On Apr 24, 2020, at 08:32, Andrey Rahmatullin <w...@debian.org> wrote: > > On Fri, Apr 24, 2020 at 07:29:40AM -0700, Matthew Fernandez wrote: >> The arm64 failure [2] looks legitimate (though I >> don’t understand why excuses also says “arch:arm64 not built yet”), > It says it cannot run autopkgtests for arm64 because the binary isn't > built yet. Ah, I see, I don’t get the same warning for mipsel and armel because the binary built fine there. It’s just that the test suite didn’t complete. >> the mipsel failure [3] looks like some kind of timeout. > It's a build failure caused by the tests not finishing or producing any > output, yes. What is the general approach for long running test suites? Is there a way to extend the timeout? Or is it a hardline that test suites have to complete within the allotted time? >> The armel build [4] looks like it also timed out, but for reasons I >> don’t understand this isn’t considered a migration blocker. > There is no armel binary in testing. Interesting. I did not notice this before and had simply assumed it was in testing because the version that is in testing for other platforms (2020.02.17-1) is listed as “Installed” for armel on the QA page [0]. The logs indicate its tests have never passed on armel, so I’m not sure what “Installed” is referring to here. [0]: https://buildd.debian.org/status/package.php?p=rumur&suite=bullseye >> What is the proper way to address such timeouts? > That depends on their reason. Looks like the tests are indeed very > time-consuming and/or CPU-heavy, as on mips64el they laster for an hour. Perhaps? Time-consuming or CPU-heavy is relative, but maybe these would fall into that category. They run in 10-15 minutes on a local underpowered dev machine, but there was a prior Debian bug about this test suite hammering a build machine (#951497). >> This package has a pretty standard cmake/ctest >> setup, so I would also be interested in hearing from anyone who knows >> how to prevent hiding test suite stdout/stderr on success without >> constructing a manual ctest --verbose invocation in CMakeLists.txt. > A "pretty standard setup" would have many small tests, while this one > evidently only has one large "Tests" test that calls some hand-written > Python script. I don't know anything about ctest but I guess it eats that > script's stdout by default? Yes, ctest swallows the normal stdout of the test script. For local dev and in CI I typically avoid this by running the test script directly, but I wanted as little magic in my debian/rules as possible so I let it do whatever its CMake defaults were.
