Am 30.04.2017 um 09:19 schrieb Niels Thykier: [...] >> Please note that the tests are not disabled but the results are simply >> ignored. >> [...] > >>From my PoV, that is basically the same as disabling them. We do not > pre-emptively check log files during rebuild tests/buildd builds, so we > would not notice the regression. :)
I think there is a subtle but important difference. :) If the tests fail to build from source they will still cause a build failure but their results will not. Not every test failure automatically implies broken or unusable software. Sometimes the tests make wrong assumptions about the available software versions (since Java is version-centric and Debian is a bit more flexible in this regard) or have hardware requirements that cannot be satisfied on the current system but don't indicate that the program is buggy. Every test failure has to be investigated separately and the severity can range from minor to grave but due to the current default behavior every test failure is equivalent to a build failure and thus release critical. I believe this often creates a wrong picture and in my opinion it is sensible to make test failures non-fatal in this situation until we have got feedback from upstream or other proof that we are dealing with a serious issue. As a side note: I wonder why QA folk come up with i386 rebuilds for arch:all packages during a hard freeze and not months before when we are way more flexible to approach those issues. As much as I appreciate the goal to make arch:all packages buildable on all release architectures, it is not a trivial task and reporting 100 bugs is much easier than fixing them. I think we should better communicate these kind of release goals at the start of a release cycle and then I would like to see more people who actually want to work on them. At the moment it appears we have a two-class society where people come up with new QA ideas every week but expect from the other side to do all the implementation. I don't think this will work long term. Regards, Markus
signature.asc
Description: OpenPGP digital signature
