Hi Lars,
Lars Wirzenius <l...@liw.fi> writes:
> On Mon, Jan 16, 2017 at 08:50:57AM +0100, Ole Streicher wrote:
>> Sean Whitton <spwhit...@spwhitton.name> writes:
>> > I agree with the principle that test failures should be RC by default.
>>
>> This is something which seems to have no disagreement here. My concern
>> is just that I want to have a simple way to override this, to assign
>> this to a different package etc. I want to have the same flexibility
>> here as for bugs.
>
> A failing test means there's a bug. It might be in the test itself, or
> in the code being tested. It might be a bug in the test environment.
>
> Personally, I'd really rather have unreliable tests fixed. Unreliable
> tests are like playing Russian roulette: mostly OK but sometimes you
> get a really loud noise that makes your parents and loved ones be
> ashamed of you.
I fully agree with you. I just think that it is not necessarily RC that
a CI test is unfixed.
The point here is: the proposed plan is to make CI test failures in
reverse dependencies a direct migration excuse, which can only be
overwritten by the release team.
I find this too unflexible, and propose that instead the failing CI test
should create an RC bug assigned to the updated package, affecting the
failing package. This would
* document the bug on the place where all bugs are documented, and keep
it in our eternal bug database, allowing to search for it etc.
* enable all the possibilities an open bug has, like discussion,
re-assignment, severity change etc.
* better link the d/changelog entry to the problem ("Increasing
tolerance in picky test. Closes: #123456" instead of "... to fix a CI
test failure with updated grampf package")
> Picture this: a cocktail party. Many people mingling around, dressed
> up and engaging in smalltalk, sipping colourful drinks.
Nice picture :-)
> But for months, they had to wait for an invitation to a new party.
At least, I would not like to go to a coctail party where the host
announces that he kicks out the people for that reason. This should be
on the decision of the parents äääh maintainers.
IMO, we should trust the maintainer and their decisions until there is
no experience that it doesn't work. Which means: keep the maintainer
fully responsible on the package, including the ability to lower
severity of a CI test or any other bug. Only if we experience that this
doesn't work, we need other measures.
Best regards
Ole
