[
https://issues.apache.org/jira/browse/SOLR-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384839#comment-16384839
]
Mark Miller commented on SOLR-12028:
------------------------------------
bq. why should any open jira saying "this test is broken" have been resolved
just to redirect here / open a new issue later?
As I mentioned above, I also don't think this is the way to go. It's fine to
open an issue to track all this work, but makes no sense to close the issues
per problem test.
bq. why should ANY test that was already marked AwaitsFix, pointing to an open
jira have been changed to BadApple?
The only reason I believe that should happen is because the test does not fail
very often vs failing a lot.
bq. we have far more productive things to do than argue over that one.
I don't think any of us are interested in arguing, just taking the best
approach. Our annotations are made to point at a single issue where we can
discuss what is wrong with a test and attach patches, etc. When it's closed we
can remove the annotation. Closing those kinds of issues for this bulk issue is
not something I want to argue about - it's something that seems ... wrong.
This issue should be about annotations. Let the other issues live and be about
fixing the individual tests. They close when the annotations go away.
> BadApple and AwaitsFix annotations usage
> ----------------------------------------
>
> Key: SOLR-12028
> URL: https://issues.apache.org/jira/browse/SOLR-12028
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
> Reporter: Erick Erickson
> Assignee: Erick Erickson
> Priority: Major
> Attachments: SOLR-12016-buildsystem.patch,
> SOLR-12028-sysprops-reproduce.patch, SOLR-12028.patch, SOLR-12028.patch
>
>
> There's a long discussion of this topic at SOLR-12016. Here's a summary:
> - BadApple annotations are used for tests that intermittently fail, say < 30%
> of the time. Tests that fail more often shold be moved to AwaitsFix. This is,
> of course, a judgement call
> - AwaitsFix annotations are used for tests that, for some reason, the problem
> can't be fixed immediately. Likely reasons are third-party dependencies,
> extreme difficulty tracking down, dependency on another JIRA etc.
> Jenkins jobs will typically run with BadApple disabled to cut down on noise.
> Periodically Jenkins jobs will be run with BadApples enabled so BadApple
> tests won't be lost and reports can be generated. Tests that run with
> BadApples disabled that fail require _immediate_ attention.
> The default for developers is that BadApple is enabled.
> If you are working on one of these tests and cannot get the test to fail
> locally, it is perfectly acceptable to comment the annotation out. You should
> let the dev list know that this is deliberate.
> This JIRA is a placeholder for BadApple tests to point to between the times
> they're identified as BadApple and they're either fixed or changed to
> AwaitsFix or assigned their own JIRA.
> I've assigned this to myself to track so I don't lose track of it. No one
> person will fix all of these issues, this will be an ongoing technical debt
> cleanup effort.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
