[
https://issues.apache.org/jira/browse/SOLR-12028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376382#comment-16376382
]
Erick Erickson commented on SOLR-12028:
---------------------------------------
The attached patch is what it's looking like so far. Below is a list of all the
BadApple and AwaitsFix annotations in the code base currently. I have to go
over this again and run the test suite, but wanted to post them in case people
want to keep some running because they're actively getting attention. Some of
the failing tests are in areas getting a lot of active work so may be good
candidates for leaving in, just let me know.
Assuming that tests pass, I'm probably going to check these in Monday sometime
after making another pass to see whether I put them in the right category.
I intend to post the below info to the dev list each Sunday (or so) so these
tests don't get lost forever.
AtomicUpdateProcessorFactoryTest.java
testMultipleThreads()
@BadApple(bugUrl = "https://issues.apache.org/jira/browse/SOLR-10734";)
AutoAddReplicasIntegrationTest.java
testSimple()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
AutoScalingHandlerTest.java
testReadApi()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
AutoscalingHistoryHandlerTest.java
testHistory()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
CdcrBootstrapTest.java
testBootstrapWithContinousIndexingOnSourceCluster()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
CdcrReplicationDistributedZkTest.java
distribSetUp()
@BadApple(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028";)
ComputePlanActionTest.java
testSelectedCollections()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
DocValuesNotIndexedTest.java
testGroupingDocAbsent()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
ForceLeaderTest.java
testReplicasInLIRNoLeader()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
HdfsChaosMonkeyNothingIsSafeTest.java
void
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
HdfsDirectoryTest.java
testEOF()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
HttpPartitionTest.java
test()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
LeaderFailoverAfterPartitionTest.java
test()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
LeaderInitiatedRecoveryOnShardRestartTest.java
void
@LuceneTestCase.BadApple(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-10071";)
MoveReplicaHDFSTest.java
testNormalFailedMove()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
MoveReplicaTest.java
testFailedMove()
// @AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11458";)
PeerSyncReplicationTest.java
test()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
ReplaceNodeNoTargetTest.java
test()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
ReplicationFactorTest.java
test()
@LuceneTestCase.BadApple(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-6944";)
SSLMigrationTest.java
test()
@BadApple(bugUrl = "https://issues.apache.org/jira/browse/SOLR-6213";)
ShardSplitTest.java
test()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
SharedFSAutoReplicaFailoverTest.java
test()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
StreamExpressionTest.java
testDistributions()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestAuthenticationFramework.java
testBasics()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestCloudPseudoReturnFields.java
testMultiValuedRTG()
// @AwaitsFix'ed)
TestCollapseQParserPlugin.java
testStringCollapse()
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/SOLR-11974";)
TestControlledRealTimeReopenThread.java
testCRTReopen()
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5737";)
TestDelegationWithHadoopAuth.java
testDelegationTokenCancelFail()
// @AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-14044";)
TestGeo3DPoint.java
testRandomBig()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestICUNormalizer2CharFilter.java
testRandomStrings()
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5595";)
TestICUTokenizerCJK.java
testRandomHugeStrings()
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5575";)
TestICUTokenizerCJK.java
testRandomStrings()
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-5575";)
TestImpersonationWithHadoopAuth.java
testForwarding()
@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-9893";)
TestJmxIntegration.java
testJmxOnCoreReload()
@Test @BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-2715";) //
timing problem?
TestLTRReRankingPipeline.java
testDifferentTopN()
@BadApple(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028";)
TestLargeCluster.java
testSearchRate()
@BadApple(bugUrl = "https://issues.apache.org/jira/browse/SOLR-11714";)
TestManagedResourceStorage.java
testZkBasedJsonStorage()
@LuceneTestCase.BadApple(bugUrl =
"https://issues.apache.org/jira/browse/SOLR-6443";)
TestMinMaxOnMultiValuedField.java
testDoubleFieldCache()
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMinMaxOnMultiValuedField.java
testFloatFieldCache()
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMinMaxOnMultiValuedField.java
testIntFieldCache()
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMinMaxOnMultiValuedField.java
testLongFieldCache()
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-6709";)
TestMoreLikeThis.java
testMultiFieldShouldReturnPerFieldBooleanQuery()
@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/LUCENE-7161";)
TestPrepRecovery.java
testLeaderNotResponding()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestPrepRecovery.java
testLeaderUnloaded()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestPullReplica.java
testAddDocs()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestReplicationHandler.java
doTestIndexAndConfigReplication()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestReplicationHandler.java
doTestIndexFetchOnMasterRestart()
//@AwaitsFix(bugUrl = "https://issues.apache.org/jira/browse/SOLR-9036";)
TestSegmentSorting.java
testSegmentTerminateEarly()
@Test@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestSolrCloudWithDelegationTokens.java
testDelegationTokenCancelFail()
//@AwaitsFix(bugUrl="https://issues.apache.org/jira/browse/HADOOP-14044";)
TestStressCloudBlindAtomicUpdates.java
test_dv_stored()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestTlogReplica.java
testRecovery()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestTriggerIntegration.java
testCooldown()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestTriggerIntegration.java
testListeners()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestTriggerIntegration.java
testSearchRate()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TestUtilizeNode.java
test()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TriggerIntegrationTest.java
testEventQueue()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
TriggerIntegrationTest.java
testMetricTrigger()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
UIMABaseAnalyzerTest.java
testRandomStrings()
@Test @AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/LUCENE-3869";)
UIMABaseAnalyzerTest.java
testRandomStringsWithConfigurationParameters()
@Test @AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/LUCENE-3869";)
UIMATypeAwareAnalyzerTest.java
testRandomStrings()
@Test @AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/LUCENE-3869";)
ZkControllerTest.java
testPublishAndWaitForDownStates()
@BadApple(bugUrl = "https://issues.apache.org/jira/browse/SOLR-12028";)
ZkSolrClientTest.java
testMultipleWatchesAsync()
@BadApple(bugUrl="https://issues.apache.org/jira/browse/SOLR-12028";)
> BadApple and AwaitsFix annotations usage
> ----------------------------------------
>
> Key: SOLR-12028
> URL: https://issues.apache.org/jira/browse/SOLR-12028
> Project: Solr
> Issue Type: Task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Tests
> Reporter: Erick Erickson
> Assignee: Erick Erickson
> Priority: Major
> Attachments: SOLR-12028.patch
>
>
> There's a long discussion of this topic at SOLR-12016. Here's a summary:
> - BadApple annotations are used for tests that intermittently fail, say < 30%
> of the time. Tests that fail more often shold be moved to AwaitsFix. This is,
> of course, a judgement call
> - AwaitsFix annotations are used for tests that, for some reason, the problem
> can't be fixed immediately. Likely reasons are third-party dependencies,
> extreme difficulty tracking down, dependency on another JIRA etc.
> Jenkins jobs will typically run with BadApple disabled to cut down on noise.
> Periodically Jenkins jobs will be run with BadApples enabled so BadApple
> tests won't be lost and reports can be generated. Tests that run with
> BadApples disabled that fail require _immediate_ attention.
> The default for developers is that BadApple is enabled.
> If you are working on one of these tests and cannot get the test to fail
> locally, it is perfectly acceptable to comment the annotation out. You should
> let the dev list know that this is deliberate.
> This JIRA is a placeholder for BadApple tests to point to between the times
> they're identified as BadApple and they're either fixed or changed to
> AwaitsFix or assigned their own JIRA.
> I've assigned this to myself to track so I don't lose track of it. No one
> person will fix all of these issues, this will be an ongoing technical debt
> cleanup effort.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
