Very useful analysis. Will look into a few of them. On Tue, Jan 12, 2016 at 12:07 AM, Mark Miller <[email protected]> wrote:
> Here is the same data split up between trunk and 5x. > > trunk: > Test: org.apache.solr.cloud.HttpPartitionTest.test Fails: 23 > Test: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test Fails: 15 > Test: org.apache.solr.cloud.SaslZkACLProviderTest Fails: 12 > Test: org.apache.solr.cloud.CollectionsAPIDistributedZkTest Fails: 12 > Test: org.apache.solr.util.TestSolrCLIRunExample Fails: 6 > Test: org.apache.solr.cloud.SaslZkACLProviderTest.testSaslZkACLProvider > Fails: 6 > Test: org.apache.solr.core.TestLazyCores Fails: 6 > Test: org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test > Fails: 5 > Test: org.apache.solr.client.solrj.impl.CloudSolrClientTest Fails: 5 > Test: org.apache.lucene.index.TestDuelingCodecs.testCrazyReaderEquals > Fails: 4 > Test: > org.apache.lucene.replicator.http.HttpReplicatorTest.testServerErrors > Fails: 4 > Test: org.apache.lucene.index.TestDuelingCodecs.testEquals Fails: 4 > Test: org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic > Fails: 4 > Test: org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir > Fails: 4 > Test: org.apache.solr.client.solrj.impl.CloudSolrClientTest.test Fails: 4 > Test: > org.apache.solr.handler.TestReplicationHandler.doTestStressReplication > Fails: 3 > Test: org.apache.solr.search.AnalyticsMergeStrategyTest.test Fails: 3 > Test: org.apache.solr.handler.TestReplicationHandler Fails: 3 > Test: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test Fails: 3 > Test: org.apache.solr.cloud.CustomCollectionTest.test Fails: 3 > Test: org.apache.lucene.search.suggest.TestInputIterator.testTerms Fails: 2 > Test: org.apache.solr.morphlines.solr.SolrMorphlineZkTest Fails: 2 > Test: org.apache.solr.update.AutoCommitTest.testMaxTime Fails: 2 > Test: > org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication > Fails: 2 > Test: org.apache.solr.cloud.UnloadDistributedZkTest.test Fails: 1 > Test: > org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.testFullImportFieldsParam > Fails: 1 > Test: org.apache.solr.rest.schema.TestSchemaNameResource.testGetSchemaName > Fails: 1 > Test: org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.test Fails: > 1 > Test: > org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.testFullImportFqParam > Fails: 1 > Test: org.apache.solr.cloud.RecoveryZkTest.test Fails: 1 > Test: org.apache.solr.schema.TestCloudSchemaless Fails: 1 > Test: org.apache.solr.schema.TestCloudSchemaless.test Fails: 1 > Test: > org.apache.solr.handler.dataimport.TestContentStreamDataSource.testCommitWithin > Fails: 1 > Test: > org.apache.lucene.search.suggest.analyzing.FuzzySuggesterTest.testKeyword > Fails: 1 > Test: org.apache.solr.core.TestDynamicLoading.testDynamicLoading Fails: 1 > Test: > org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.testFullImportInnerEntity > Fails: 1 > Test: org.apache.solr.morphlines.solr.SolrMorphlineZkTest.test Fails: 1 > Test: > org.apache.solr.util.TestSolrCLIRunExample.testInteractiveSolrCloudExample > Fails: 1 > Test: > org.apache.solr.handler.PingRequestHandlerTest.testPingInClusterWithNoHealthCheck > Fails: 1 > Test: org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test Fails: 1 > Test: org.apache.solr.handler.TestSolrConfigHandlerCloud Fails: 1 > Test: org.apache.solr.cloud.TestCloudPivotFacet Fails: 1 > Test: org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup > Fails: 1 > Test: > org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy > Fails: 1 > Test: org.apache.solr.cloud.SyncSliceTest.test Fails: 1 > Test: org.apache.solr.cloud.BasicDistributedZkTest.test Fails: 1 > Test: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test Fails: 1 > Test: org.apache.solr.handler.component.DistributedDebugComponentTest > Fails: 1 > Test: > org.apache.solr.client.solrj.impl.CloudSolrClientTest.testWrongZkChrootTest > Fails: 1 > Test: > org.apache.lucene.search.suggest.analyzing.AnalyzingSuggesterTest.testKeyword > Fails: 1 > Test: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test Fails: 1 > Test: > org.apache.solr.handler.dataimport.TestContentStreamDataSource.testSimple > Fails: 1 > Test: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest Fails: 1 > Test: > org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.testFullImport > Fails: 1 > Test: org.apache.solr.cloud.UnloadDistributedZkTest Fails: 1 > Test: org.apache.solr.cloud.TestCloudPivotFacet.test Fails: 1 > Test: org.apache.solr.cloud.TestDistribDocBasedVersion.test Fails: 1 > > > 5x Branch: > Test: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test Fails: 12 > Test: org.apache.solr.cloud.ShardSplitTest.test Fails: 10 > Test: org.apache.solr.cloud.CollectionsAPIDistributedZkTest Fails: 10 > Test: org.apache.solr.handler.TestReplicationHandler Fails: 8 > Test: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test Fails: 8 > Test: org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics Fails: > 7 > Test: org.apache.solr.cloud.UnloadDistributedZkTest.test Fails: 6 > Test: org.apache.solr.cloud.ShardRoutingTest.test Fails: 6 > Test: > org.apache.solr.cloud.TestAuthenticationFramework.testCollectionCreateSearchDelete > Fails: 6 > Test: org.apache.solr.core.TestCoreDiscovery Fails: 6 > Test: org.apache.solr.core.TestLazyCores Fails: 6 > Test: org.apache.solr.cloud.TestAuthenticationFramework.testBasics Fails: 6 > Test: org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test > Fails: 6 > Test: org.apache.solr.cloud.HttpPartitionTest.test Fails: 6 > Test: org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader > Fails: 6 > Test: org.apache.solr.cloud.TestDistribDocBasedVersion.test Fails: 6 > Test: org.apache.solr.cloud.RecoveryAfterSoftCommitTest.test Fails: 5 > Test: org.apache.solr.cloud.DistribJoinFromCollectionTest.test Fails: 5 > Test: org.apache.solr.cloud.CustomCollectionTest.test Fails: 5 > Test: org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test Fails: 4 > Test: org.apache.solr.handler.TestSolrConfigHandlerCloud Fails: 4 > Test: org.apache.solr.update.processor.TestNamedUpdateProcessors.test > Fails: 4 > Test: org.apache.solr.cloud.SyncSliceTest.test Fails: 4 > Test: org.apache.solr.cloud.BasicDistributedZkTest.test Fails: 4 > Test: org.apache.solr.cloud.BasicDistributedZk2Test.test Fails: 4 > Test: org.apache.solr.cloud.DistributedVersionInfoTest.test Fails: 4 > Test: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test Fails: 3 > Test: org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir > Fails: 3 > Test: org.apache.solr.update.AutoCommitTest.testMaxDocs Fails: 3 > Test: > org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication > Fails: 2 > Test: org.apache.solr.schema.TestCloudSchemaless.test Fails: 2 > Test: org.apache.solr.cloud.SaslZkACLProviderTest Fails: 2 > Test: > org.apache.solr.cloud.TestAuthenticationFramework.testStopAllStartAll > Fails: 2 > Test: org.apache.solr.security.BasicAuthIntegrationTest.testBasics Fails: 2 > Test: > org.apache.solr.handler.TestReplicationHandler.doTestStressReplication > Fails: 2 > Test: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test Fails: 2 > Test: org.apache.solr.cloud.hdfs.StressHdfsTest.test Fails: 1 > Test: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest Fails: 1 > Test: > org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields > Fails: 1 > Test: org.apache.solr.core.TestDynamicLoading.testDynamicLoading Fails: 1 > Test: org.apache.solr.search.stats.TestDistribIDF.testMultiCollectionQuery > Fails: 1 > Test: org.apache.solr.util.TestSolrCLIRunExample Fails: 1 > Test: > org.apache.solr.handler.PingRequestHandlerTest.testPingInClusterWithNoHealthCheck > Fails: 1 > Test: org.apache.solr.cloud.SaslZkACLProviderTest.testSaslZkACLProvider > Fails: 1 > Test: org.apache.solr.cloud.TestRandomRequestDistribution.test Fails: 1 > Test: org.apache.solr.cloud.CleanupOldIndexTest.test Fails: 1 > Test: > org.apache.lucene.analysis.cn.smart.TestSmartChineseAnalyzer.testRandomHugeStrings > Fails: 1 > Test: > org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy > Fails: 1 > Test: org.apache.solr.cloud.MigrateRouteKeyTest Fails: 1 > Test: org.apache.solr.cloud.TestMiniSolrCloudCluster.testStopAllStartAll > Fails: 1 > Test: > org.apache.solr.security.TestAuthorizationFramework.authorizationFrameworkTest > Fails: 1 > Test: org.apache.solr.cloud.MultiThreadedOCPTest.test Fails: 1 > Test: org.apache.solr.update.AutoCommitTest.testCommitWithin Fails: 1 > Test: > org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication > Fails: 1 > Test: org.apache.solr.TestDistributedSearch.test Fails: 1 > Test: org.apache.solr.cloud.HttpPartitionTest Fails: 1 > > On Mon, Jan 11, 2016 at 1:13 PM Mark Miller <[email protected]> wrote: > >> I've collected some numbers for test fails for our Jenkins cluster for >> January. I excluded counts when tons of tests fail. >> >> Test: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test Fails: 38 >> Test: org.apache.solr.cloud.HttpPartitionTest.test Fails: 29 >> Test: org.apache.solr.cloud.CollectionsAPIDistributedZkTest Fails: 24 >> Test: org.apache.solr.cloud.SaslZkACLProviderTest Fails: 14 >> Test: org.apache.solr.core.TestLazyCores Fails: 12 >> Test: org.apache.solr.cloud.ShardSplitTest.test Fails: 11 >> Test: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test Fails: >> 11 >> Test: org.apache.solr.cloud.DistribDocExpirationUpdateProcessorTest.test >> Fails: 11 >> Test: org.apache.solr.handler.TestReplicationHandler Fails: 11 >> Test: org.apache.solr.cloud.CustomCollectionTest.test Fails: 8 >> Test: org.apache.solr.cloud.UnloadDistributedZkTest.test Fails: 7 >> Test: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test >> Fails: 7 >> Test: org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics >> Fails: 7 >> Test: org.apache.solr.util.TestSolrCLIRunExample Fails: 7 >> Test: org.apache.solr.cloud.SaslZkACLProviderTest.testSaslZkACLProvider >> Fails: 7 >> Test: org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir >> Fails: 7 >> Test: org.apache.solr.cloud.TestDistribDocBasedVersion.test Fails: 7 >> Test: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test Fails: 7 >> Test: org.apache.solr.cloud.TestAuthenticationFramework.testBasics Fails: >> 7 >> Test: org.apache.solr.cloud.ShardRoutingTest.test Fails: 6 >> Test: >> org.apache.solr.cloud.TestAuthenticationFramework.testCollectionCreateSearchDelete >> Fails: 6 >> Test: org.apache.solr.core.TestCoreDiscovery Fails: 6 >> Test: >> org.apache.solr.handler.TestReplicationHandler.doTestStressReplication >> Fails: 6 >> Test: org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader >> Fails: 6 >> Test: org.apache.solr.cloud.RecoveryAfterSoftCommitTest.test Fails: 5 >> Test: org.apache.solr.cloud.DistribJoinFromCollectionTest.test Fails: 5 >> Test: org.apache.solr.cloud.SyncSliceTest.test Fails: 5 >> Test: org.apache.solr.client.solrj.impl.CloudSolrClientTest.test Fails: 5 >> Test: org.apache.solr.client.solrj.impl.CloudSolrClientTest Fails: 5 >> Test: org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test Fails: 5 >> Test: org.apache.solr.handler.TestSolrConfigHandlerCloud Fails: 5 >> Test: org.apache.solr.cloud.BasicDistributedZkTest.test Fails: 5 >> Test: org.apache.solr.cloud.BasicDistributedZk2Test.test Fails: 5 >> Test: >> org.apache.lucene.replicator.http.HttpReplicatorTest.testServerErrors >> Fails: 4 >> Test: org.apache.solr.update.processor.TestNamedUpdateProcessors.test >> Fails: 4 >> Test: org.apache.lucene.index.TestDuelingCodecs.testEquals Fails: 4 >> Test: org.apache.solr.cloud.DistributedVersionInfoTest.test Fails: 4 >> Test: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest Fails: 4 >> Test: org.apache.lucene.index.TestDuelingCodecs.testCrazyReaderEquals >> Fails: 4 >> Test: org.apache.lucene.replicator.http.HttpReplicatorTest.testBasic >> Fails: 4 >> Test: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test Fails: 4 >> Test: >> org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication >> Fails: 4 >> Test: org.apache.solr.schema.TestCloudSchemaless.test Fails: 3 >> Test: org.apache.solr.cloud.SolrCloudExampleTest Fails: 3 >> Test: org.apache.solr.cloud.HttpPartitionTest Fails: 3 >> Test: org.apache.solr.search.AnalyticsMergeStrategyTest.test Fails: 3 >> Test: org.apache.solr.update.AutoCommitTest.testMaxDocs Fails: 3 >> Test: org.apache.solr.cloud.hdfs.StressHdfsTest.test Fails: 2 >> Test: >> org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication >> Fails: 2 >> Test: >> org.apache.solr.util.TestSolrCLIRunExample.testInteractiveSolrCloudExample >> Fails: 2 >> Test: org.apache.solr.security.BasicAuthIntegrationTest.testBasics Fails: >> 2 >> Test: org.apache.solr.morphlines.solr.SolrMorphlineZkTest Fails: 2 >> Test: >> org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR >> Fails: 2 >> Test: org.apache.solr.update.AutoCommitTest.testMaxTime Fails: 2 >> Test: >> org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig >> Fails: 2 >> Test: >> org.apache.solr.cloud.TestAuthenticationFramework.testStopAllStartAll >> Fails: 2 >> Test: org.apache.solr.core.TestDynamicLoading.testDynamicLoading Fails: 2 >> Test: org.apache.lucene.search.suggest.TestInputIterator.testTerms Fails: >> 2 >> Test: >> org.apache.solr.handler.PingRequestHandlerTest.testPingInClusterWithNoHealthCheck >> Fails: 2 >> Test: >> org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy >> Fails: 2 >> Test: >> org.apache.solr.rest.schema.TestSchemaNameResource.testGetSchemaName Fails: >> 1 >> Test: >> org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.testFullImportFqParam >> Fails: 1 >> Test: org.apache.solr.cloud.RecoveryZkTest.test Fails: 1 >> Test: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test Fails: 1 >> Test: org.apache.solr.schema.TestCloudSchemaless Fails: 1 >> Test: >> org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields >> Fails: 1 >> Test: >> org.apache.solr.handler.dataimport.TestContentStreamDataSource.testCommitWithin >> Fails: 1 >> Test: >> org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.testFullImportInnerEntity >> Fails: 1 >> Test: org.apache.solr.morphlines.solr.SolrMorphlineZkTest.test Fails: 1 >> Test: org.apache.lucene.util.TestGeoUtils.testGeoRelations Fails: 1 >> Test: org.apache.lucene.codecs.asserting.TestAssertingStoredFieldsFormat >> Fails: 1 >> Test: >> org.apache.lucene.codecs.asserting.TestAssertingTermVectorsFormat.testRamBytesUsed >> Fails: 1 >> Test: org.apache.solr.cloud.TestCloudPivotFacet Fails: 1 >> Test: org.apache.lucene.codecs.asserting.TestAssertingTermVectorsFormat >> Fails: 1 >> Test: org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup >> Fails: 1 >> Test: org.apache.solr.index.hdfs.CheckHdfsIndexTest.doTest Fails: 1 >> Test: org.apache.solr.cloud.TestRandomRequestDistribution.test Fails: 1 >> Test: >> org.apache.lucene.analysis.cn.smart.TestSmartChineseAnalyzer.testRandomHugeStrings >> Fails: 1 >> Test: org.apache.solr.cloud.MigrateRouteKeyTest Fails: 1 >> Test: org.apache.solr.cloud.TestMiniSolrCloudCluster.testStopAllStartAll >> Fails: 1 >> Test: >> org.apache.solr.security.TestAuthorizationFramework.authorizationFrameworkTest >> Fails: 1 >> Test: org.apache.solr.handler.component.DistributedDebugComponentTest >> Fails: 1 >> Test: >> org.apache.lucene.codecs.asserting.TestAssertingStoredFieldsFormat.testRamBytesUsed >> Fails: 1 >> Test: >> org.apache.solr.handler.dataimport.TestContentStreamDataSource.testSimple >> Fails: 1 >> Test: >> org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.testFullImport >> Fails: 1 >> Test: org.apache.solr.cloud.TestCloudPivotFacet.test Fails: 1 >> Test: >> org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.testFullImportFieldsParam >> Fails: 1 >> Test: org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.test >> Fails: 1 >> Test: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest Fails: 1 >> Test: org.apache.lucene.util.TestOfflineSorter.testSingleLine Fails: 1 >> Test: >> org.apache.lucene.search.suggest.analyzing.FuzzySuggesterTest.testKeyword >> Fails: 1 >> Test: >> org.apache.solr.search.stats.TestDistribIDF.testMultiCollectionQuery Fails: >> 1 >> Test: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test Fails: 1 >> Test: >> org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload >> Fails: 1 >> Test: org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest Fails: 1 >> Test: org.apache.lucene.index.TestDuelingCodecsAtNight.testBigEquals >> Fails: 1 >> Test: org.apache.solr.cloud.CleanupOldIndexTest.test Fails: 1 >> Test: org.apache.solr.cloud.MultiThreadedOCPTest.test Fails: 1 >> Test: >> org.apache.solr.client.solrj.impl.CloudSolrClientTest.testWrongZkChrootTest >> Fails: 1 >> Test: >> org.apache.lucene.search.suggest.analyzing.AnalyzingSuggesterTest.testKeyword >> Fails: 1 >> Test: org.apache.solr.update.AutoCommitTest.testCommitWithin Fails: 1 >> Test: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test >> Fails: 1 >> Test: org.apache.lucene.util.TestOfflineSorter.testThreadSafety Fails: 1 >> Test: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest Fails: 1 >> Test: org.apache.solr.TestDistributedSearch.test Fails: 1 >> Test: org.apache.solr.cloud.UnloadDistributedZkTest Fails: 1 >> >> -- >> - Mark >> about.me/markrmiller >> > -- > - Mark > about.me/markrmiller >
