[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1071 - Still Failing

Apache Jenkins Server Thu, 14 Jul 2016 07:44:40 -0700

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1071/


14 tests failed.
FAILED:  org.apache.solr.core.OpenCloseCoreStressTest.test10Minutes

Error Message:
Captured an uncaught exception in thread: Thread[id=18199, name=Thread-7430, 
state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=18199, name=Thread-7430, state=RUNNABLE, 
group=TGRP-OpenCloseCoreStressTest]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.HashSet.<init>(HashSet.java:105)
        at 
sun.security.ssl.SSLAlgorithmDecomposer.decomposes(SSLAlgorithmDecomposer.java:146)
        at 
sun.security.ssl.SSLAlgorithmDecomposer.decompose(SSLAlgorithmDecomposer.java:222)
        at 
sun.security.ssl.SSLAlgorithmDecomposer.decompose(SSLAlgorithmDecomposer.java:243)
        at 
sun.security.util.AbstractAlgorithmConstraints.checkAlgorithm(AbstractAlgorithmConstraints.java:104)
        at 
sun.security.util.DisabledAlgorithmConstraints.permits(DisabledAlgorithmConstraints.java:91)
        at 
sun.security.ssl.SSLAlgorithmConstraints.permits(SSLAlgorithmConstraints.java:144)
        at 
sun.security.ssl.Handshaker.getActiveCipherSuites(Handshaker.java:642)
        at sun.security.ssl.Handshaker.activate(Handshaker.java:509)
        at 
sun.security.ssl.SSLSocketImpl.kickstartHandshake(SSLSocketImpl.java:1482)
        at 
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1351)
        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
        at 
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
        at 
org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
        at 
org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
        at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
        at 
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
        at 
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
        at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at null
        at null
        at null
        at null
        at null
        at null
        at null
        at null


FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1468503383301,generation=2,filelist=[_2fq.fdt, 
_2fq.fdx, _2fq.fnm, _2fq.nvd, _2fq.nvm, _2fq.si, _2fq_Lucene50_0.doc, 
_2fq_Lucene50_0.tim, _2fq_Lucene50_0.tip, _3a4.cfe, _3a4.cfs, _3a4.si, 
_3ab.cfe, _3ab.cfs, _3ab.si, _3pk.cfe, _3pk.cfs, _3pk.si, _4il.cfe, _4il.cfs, 
_4il.si, _4iy.cfe, _4iy.cfs, _4iy.si, _4xk.cfe, _4xk.cfs, _4xk.si, _5cc.cfe, 
_5cc.cfs, _5cc.si, _5qy.cfe, _5qy.cfs, _5qy.si, _65r.cfe, _65r.cfs, _65r.si, 
_6k6.cfe, _6k6.cfs, _6k6.si, _6z2.cfe, _6z2.cfs, _6z2.si, _76y.fdt, _76y.fdx, 
_76y.fnm, _76y.nvd, _76y.nvm, _76y.si, _76y_Lucene50_0.doc, 
_76y_Lucene50_0.tim, _76y_Lucene50_0.tip, _76z.cfe, _76z.cfs, _76z.si, 
_770.fdt, _770.fdx, _770.fnm, _770.nvd, _770.nvm, _770.si, _770_Lucene50_0.doc, 
_770_Lucene50_0.tim, _770_Lucene50_0.tip, _771.fdt, _771.fdx, _771.fnm, 
_771.nvd, _771.nvm, _771.si, _771_Lucene50_0.doc, _771_Lucene50_0.tim, 
_771_Lucene50_0.tip, _772.fdt, _772.fdx, _772.fnm, _772.nvd, _772.nvm, _772.si, 
_772_Lucene50_0.doc, _772_Lucene50_0.tim, _772_Lucene50_0.tip, _773.fdt, 
_773.fdx, _773.fnm, _773.nvd, _773.nvm, _773.si, _773_Lucene50_0.doc, 
_773_Lucene50_0.tim, _773_Lucene50_0.tip, _774.fdt, _774.fdx, _774.fnm, 
_774.nvd, _774.nvm, _774.si, _774_Lucene50_0.doc, _774_Lucene50_0.tim, 
_774_Lucene50_0.tip, _775.fdt, _775.fdx, _775.fnm, _775.nvd, _775.nvm, _775.si, 
_775_Lucene50_0.doc, _775_Lucene50_0.tim, _775_Lucene50_0.tip, segments_2]}]> 
but was:<[{indexVersion=1468503383301,generation=2,filelist=[_2fq.fdt, 
_2fq.fdx, _2fq.fnm, _2fq.nvd, _2fq.nvm, _2fq.si, _2fq_Lucene50_0.doc, 
_2fq_Lucene50_0.tim, _2fq_Lucene50_0.tip, _3a4.cfe, _3a4.cfs, _3a4.si, 
_3ab.cfe, _3ab.cfs, _3ab.si, _3pk.cfe, _3pk.cfs, _3pk.si, _4il.cfe, _4il.cfs, 
_4il.si, _4iy.cfe, _4iy.cfs, _4iy.si, _4xk.cfe, _4xk.cfs, _4xk.si, _5cc.cfe, 
_5cc.cfs, _5cc.si, _5qy.cfe, _5qy.cfs, _5qy.si, _65r.cfe, _65r.cfs, _65r.si, 
_6k6.cfe, _6k6.cfs, _6k6.si, _6z2.cfe, _6z2.cfs, _6z2.si, _76y.fdt, _76y.fdx, 
_76y.fnm, _76y.nvd, _76y.nvm, _76y.si, _76y_Lucene50_0.doc, 
_76y_Lucene50_0.tim, _76y_Lucene50_0.tip, _76z.cfe, _76z.cfs, _76z.si, 
_770.fdt, _770.fdx, _770.fnm, _770.nvd, _770.nvm, _770.si, _770_Lucene50_0.doc, 
_770_Lucene50_0.tim, _770_Lucene50_0.tip, _771.fdt, _771.fdx, _771.fnm, 
_771.nvd, _771.nvm, _771.si, _771_Lucene50_0.doc, _771_Lucene50_0.tim, 
_771_Lucene50_0.tip, _772.fdt, _772.fdx, _772.fnm, _772.nvd, _772.nvm, _772.si, 
_772_Lucene50_0.doc, _772_Lucene50_0.tim, _772_Lucene50_0.tip, _773.fdt, 
_773.fdx, _773.fnm, _773.nvd, _773.nvm, _773.si, _773_Lucene50_0.doc, 
_773_Lucene50_0.tim, _773_Lucene50_0.tip, _774.fdt, _774.fdx, _774.fnm, 
_774.nvd, _774.nvm, _774.si, _774_Lucene50_0.doc, _774_Lucene50_0.tim, 
_774_Lucene50_0.tip, _775.fdt, _775.fdx, _775.fnm, _775.nvd, _775.nvm, _775.si, 
_775_Lucene50_0.doc, _775_Lucene50_0.tim, _775_Lucene50_0.tip, segments_2]}, 
{indexVersion=1468503383301,generation=3,filelist=[_2fq.fdt, _2fq.fdx, 
_2fq.fnm, _2fq.nvd, _2fq.nvm, _2fq.si, _2fq_Lucene50_0.doc, 
_2fq_Lucene50_0.tim, _2fq_Lucene50_0.tip, _3a4.cfe, _3a4.cfs, _3a4.si, 
_3ab.cfe, _3ab.cfs, _3ab.si, _3pk.cfe, _3pk.cfs, _3pk.si, _4il.cfe, _4il.cfs, 
_4il.si, _4iy.cfe, _4iy.cfs, _4iy.si, _4xk.cfe, _4xk.cfs, _4xk.si, _5cc.cfe, 
_5cc.cfs, _5cc.si, _5qy.cfe, _5qy.cfs, _5qy.si, _65r.cfe, _65r.cfs, _65r.si, 
_6k6.cfe, _6k6.cfs, _6k6.si, _6z2.cfe, _6z2.cfs, _6z2.si, _775.fdt, _775.fdx, 
_775.fnm, _775.nvd, _775.nvm, _775.si, _775_Lucene50_0.doc, 
_775_Lucene50_0.tim, _775_Lucene50_0.tip, _776.cfe, _776.cfs, _776.si, 
segments_3]}]>

Stack Trace:
java.lang.AssertionError: 
expected:<[{indexVersion=1468503383301,generation=2,filelist=[_2fq.fdt, 
_2fq.fdx, _2fq.fnm, _2fq.nvd, _2fq.nvm, _2fq.si, _2fq_Lucene50_0.doc, 
_2fq_Lucene50_0.tim, _2fq_Lucene50_0.tip, _3a4.cfe, _3a4.cfs, _3a4.si, 
_3ab.cfe, _3ab.cfs, _3ab.si, _3pk.cfe, _3pk.cfs, _3pk.si, _4il.cfe, _4il.cfs, 
_4il.si, _4iy.cfe, _4iy.cfs, _4iy.si, _4xk.cfe, _4xk.cfs, _4xk.si, _5cc.cfe, 
_5cc.cfs, _5cc.si, _5qy.cfe, _5qy.cfs, _5qy.si, _65r.cfe, _65r.cfs, _65r.si, 
_6k6.cfe, _6k6.cfs, _6k6.si, _6z2.cfe, _6z2.cfs, _6z2.si, _76y.fdt, _76y.fdx, 
_76y.fnm, _76y.nvd, _76y.nvm, _76y.si, _76y_Lucene50_0.doc, 
_76y_Lucene50_0.tim, _76y_Lucene50_0.tip, _76z.cfe, _76z.cfs, _76z.si, 
_770.fdt, _770.fdx, _770.fnm, _770.nvd, _770.nvm, _770.si, _770_Lucene50_0.doc, 
_770_Lucene50_0.tim, _770_Lucene50_0.tip, _771.fdt, _771.fdx, _771.fnm, 
_771.nvd, _771.nvm, _771.si, _771_Lucene50_0.doc, _771_Lucene50_0.tim, 
_771_Lucene50_0.tip, _772.fdt, _772.fdx, _772.fnm, _772.nvd, _772.nvm, _772.si, 
_772_Lucene50_0.doc, _772_Lucene50_0.tim, _772_Lucene50_0.tip, _773.fdt, 
_773.fdx, _773.fnm, _773.nvd, _773.nvm, _773.si, _773_Lucene50_0.doc, 
_773_Lucene50_0.tim, _773_Lucene50_0.tip, _774.fdt, _774.fdx, _774.fnm, 
_774.nvd, _774.nvm, _774.si, _774_Lucene50_0.doc, _774_Lucene50_0.tim, 
_774_Lucene50_0.tip, _775.fdt, _775.fdx, _775.fnm, _775.nvd, _775.nvm, _775.si, 
_775_Lucene50_0.doc, _775_Lucene50_0.tim, _775_Lucene50_0.tip, segments_2]}]> 
but was:<[{indexVersion=1468503383301,generation=2,filelist=[_2fq.fdt, 
_2fq.fdx, _2fq.fnm, _2fq.nvd, _2fq.nvm, _2fq.si, _2fq_Lucene50_0.doc, 
_2fq_Lucene50_0.tim, _2fq_Lucene50_0.tip, _3a4.cfe, _3a4.cfs, _3a4.si, 
_3ab.cfe, _3ab.cfs, _3ab.si, _3pk.cfe, _3pk.cfs, _3pk.si, _4il.cfe, _4il.cfs, 
_4il.si, _4iy.cfe, _4iy.cfs, _4iy.si, _4xk.cfe, _4xk.cfs, _4xk.si, _5cc.cfe, 
_5cc.cfs, _5cc.si, _5qy.cfe, _5qy.cfs, _5qy.si, _65r.cfe, _65r.cfs, _65r.si, 
_6k6.cfe, _6k6.cfs, _6k6.si, _6z2.cfe, _6z2.cfs, _6z2.si, _76y.fdt, _76y.fdx, 
_76y.fnm, _76y.nvd, _76y.nvm, _76y.si, _76y_Lucene50_0.doc, 
_76y_Lucene50_0.tim, _76y_Lucene50_0.tip, _76z.cfe, _76z.cfs, _76z.si, 
_770.fdt, _770.fdx, _770.fnm, _770.nvd, _770.nvm, _770.si, _770_Lucene50_0.doc, 
_770_Lucene50_0.tim, _770_Lucene50_0.tip, _771.fdt, _771.fdx, _771.fnm, 
_771.nvd, _771.nvm, _771.si, _771_Lucene50_0.doc, _771_Lucene50_0.tim, 
_771_Lucene50_0.tip, _772.fdt, _772.fdx, _772.fnm, _772.nvd, _772.nvm, _772.si, 
_772_Lucene50_0.doc, _772_Lucene50_0.tim, _772_Lucene50_0.tip, _773.fdt, 
_773.fdx, _773.fnm, _773.nvd, _773.nvm, _773.si, _773_Lucene50_0.doc, 
_773_Lucene50_0.tim, _773_Lucene50_0.tip, _774.fdt, _774.fdx, _774.fnm, 
_774.nvd, _774.nvm, _774.si, _774_Lucene50_0.doc, _774_Lucene50_0.tim, 
_774_Lucene50_0.tip, _775.fdt, _775.fdx, _775.fnm, _775.nvd, _775.nvm, _775.si, 
_775_Lucene50_0.doc, _775_Lucene50_0.tim, _775_Lucene50_0.tip, segments_2]}, 
{indexVersion=1468503383301,generation=3,filelist=[_2fq.fdt, _2fq.fdx, 
_2fq.fnm, _2fq.nvd, _2fq.nvm, _2fq.si, _2fq_Lucene50_0.doc, 
_2fq_Lucene50_0.tim, _2fq_Lucene50_0.tip, _3a4.cfe, _3a4.cfs, _3a4.si, 
_3ab.cfe, _3ab.cfs, _3ab.si, _3pk.cfe, _3pk.cfs, _3pk.si, _4il.cfe, _4il.cfs, 
_4il.si, _4iy.cfe, _4iy.cfs, _4iy.si, _4xk.cfe, _4xk.cfs, _4xk.si, _5cc.cfe, 
_5cc.cfs, _5cc.si, _5qy.cfe, _5qy.cfs, _5qy.si, _65r.cfe, _65r.cfs, _65r.si, 
_6k6.cfe, _6k6.cfs, _6k6.si, _6z2.cfe, _6z2.cfs, _6z2.si, _775.fdt, _775.fdx, 
_775.fnm, _775.nvd, _775.nvm, _775.si, _775_Lucene50_0.doc, 
_775_Lucene50_0.tim, _775_Lucene50_0.tip, _776.cfe, _776.cfs, _776.si, 
segments_3]}]>
        at 
__randomizedtesting.SeedInfo.seed([7C54E247A2318917:5983F977D2798714]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:147)
        at 
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1225)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestManagedSchemaAPI

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.schema.TestManagedSchemaAPI:     1) Thread[id=198887, 
name=OverseerHdfsCoreFailoverThread-96239999358599173-127.0.0.1:60468_solr-n_0000000001,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.schema.TestManagedSchemaAPI: 
   1) Thread[id=198887, 
name=OverseerHdfsCoreFailoverThread-96239999358599173-127.0.0.1:60468_solr-n_0000000001,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestManagedSchemaAPI

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=198887, 
name=OverseerHdfsCoreFailoverThread-96239999358599173-127.0.0.1:60468_solr-n_0000000001,
 state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=198887, 
name=OverseerHdfsCoreFailoverThread-96239999358599173-127.0.0.1:60468_solr-n_0000000001,
 state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: 
https://127.0.0.1:51235/_n/bq

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: https://127.0.0.1:51235/_n/bq
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:617)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:400)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:458)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:180)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:170)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
        at sun.security.ssl.InputRecord.read(InputRecord.java:503)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
        at 
org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
        at 
org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
        at 
org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at 
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
        at 
org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
        at 
org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
        at 
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
        at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
        at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
        at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:511)
        ... 47 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CdcrRequestHandlerTest

Error Message:
ObjectTracker found 8 object(s) that were not released!!! [InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 8 object(s) that were not 
released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient]
        at __randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
        at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CdcrRequestHandlerTest

Error Message:
8 threads leaked from SUITE scope at 
org.apache.solr.cloud.CdcrRequestHandlerTest:     1) Thread[id=76081, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=76731, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=76080, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=76732, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=76734, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=76079, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=76733, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=76078, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE 
scope at org.apache.solr.cloud.CdcrRequestHandlerTest: 
   1) Thread[id=76081, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=76731, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=76080, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=76732, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=76734, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=76079, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=76733, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=76078, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrRequestHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest

Error Message:
ObjectTracker found 10 object(s) that were not released!!! [InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 10 object(s) that were not 
released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient]
        at __randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
        at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest

Error Message:
10 threads leaked from SUITE scope at 
org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest:     1) 
Thread[id=77309, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=77334, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=77354, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=77324, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=77349, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=77319, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=77329, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=77339, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=77314, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=77344, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from 
SUITE scope at org.apache.solr.cloud.ConcurrentDeleteAndCreateCollectionTest: 
   1) Thread[id=77309, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=77334, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=77354, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=77324, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=77349, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=77319, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=77329, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=77339, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=77314, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=77344, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-ConcurrentDeleteAndCreateCollectionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
replicaCount expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: replicaCount expected:<0> but was:<1>
        at 
__randomizedtesting.SeedInfo.seed([7C54E247A2318917:F400DD9D0CCDE4EF]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testNoConfigSetExist(CollectionsAPIDistributedZkTest.java:619)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:177)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrReplicationHandlerTest

Error Message:
ObjectTracker found 12 object(s) that were not released!!! [InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 12 object(s) that were not 
released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient]
        at __randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
        at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrReplicationHandlerTest

Error Message:
12 threads leaked from SUITE scope at 
org.apache.solr.cloud.CdcrReplicationHandlerTest:     1) Thread[id=75065, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=74181, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=74800, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=75704, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=75705, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=74572, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=74449, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=74573, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=75066, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=74182, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=74448, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=74801, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]         at java.lang.Thread.sleep(Native 
Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 12 threads leaked from 
SUITE scope at org.apache.solr.cloud.CdcrReplicationHandlerTest: 
   1) Thread[id=75065, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=74181, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=74800, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=75704, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=75705, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=74572, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=74449, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=74573, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=75066, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=74182, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=74448, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=74801, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationHandlerTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrReplicationDistributedZkTest

Error Message:
ObjectTracker found 48 object(s) that were not released!!! [InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 48 object(s) that were not 
released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient]
        at __randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrReplicationDistributedZkTest

Error Message:
48 threads leaked from SUITE scope at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest:     1) Thread[id=32446, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=33973, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=34427, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=34673, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=31796, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=31797, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=27085, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=30623, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=33974, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=32206, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=33976, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=30622, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=34674, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=27554, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=29258, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=29257, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=30621, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=27083, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=34672, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=34426, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=31798, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=34250, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=27552, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=32445, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=34671, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=27082, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=33975, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=34425, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=29259, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=27556, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=32067, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=27553, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=32443, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=32068, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=34249, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=34424, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=32207, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=34251, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=27084, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=29260, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=32444, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=30624, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=32066, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   44) Thread[id=32069, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=32205, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=31795, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=32208, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=34252, 
name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 48 threads leaked from 
SUITE scope at org.apache.solr.cloud.CdcrReplicationDistributedZkTest: 
   1) Thread[id=32446, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=33973, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=34427, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=34673, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=31796, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=31797, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=27085, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=30623, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=33974, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=32206, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=33976, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=30622, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=34674, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  14) Thread[id=27554, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  15) Thread[id=29258, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  16) Thread[id=29257, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  17) Thread[id=30621, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  18) Thread[id=27083, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  19) Thread[id=34672, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  20) Thread[id=34426, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  21) Thread[id=31798, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  22) Thread[id=34250, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  23) Thread[id=27552, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  24) Thread[id=32445, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  25) Thread[id=34671, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  26) Thread[id=27082, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  27) Thread[id=33975, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  28) Thread[id=34425, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  29) Thread[id=29259, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  30) Thread[id=27556, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  31) Thread[id=32067, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  32) Thread[id=27553, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  33) Thread[id=32443, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  34) Thread[id=32068, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  35) Thread[id=34249, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  36) Thread[id=34424, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  37) Thread[id=32207, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  38) Thread[id=34251, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  39) Thread[id=27084, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  40) Thread[id=29260, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  41) Thread[id=32444, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:745)
  42) Thread[id=30624, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-CdcrReplicationDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.la

[...truncated too long message...]

iloverThread.java:137)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
{_version_=FSTOrd50, id=FST50, myNewField1=FST50, myNewField=FSTOrd50}, 
docValues:{}, maxPointsInLeafNode=926, maxMBSortInHeap=6.395709420601184, 
sim=ClassicSimilarity, locale=it-IT, timezone=SystemV/YST9
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_74 (64-bit)/cpus=4,threads=2,free=83499408,total=523239424
   [junit4]   2> NOTE: All tests run in this JVM: [ConnectionReuseTest, 
TestConfig, DistributedDebugComponentTest, TestLazyCores, JavabinLoaderTest, 
RemoteQueryErrorTest, EchoParamsTest, TestSearcherReuse, 
TestBlendedInfixSuggestions, ZkNodePropsTest, OutputWriterTest, 
MultiThreadedOCPTest, TestStressLucene, TestFieldSortValues, 
DefaultValueUpdateProcessorTest, BasicAuthIntegrationTest, 
TestTolerantUpdateProcessorRandomCloud, DistributedSuggestComponentTest, 
HdfsChaosMonkeyNothingIsSafeTest, CachingDirectoryFactoryTest, TestMacros, 
CloudExitableDirectoryReaderTest, TestConfigSets, 
DistributedFacetPivotLongTailTest, PeerSyncTest, TestFaceting, 
QueryEqualityTest, TestBinaryField, TestRemoteStreaming, RollingRestartTest, 
RequiredFieldsTest, HLLUtilTest, TestLockTree, TestLRUCache, 
DistribJoinFromCollectionTest, TestDownShardTolerantSearch, 
AlternateDirectoryTest, TestManagedStopFilterFactory, TestQueryWrapperFilter, 
HdfsRestartWhileUpdatingTest, AnalysisAfterCoreReloadTest, JSONWriterTest, 
TestCloudInspectUtil, DeleteStatusTest, StandardRequestHandlerTest, 
DistributedQueryElevationComponentTest, TestDistributedSearch, 
TestSortByMinMaxFunction, CoreAdminCreateDiscoverTest, TestRTGBase, 
TestJsonFacets, TestFieldTypeResource, TestHdfsUpdateLog, 
TestInfoStreamLogging, JsonLoaderTest, TestIBSimilarityFactory, 
TestPhraseSuggestions, TestSolr4Spatial, RulesTest, TestUseDocValuesAsStored2, 
PrimUtilsTest, ResponseLogComponentTest, TestCollectionAPI, AddBlockUpdateTest, 
CreateCollectionCleanupTest, DataDrivenBlockJoinTest, 
ExternalFileFieldSortTest, SyncSliceTest, SortSpecParsingTest, 
SuggestComponentContextFilterQueryTest, BigEndianAscendingWordDeserializerTest, 
DistributedFacetPivotWhiteBoxTest, ForceLeaderTest, TestRandomDVFaceting, 
TestAtomicUpdateErrorCases, OverseerCollectionConfigSetProcessorTest, 
TestSolr4Spatial2, LeaderInitiatedRecoveryOnCommitTest, 
WrapperMergePolicyFactoryTest, ActionThrottleTest, TestFieldCacheSanityChecker, 
TestMergePolicyConfig, TestComponentsName, SystemInfoHandlerTest, 
UUIDUpdateProcessorFallbackTest, TestCrossCoreJoin, TestScoreJoinQPNoScore, 
IndexSchemaTest, CollectionTooManyReplicasTest, TestDFISimilarityFactory, 
TestSolrQueryResponse, TestSolrDynamicMBean, TestRandomRequestDistribution, 
TestHashQParserPlugin, TestSolrQueryParserDefaultOperatorResource, 
TestLuceneMatchVersion, TestRequestForwarding, TestLeaderElectionZkExpiry, 
TestHighFrequencyDictionaryFactory, TestNumericTerms32, TestBadConfig, 
TestDefaultSearchFieldResource, LeaderElectionIntegrationTest, 
ShardRoutingTest, BasicZkTest, FullSolrCloudDistribCmdsTest, 
ClusterStateUpdateTest, TestRandomFaceting, ShardRoutingCustomTest, 
TestRecovery, DistributedSpellCheckComponentTest, 
TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, 
TestStressReorder, TestJoin, TestReload, TestRangeQuery, SolrCoreTest, 
TestGroupingSearch, SolrCmdDistributorTest, QueryElevationComponentTest, 
BadIndexSchemaTest, ConvertedLegacyTest, TestFiltering, TestFunctionQuery, 
HighlighterTest, CurrencyFieldXmlFileTest, SuggesterFSTTest, 
SolrRequestParserTest, TestFoldingMultitermQuery, TestTrie, SpatialFilterTest, 
TestCSVLoader, PolyFieldTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, 
SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, 
FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, 
SortByFunctionTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, 
CacheHeaderTest, LukeRequestHandlerTest, DocumentAnalysisRequestHandlerTest, 
FileBasedSpellCheckerTest, RequestHandlersTest, 
PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, 
TestArbitraryIndexDir, LoggingHandlerTest, ReturnFieldsTest, QueryParsingTest, 
UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, 
CSVRequestHandlerTest, SearchHandlerTest, CopyFieldTest, BadComponentTest, 
TestSolrDeletionPolicy2, SampleTest, TestFuzzyAnalyzedSuggestions, 
NotRequiredUniqueKeyTest, TestCodecSupport, TimeZoneUtilsTest, 
ResourceLoaderTest, ScriptEngineTest, PluginInfoTest, TestFastLRUCache, 
PreAnalyzedFieldTest, TestSystemIdResolver, TestSuggestSpellingConverter, 
DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, 
RAMDirectoryFactoryTest, TestSolrJ, TestDocumentBuilder, UUIDFieldTest, 
FileUtilsTest, DistributedMLTComponentTest, CursorPagingTest, 
DistributedIntervalFacetingTest, AsyncCallRequestStatusResponseTest, 
CdcrReplicationHandlerTest, CdcrRequestHandlerTest, CdcrVersionReplicationTest, 
ConcurrentDeleteAndCreateCollectionTest, ConnectionManagerTest, 
CustomCollectionTest, LeaderFailoverAfterPartitionTest, MigrateRouteKeyTest, 
RestartWhileUpdatingTest, SharedFSAutoReplicaFailoverTest, 
TestOnReconnectListenerSupport, TestReplicaProperties, TestSSLRandomization, 
TestShortCircuitedRequests, TestStressLiveNodes, HdfsNNFailoverTest, 
HdfsRecoverLeaseTest, HdfsRecoveryZkTest, 
HdfsTlogReplayBufferedWhileIndexingTest, TestCollationFieldDocValues, 
TestManagedSchema, TestManagedSchemaAPI]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestManagedSchemaAPI -Dtests.seed=7C54E247A2318917 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=it-IT -Dtests.timezone=SystemV/YST9 -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | TestManagedSchemaAPI (suite) <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.schema.TestManagedSchemaAPI: 
   [junit4]    >    1) Thread[id=198887, 
name=OverseerHdfsCoreFailoverThread-96239999358599173-127.0.0.1:60468_solr-n_0000000001,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=198887, 
name=OverseerHdfsCoreFailoverThread-96239999358599173-127.0.0.1:60468_solr-n_0000000001,
 state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([7C54E247A2318917]:0)
   [junit4] Completed [561/620 (9!)] on J0 in 36.75s, 1 test, 2 errors <<< 
FAILURES!

[...truncated 170 lines...]
   [junit4] JVM J1: stdout was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/temp/junit4-J1-20160714_124327_506.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/heapdumps/java_pid10109.hprof
 ...
   [junit4] Heap dump file created [549647964 bytes in 4.304 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 10 lines...]
   [junit4] JVM J2: stdout was not empty, see: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build/solr-core/test/temp/junit4-J2-20160714_124327_506.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/heapdumps/java_pid10112.hprof
 ...
   [junit4] Heap dump file created [651089263 bytes in 10.533 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 18 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/build.xml:770:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/build.xml:707:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/build.xml:59:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/solr/common-build.xml:530:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/lucene/common-build.xml:1443:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/lucene/common-build.xml:984:
 There were test failures: 620 suites (10 ignored), 2635 tests, 10 suite-level 
errors, 2 errors, 2 failures, 318 ignored (30 assumptions) [seed: 
7C54E247A2318917]

Total time: 318 minutes 1 second
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1071 - Still Failing

Reply via email to