Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/12499/
Java: 64bit/jdk1.8.0_45 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestMiniSolrCloudCluster

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([E4D72758D4844D12]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestQuerySenderListener

Error Message:
Some resources were not closed, shutdown, or released.

Stack Trace:
java.lang.AssertionError: Some resources were not closed, shutdown, or released.
        at __randomizedtesting.SeedInfo.seed([E4D72758D4844D12]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:234)
        at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:799)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.TestMiniSolrCloudCluster.testErrorsInShutdown

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([E4D72758D4844D12]:0)




Build Log:
[...truncated 10802 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestMiniSolrCloudCluster
   [junit4]   2> 393923 T2851 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 393923 T2852 oasc.ZkTestServer$2$1.setClientPort client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 393923 T2852 oasc.ZkTestServer$ZKServerMain.runFromConfig 
Starting server
   [junit4]   2> 394023 T2851 oasc.ZkTestServer.run start zk server on 
port:59880
   [junit4]   2> 394086 T2861 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 394087 T2862 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 394088 T2860 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 394091 T2860 oejsh.ContextHandler.doStart Started 
o.e.j.s.ServletContextHandler@378e65b2{/solr,null,AVAILABLE}
   [junit4]   2> 394096 T2860 oejs.AbstractConnector.doStart Started 
ServerConnector@6a2cd6ad{HTTP/1.1}{127.0.0.1:45239}
   [junit4]   2> 394097 T2860 oejs.Server.doStart Started @395001ms
   [junit4]   2> 394097 T2860 oascse.JettySolrRunner$1.lifeCycleStarted Jetty 
properties: {hostContext=/solr, hostPort=45239}
   [junit4]   2> 394098 T2860 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 394098 T2860 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/'
   [junit4]   2> 394108 T2861 oejsh.ContextHandler.doStart Started 
o.e.j.s.ServletContextHandler@2ea0ba5b{/solr,null,AVAILABLE}
   [junit4]   2> 394108 T2862 oejsh.ContextHandler.doStart Started 
o.e.j.s.ServletContextHandler@1356208e{/solr,null,AVAILABLE}
   [junit4]   2> 394109 T2861 oejs.AbstractConnector.doStart Started 
ServerConnector@3b93db0a{HTTP/1.1}{127.0.0.1:37357}
   [junit4]   2> 394109 T2862 oejs.AbstractConnector.doStart Started 
ServerConnector@5d44d1cb{HTTP/1.1}{127.0.0.1:54628}
   [junit4]   2> 394109 T2861 oejs.Server.doStart Started @395013ms
   [junit4]   2> 394109 T2862 oejs.Server.doStart Started @395013ms
   [junit4]   2> 394109 T2861 oascse.JettySolrRunner$1.lifeCycleStarted Jetty 
properties: {hostContext=/solr, hostPort=37357}
   [junit4]   2> 394109 T2862 oascse.JettySolrRunner$1.lifeCycleStarted Jetty 
properties: {hostContext=/solr, hostPort=54628}
   [junit4]   2> 394109 T2861 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 394110 T2862 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 394110 T2861 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/'
   [junit4]   2> 394110 T2862 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/'
   [junit4]   2> 394122 T2860 oass.SolrDispatchFilter.loadNodeConfig Trying to 
read solr.xml from 127.0.0.1:59880/solr
   [junit4]   2> 394144 T2862 oass.SolrDispatchFilter.loadNodeConfig Trying to 
read solr.xml from 127.0.0.1:59880/solr
   [junit4]   2> 394146 T2861 oass.SolrDispatchFilter.loadNodeConfig Trying to 
read solr.xml from 127.0.0.1:59880/solr
   [junit4]   2> 394222 T2860 oasc.CorePropertiesLocator.<init> Config-defined 
core root directory: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/.
   [junit4]   2> 394222 T2860 oasc.CoreContainer.<init> New CoreContainer 
1682560040
   [junit4]   2> 394222 T2860 oasc.CoreContainer.load Loading cores into 
CoreContainer 
[instanceDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/]
   [junit4]   2> 394222 T2860 oasc.CoreContainer.load loading shared library: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/lib
   [junit4]   2> 394223 T2860 oasc.SolrResourceLoader.addToClassLoader WARN 
Can't find (or read) directory to add to classloader: lib (resolved as: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/lib).
   [junit4]   2> 394224 T2862 oasc.CorePropertiesLocator.<init> Config-defined 
core root directory: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/.
   [junit4]   2> 394224 T2862 oasc.CoreContainer.<init> New CoreContainer 
260166876
   [junit4]   2> 394225 T2862 oasc.CoreContainer.load Loading cores into 
CoreContainer 
[instanceDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/]
   [junit4]   2> 394225 T2862 oasc.CoreContainer.load loading shared library: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/lib
   [junit4]   2> 394225 T2862 oasc.SolrResourceLoader.addToClassLoader WARN 
Can't find (or read) directory to add to classloader: lib (resolved as: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/lib).
   [junit4]   2> 394228 T2860 oashc.HttpShardHandlerFactory.init created with 
socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 
20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 
2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : 
false,useRetries : false,
   [junit4]   2> 394232 T2860 oasu.UpdateShardHandler.<init> Creating 
UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 394232 T2860 oasl.LogWatcher.createWatcher SLF4J impl is 
org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 394233 T2860 oasl.LogWatcher.newRegisteredLogWatcher 
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 394233 T2860 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 394233 T2861 oasc.CorePropertiesLocator.<init> Config-defined 
core root directory: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/.
   [junit4]   2> 394233 T2860 oasc.ZkContainer.initZooKeeper Zookeeper 
client=127.0.0.1:59880/solr
   [junit4]   2> 394236 T2860 oasc.ZkController.checkChrootPath zkHost includes 
chroot
   [junit4]   2> 394236 T2862 oashc.HttpShardHandlerFactory.init created with 
socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 
20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 
2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : 
false,useRetries : false,
   [junit4]   2> 394233 T2861 oasc.CoreContainer.<init> New CoreContainer 
1404940863
   [junit4]   2> 394238 T2861 oasc.CoreContainer.load Loading cores into 
CoreContainer 
[instanceDir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/]
   [junit4]   2> 394238 T2861 oasc.CoreContainer.load loading shared library: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/lib
   [junit4]   2> 394239 T2861 oasc.SolrResourceLoader.addToClassLoader WARN 
Can't find (or read) directory to add to classloader: lib (resolved as: 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/lib).
   [junit4]   2> 394243 T2861 oashc.HttpShardHandlerFactory.init created with 
socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 
20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 
2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : 
false,useRetries : false,
   [junit4]   2> 394243 T2861 oasu.UpdateShardHandler.<init> Creating 
UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 394244 T2861 oasl.LogWatcher.createWatcher SLF4J impl is 
org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 394244 T2861 oasl.LogWatcher.newRegisteredLogWatcher 
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 394244 T2861 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 394244 T2861 oasc.ZkContainer.initZooKeeper Zookeeper 
client=127.0.0.1:59880/solr
   [junit4]   2> 394244 T2861 oasc.ZkController.checkChrootPath zkHost includes 
chroot
   [junit4]   2> 394238 T2862 oasu.UpdateShardHandler.<init> Creating 
UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 394268 T2862 oasl.LogWatcher.createWatcher SLF4J impl is 
org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 394268 T2862 oasl.LogWatcher.newRegisteredLogWatcher 
Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 394269 T2862 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 394269 T2862 oasc.ZkContainer.initZooKeeper Zookeeper 
client=127.0.0.1:59880/solr
   [junit4]   2> 394269 T2862 oasc.ZkController.checkChrootPath zkHost includes 
chroot
   [junit4]   2> 394392 T2862 n:127.0.0.1:54628_solr 
oasc.ZkController.createEphemeralLiveNode Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:54628_solr
   [junit4]   2> 394393 T2861 n:127.0.0.1:37357_solr 
oasc.ZkController.createEphemeralLiveNode Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:37357_solr
   [junit4]   2> 394393 T2860 n:127.0.0.1:45239_solr 
oasc.ZkController.createEphemeralLiveNode Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:45239_solr
   [junit4]   2> 394417 T2860 n:127.0.0.1:45239_solr oasc.Overseer.close 
Overseer (id=null) closing
   [junit4]   2> 394418 T2862 n:127.0.0.1:54628_solr oasc.Overseer.close 
Overseer (id=null) closing
   [junit4]   2> 394420 T2861 n:127.0.0.1:37357_solr oasc.Overseer.close 
Overseer (id=null) closing
   [junit4]   2> 394426 T2860 n:127.0.0.1:45239_solr 
oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 
127.0.0.1:45239_solr
   [junit4]   2> 394429 T2861 n:127.0.0.1:37357_solr 
oasc.CorePropertiesLocator.discover Looking for core definitions underneath 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/.
   [junit4]   2> 394431 T2861 n:127.0.0.1:37357_solr 
oasc.CorePropertiesLocator.discover Found 0 core definitions
   [junit4]   2> 394431 T2861 n:127.0.0.1:37357_solr 
oass.SolrDispatchFilter.init 
user.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 394431 T2861 n:127.0.0.1:37357_solr 
oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 394432 T2862 n:127.0.0.1:54628_solr 
oasc.CorePropertiesLocator.discover Looking for core definitions underneath 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/.
   [junit4]   2> 394433 T2862 n:127.0.0.1:54628_solr 
oasc.CorePropertiesLocator.discover Found 0 core definitions
   [junit4]   2> 394433 T2860 n:127.0.0.1:45239_solr oasc.Overseer.start 
Overseer (id=93736187175174150-127.0.0.1:45239_solr-n_0000000000) starting
   [junit4]   2> 394433 T2862 n:127.0.0.1:54628_solr 
oass.SolrDispatchFilter.init 
user.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 394433 T2862 n:127.0.0.1:54628_solr 
oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 394456 T2860 n:127.0.0.1:45239_solr 
oasc.OverseerAutoReplicaFailoverThread.<init> Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=10000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 394456 T2926 n:127.0.0.1:45239_solr 
oasc.OverseerCollectionProcessor.run Process current queue of collection 
creations
   [junit4]   2> 394460 T2925 n:127.0.0.1:45239_solr 
oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 394465 T2860 n:127.0.0.1:45239_solr 
oasc.CorePropertiesLocator.discover Looking for core definitions underneath 
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-001/.
   [junit4]   2> 394466 T2860 n:127.0.0.1:45239_solr 
oasc.CorePropertiesLocator.discover Found 0 core definitions
   [junit4]   2> 394466 T2860 n:127.0.0.1:45239_solr 
oass.SolrDispatchFilter.init 
user.dir=/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 394466 T2860 n:127.0.0.1:45239_solr 
oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 394477 T2862 oejs.AbstractConnector.doStop Stopped 
ServerConnector@5d44d1cb{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 394477 T2862 oasc.CoreContainer.shutdown Shutting down 
CoreContainer instance=260166876
   [junit4]   2> 394480 T2860 oejs.AbstractConnector.doStop Stopped 
ServerConnector@3b93db0a{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 394481 T2860 oasc.CoreContainer.shutdown Shutting down 
CoreContainer instance=1404940863
   [junit4]   2> 394481 T2861 oejs.AbstractConnector.doStop Stopped 
ServerConnector@6a2cd6ad{HTTP/1.1}{127.0.0.1:0}
   [junit4]   2> 394484 T2861 oasc.CoreContainer.shutdown Shutting down 
CoreContainer instance=1682560040
   [junit4]   2> 395980 T2922 n:127.0.0.1:54628_solr 
oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot 
talk to ZK
   [junit4]   2> 395981 T2862 oejsh.ContextHandler.doStop Stopped 
o.e.j.s.ServletContextHandler@1356208e{/solr,null,UNAVAILABLE}
   [junit4]   2> 395989 T2919 n:127.0.0.1:37357_solr 
oasc.LeaderElector$ElectionWatcher.process WARN  
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = 
Session expired for /overseer_elect/election
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2>        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2>        at 
org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:328)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:325)
   [junit4]   2>        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:325)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:95)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
   [junit4]   2>        at 
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:390)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:264)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 395990 T2932 n:127.0.0.1:37357_solr 
oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot 
talk to ZK
   [junit4]   2> 395990 T2860 oejsh.ContextHandler.doStop Stopped 
o.e.j.s.ServletContextHandler@2ea0ba5b{/solr,null,UNAVAILABLE}
   [junit4]   2> 7200669 T2850 ccr.ThreadLeakControl$2.evaluate WARNING Suite 
execution timed out: org.apache.solr.cloud.TestMiniSolrCloudCluster
   [junit4]   2>        ==== jstack at approximately timeout time ====
   [junit4]   2>        "qtp869008505-2930" ID=2930 TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "Thread-1327" ID=2928 WAITING on 
java.lang.Object@11cafb0
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                - waiting on java.lang.Object@11cafb0
   [junit4]   2>                at java.lang.Object.wait(Object.java:502)
   [junit4]   2>                at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:911)
   [junit4]   2>        
   [junit4]   2>        
"OverseerHdfsCoreFailoverThread-93736187175174150-127.0.0.1:45239_solr-n_0000000000"
 ID=2927 TIMED_WAITING
   [junit4]   2>                at java.lang.Thread.sleep(Native Method)
   [junit4]   2>                at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        
"OverseerCollectionProcessor-93736187175174150-127.0.0.1:45239_solr-n_0000000000"
 ID=2926 TIMED_WAITING on java.lang.Object@35cdd1ec
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                - timed waiting on java.lang.Object@35cdd1ec
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue$LatchWatcher.await(DistributedQueue.java:276)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:320)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.peekTopN(DistributedQueue.java:447)
   [junit4]   2>                at 
org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:290)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        
"OverseerStateUpdate-93736187175174150-127.0.0.1:45239_solr-n_0000000000" 
ID=2925 TIMED_WAITING on java.lang.Object@79b811b1
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                - timed waiting on java.lang.Object@79b811b1
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue$LatchWatcher.await(DistributedQueue.java:276)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:320)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:594)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:572)
   [junit4]   2>                at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:189)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "jetty-launcher-279-thread-1-EventThread" ID=2911 
WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64acce10
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@64acce10
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
   [junit4]   2>        
   [junit4]   2>        
"jetty-launcher-279-thread-1-SendThread(127.0.0.1:59880)" ID=2910 RUNNABLE (in 
native code)
   [junit4]   2>                at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]   2>                at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]   2>                at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
   [junit4]   2>                at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2>                - locked sun.nio.ch.Util$2@299c1a
   [junit4]   2>                - locked 
java.util.Collections$UnmodifiableSet@5bb06f42
   [junit4]   2>                - locked sun.nio.ch.EPollSelectorImpl@62bcd81c
   [junit4]   2>                at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2>        
   [junit4]   2>        
"org.eclipse.jetty.server.session.HashSessionManager@1c191904Timer" ID=2882 
TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@60e25615
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@60e25615
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>                at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "Scheduler-789236344" ID=2881 TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c67df5f
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c67df5f
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>                at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "qtp869008505-2880" ID=2880 TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "qtp869008505-2879" ID=2879 TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "qtp869008505-2878" ID=2878 TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "qtp869008505-2877" ID=2877 TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "qtp869008505-2876" ID=2876 TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "qtp869008505-2875" ID=2875 TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "qtp869008505-2874" ID=2874 TIMED_WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - timed waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@483315f
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "jetty-launcher-279-thread-2" ID=2861 WAITING on 
org.apache.solr.core.CloserThread@3837ecc9
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                - waiting on 
org.apache.solr.core.CloserThread@3837ecc9
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2>                at 
org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:366)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:213)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.FilterHolder.destroyInstance(FilterHolder.java:173)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.FilterHolder.doStop(FilterHolder.java:151)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                - locked java.lang.Object@74f56fee
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.doStop(ServletHandler.java:241)
   [junit4]   2>                - locked 
org.eclipse.jetty.servlet.ServletHandler@708d2554
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                - locked java.lang.Object@2ad87ae
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2>                at 
org.eclipse.jetty.server.session.SessionHandler.doStop(SessionHandler.java:127)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                - locked java.lang.Object@10223d83
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:835)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:215)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                - locked java.lang.Object@4f77aa76
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2>                at 
org.eclipse.jetty.server.Server.doStop(Server.java:456)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                - locked java.lang.Object@56c99149
   [junit4]   2>                at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:417)
   [junit4]   2>                at 
org.apache.solr.cloud.MiniSolrCloudCluster.stopJettySolrRunner(MiniSolrCloudCluster.java:273)
   [junit4]   2>                at 
org.apache.solr.cloud.TestMiniSolrCloudCluster$2.stopJettySolrRunner(TestMiniSolrCloudCluster.java:220)
   [junit4]   2>                at 
org.apache.solr.cloud.MiniSolrCloudCluster$2.call(MiniSolrCloudCluster.java:330)
   [junit4]   2>                at 
org.apache.solr.cloud.MiniSolrCloudCluster$2.call(MiniSolrCloudCluster.java:327)
   [junit4]   2>                at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>                at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>                Locked synchronizers:
   [junit4]   2>                - 
java.util.concurrent.ThreadPoolExecutor$Worker@57d46826
   [junit4]   2>        
   [junit4]   2>        "ProcessThread(sid:0 cport:-1):" ID=2856 WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39f81069
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@39f81069
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
   [junit4]   2>        
   [junit4]   2>        "SyncThread:0" ID=2855 WAITING on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4bae393a
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - waiting on 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4bae393a
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:127)
   [junit4]   2>        
   [junit4]   2>        "SessionTracker" ID=2854 TIMED_WAITING on 
org.apache.zookeeper.server.SessionTrackerImpl@5177badf
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                - timed waiting on 
org.apache.zookeeper.server.SessionTrackerImpl@5177badf
   [junit4]   2>                at 
org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]   2>        
   [junit4]   2>        "NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0" ID=2853 
RUNNABLE (in native code)
   [junit4]   2>                at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]   2>                at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]   2>                at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
   [junit4]   2>                at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2>                - locked sun.nio.ch.Util$2@5621459c
   [junit4]   2>                - locked 
java.util.Collections$UnmodifiableSet@2f236c69
   [junit4]   2>                - locked sun.nio.ch.EPollSelectorImpl@15fee75e
   [junit4]   2>                at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2>                at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        "Thread-1297" ID=2852 WAITING on 
java.lang.Thread@79b3167c
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                - waiting on java.lang.Thread@79b3167c
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2>                at 
org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)
   [junit4]   2>                at 
org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:308)
   [junit4]   2>                at 
org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:481)
   [junit4]   2>        
   [junit4]   2>        
"TEST-TestMiniSolrCloudCluster.testErrorsInShutdown-seed#[E4D72758D4844D12]" 
ID=2851 WAITING on java.util.concurrent.FutureTask@25e2b173
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                - waiting on 
java.util.concurrent.FutureTask@25e2b173
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>                at 
java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
   [junit4]   2>                at 
java.util.concurrent.FutureTask.get(FutureTask.java:191)
   [junit4]   2>                at 
java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:244)
   [junit4]   2>                at 
org.apache.solr.cloud.MiniSolrCloudCluster.shutdown(MiniSolrCloudCluster.java:335)
   [junit4]   2>                at 
org.apache.solr.cloud.TestMiniSolrCloudCluster.testErrorsInShutdown(TestMiniSolrCloudCluster.java:228)
   [junit4]   2>                at 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>                at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>                at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>                at 
java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>                at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>                at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        
   [junit4]   2>        
"SUITE-TestMiniSolrCloudCluster-seed#[E4D72758D4844D12]" ID=2850 RUNNABLE
   [junit4]   2>                at 
sun.management.ThreadImpl.dumpThreads0(Native Method)
   [junit4]   2>                at 
sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:671)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.access$900(ThreadLeakControl.java:62)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:412)
   [junit4]   2>                - locked java.lang.Object@1196d182
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:651)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:568)
   [junit4]   2>        
   [junit4]   2>        "JUnit4-serializer-daemon" ID=12 TIMED_WAITING
   [junit4]   2>                at java.lang.Thread.sleep(Native Method)
   [junit4]   2>                at 
com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2>        
   [junit4]   2>        "Signal Dispatcher" ID=5 RUNNABLE
   [junit4]   2>        
   [junit4]   2>        "Finalizer" ID=3 WAITING on 
java.lang.ref.ReferenceQueue$Lock@733000f9
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                - waiting on 
java.lang.ref.ReferenceQueue$Lock@733000f9
   [junit4]   2>                at 
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
   [junit4]   2>                at 
java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
   [junit4]   2>                at 
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
   [junit4]   2>        
   [junit4]   2>        "Reference Handler" ID=2 WAITING on 
java.lang.ref.Reference$Lock@12d1f815
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                - waiting on 
java.lang.ref.Reference$Lock@12d1f815
   [junit4]   2>                at java.lang.Object.wait(Object.java:502)
   [junit4]   2>                at 
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
   [junit4]   2>        
   [junit4]   2>        "main" ID=1 WAITING on 
com.carrotsearch.randomizedtesting.RandomizedRunner$1@45be3b64
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                - waiting on 
com.carrotsearch.randomizedtesting.RandomizedRunner$1@45be3b64
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:578)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:444)
   [junit4]   2>                at 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
   [junit4]   2>                at 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
   [junit4]   2>                at 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
   [junit4]   2>        
   [junit4]   2>        ^^==============================================
   [junit4]   2>        
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestMiniSolrCloudCluster -Dtests.method=testErrorsInShutdown 
-Dtests.seed=E4D72758D4844D12 -Dtests.multiplier=3 -Dtests.slow=true 
-Dtests.locale=sr_RS_#Latn -Dtests.timezone=Etc/GMT-7 -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   7200s J0 | TestMiniSolrCloudCluster.testErrorsInShutdown <<<
   [junit4]    > Throwable #1: java.lang.Exception: Test abandoned because 
suite timeout was reached.
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([E4D72758D4844D12]:0)
   [junit4]   2> 7200679 T2850 ccr.ThreadLeakControl.checkThreadLeaks WARNING 
Will linger awaiting termination of 23 leaked thread(s).
   [junit4]   2> 7220703 T2850 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 23 
threads leaked from SUITE scope at 
org.apache.solr.cloud.TestMiniSolrCloudCluster: 
   [junit4]   2>           1) Thread[id=2852, name=Thread-1297, state=WAITING, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2>                at 
org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)
   [junit4]   2>                at 
org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:308)
   [junit4]   2>                at 
org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:481)
   [junit4]   2>           2) Thread[id=2878, name=qtp869008505-2878, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>           3) Thread[id=2876, name=qtp869008505-2876, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>           4) Thread[id=2855, name=SyncThread:0, state=WAITING, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:127)
   [junit4]   2>           5) Thread[id=2856, name=ProcessThread(sid:0 
cport:-1):, state=WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
   [junit4]   2>           6) Thread[id=2879, name=qtp869008505-2879, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>           7) Thread[id=2911, 
name=jetty-launcher-279-thread-1-EventThread, state=WAITING, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>                at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
   [junit4]   2>           8) Thread[id=2854, name=SessionTracker, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                at 
org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]   2>           9) Thread[id=2851, 
name=TEST-TestMiniSolrCloudCluster.testErrorsInShutdown-seed#[E4D72758D4844D12],
 state=WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>                at 
java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
   [junit4]   2>                at 
java.util.concurrent.FutureTask.get(FutureTask.java:191)
   [junit4]   2>                at 
java.util.concurrent.AbstractExecutorService.invokeAll(AbstractExecutorService.java:244)
   [junit4]   2>                at 
org.apache.solr.cloud.MiniSolrCloudCluster.shutdown(MiniSolrCloudCluster.java:335)
   [junit4]   2>                at 
org.apache.solr.cloud.TestMiniSolrCloudCluster.testErrorsInShutdown(TestMiniSolrCloudCluster.java:228)
   [junit4]   2>                at 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2>                at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>                at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>                at 
java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>                at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>                at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>                at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>                at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          10) Thread[id=2853, 
name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]   2>                at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]   2>                at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
   [junit4]   2>                at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2>                at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2>                at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          11) Thread[id=2925, 
name=OverseerStateUpdate-93736187175174150-127.0.0.1:45239_solr-n_0000000000, 
state=TIMED_WAITING, group=Overseer state updater.]
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue$LatchWatcher.await(DistributedQueue.java:276)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:320)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:594)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:572)
   [junit4]   2>                at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:189)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          12) Thread[id=2910, 
name=jetty-launcher-279-thread-1-SendThread(127.0.0.1:59880), state=RUNNABLE, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
   [junit4]   2>                at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]   2>                at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
   [junit4]   2>                at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]   2>                at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2>          13) Thread[id=2874, name=qtp869008505-2874, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          14) Thread[id=2930, name=qtp869008505-2930, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          15) Thread[id=2877, name=qtp869008505-2877, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          16) Thread[id=2875, name=qtp869008505-2875, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          17) Thread[id=2882, 
name=org.eclipse.jetty.server.session.HashSessionManager@1c191904Timer, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>                at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          18) Thread[id=2861, name=jetty-launcher-279-thread-2, 
state=WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2>                at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2>                at 
org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:366)
   [junit4]   2>                at 
org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:213)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.FilterHolder.destroyInstance(FilterHolder.java:173)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.FilterHolder.doStop(FilterHolder.java:151)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletHandler.doStop(ServletHandler.java:241)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2>                at 
org.eclipse.jetty.server.session.SessionHandler.doStop(SessionHandler.java:127)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:835)
   [junit4]   2>                at 
org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:215)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
   [junit4]   2>                at 
org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2>                at 
org.eclipse.jetty.server.Server.doStop(Server.java:456)
   [junit4]   2>                at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2>                at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:417)
   [junit4]   2>                at 
org.apache.solr.cloud.MiniSolrCloudCluster.stopJettySolrRunner(MiniSolrCloudCluster.java:273)
   [junit4]   2>                at 
org.apache.solr.cloud.TestMiniSolrCloudCluster$2.stopJettySolrRunner(TestMiniSolrCloudCluster.java:220)
   [junit4]   2>                at 
org.apache.solr.cloud.MiniSolrCloudCluster$2.call(MiniSolrCloudCluster.java:330)
   [junit4]   2>                at 
org.apache.solr.cloud.MiniSolrCloudCluster$2.call(MiniSolrCloudCluster.java:327)
   [junit4]   2>                at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>                at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          19) Thread[id=2926, 
name=OverseerCollectionProcessor-93736187175174150-127.0.0.1:45239_solr-n_0000000000,
 state=TIMED_WAITING, group=Overseer collection creation process.]
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue$LatchWatcher.await(DistributedQueue.java:276)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:320)
   [junit4]   2>                at 
org.apache.solr.cloud.DistributedQueue.peekTopN(DistributedQueue.java:447)
   [junit4]   2>                at 
org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:290)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          20) Thread[id=2881, name=Scheduler-789236344, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>                at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>                at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          21) Thread[id=2927, 
name=OverseerHdfsCoreFailoverThread-93736187175174150-127.0.0.1:45239_solr-n_0000000000,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
   [junit4]   2>                at java.lang.Thread.sleep(Native Method)
   [junit4]   2>                at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>          22) Thread[id=2928, name=Thread-1327, state=WAITING, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                at java.lang.Object.wait(Object.java:502)
   [junit4]   2>                at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:911)
   [junit4]   2>          23) Thread[id=2880, name=qtp869008505-2880, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>                at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>                at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>                at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>                at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)
   [junit4]   2>                at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 7220706 T2850 ccr.ThreadLeakControl.tryToInterruptAll Starting 
to interrupt leaked threads:
   [junit4]   2>           1) Thread[id=2852, name=Thread-1297, state=WAITING, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>           2) Thread[id=2878, name=qtp869008505-2878, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>           3) Thread[id=2876, name=qtp869008505-2876, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>           4) Thread[id=2855, name=SyncThread:0, state=WAITING, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>           5) Thread[id=2856, name=ProcessThread(sid:0 
cport:-1):, state=WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>           6) Thread[id=2879, name=qtp869008505-2879, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>           7) Thread[id=2911, 
name=jetty-launcher-279-thread-1-EventThread, state=WAITING, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>           8) Thread[id=2854, name=SessionTracker, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>           9) Thread[id=2851, 
name=TEST-TestMiniSolrCloudCluster.testErrorsInShutdown-seed#[E4D72758D4844D12],
 state=WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          10) Thread[id=2853, 
name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          11) Thread[id=2925, 
name=OverseerStateUpdate-93736187175174150-127.0.0.1:45239_solr-n_0000000000, 
state=TIMED_WAITING, group=Overseer state updater.]
   [junit4]   2>          12) Thread[id=2910, 
name=jetty-launcher-279-thread-1-SendThread(127.0.0.1:59880), state=RUNNABLE, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          13) Thread[id=2874, name=qtp869008505-2874, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          14) Thread[id=2930, name=qtp869008505-2930, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          15) Thread[id=2877, name=qtp869008505-2877, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          16) Thread[id=2875, name=qtp869008505-2875, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          17) Thread[id=2882, 
name=org.eclipse.jetty.server.session.HashSessionManager@1c191904Timer, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          18) Thread[id=2861, name=jetty-launcher-279-thread-2, 
state=WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          19) Thread[id=2926, 
name=OverseerCollectionProcessor-93736187175174150-127.0.0.1:45239_solr-n_0000000000,
 state=TIMED_WAITING, group=Overseer collection creation process.]
   [junit4]   2>          20) Thread[id=2881, name=Scheduler-789236344, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          21) Thread[id=2927, 
name=OverseerHdfsCoreFailoverThread-93736187175174150-127.0.0.1:45239_solr-n_0000000000,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
   [junit4]   2>          22) Thread[id=2928, name=Thread-1327, state=WAITING, 
group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2>          23) Thread[id=2880, name=qtp869008505-2880, 
state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudCluster]
   [junit4]   2> 7614029 T2856 oazs.PrepRequestProcessor.run ERROR Unexpected 
interruption java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
   [junit4]   2>        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>        at 
org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
   [junit4]   2> 
   [junit4]   2> 7614032 T2927 n:127.0.0.1:45239_solr oasc.SolrException.log 
ERROR OverseerAutoReplicaFailoverThread had an error in its thread work 
loop.:org.apache.solr.common.SolrException: Error reading cluster properties
   [junit4]   2>                at 
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:773)
   [junit4]   2>                at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.doWork(OverseerAutoReplicaFailoverThread.java:152)
   [junit4]   2>                at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:131)
   [junit4]   2>                at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>        Caused by: java.lang.InterruptedException
   [junit4]   2>                at java.lang.Object.wait(Native Method)
   [junit4]   2>                at java.lang.Object.wait(Object.java:502)
   [junit4]   2>                at 
org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1342)
   [junit4]   2>                at 
org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1040)
   [junit4]   2>                at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:311)
   [junit4]   2>                at 
org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:308)
   [junit4]   2>                at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
   [junit4]   2>                at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:308)
   [junit4]   2>                at 
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:766)
   [junit4]   2>                ... 3 more
   [junit4]   2>        
   [junit4]   2> 7614030 T2861 oasc.Overseer.close Overseer 
(id=93736187175174150-127.0.0.1:45239_solr-n_0000000000) closing
   [junit4]   2> 7614029 T2911 n:127.0.0.1:45239_solr 
oaz.ClientCnxn$EventThread.run ERROR Event thread exiting due to interruption 
java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
   [junit4]   2>        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>        at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
   [junit4]   2> 
   [junit4]   2> 7614029 T2854 oazs.SessionTrackerImpl.run ERROR Unexpected 
interruption java.lang.InterruptedException
   [junit4]   2>        at java.lang.Object.wait(Native Method)
   [junit4]   2>        at 
org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]   2> 
   [junit4]   2> 7614029 T2852 oasc.ZkTestServer$ZKServerMain.runFromConfig 
WARN Server interrupted java.lang.InterruptedException
   [junit4]   2>        at java.lang.Object.wait(Native Method)
   [junit4]   2>        at java.lang.Thread.join(Thread.java:1245)
   [junit4]   2>        at java.lang.Thread.join(Thread.java:1319)
   [junit4]   2>        at 
org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:308)
   [junit4]   2>        at 
org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:481)
   [junit4]   2> 
   [junit4]   2> 7614033 T2925 n:127.0.0.1:45239_solr 
oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 
127.0.0.1:45239_solr
   [junit4]   2> 7614029 T2855 oazs.SyncRequestProcessor.run ERROR Severe 
unrecoverable error, exiting java.lang.InterruptedException
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
   [junit4]   2>        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
   [junit4]   2>        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>        at 
org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:127)
   [junit4]   2> 
   [junit4]   2> 7614036 T2861 oejsh.ContextHandler.doStop Stopped 
o.e.j.s.ServletContextHandler@378e65b2{/solr,null,UNAVAILABLE}
   [junit4]   2> 7614037 T2851 oasc.ZkTestServer.send4LetterWord connecting to 
127.0.0.1:59880 59880
   [junit4]   2> 7614069 T2851 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 7614069 T2942 oasc.ZkTestServer$2$1.setClientPort client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 7614069 T2942 oasc.ZkTestServer$ZKServerMain.runFromConfig 
Starting server
   [junit4]   2> 7614169 T2851 oasc.ZkTestServer.run start zk server on 
port:56242
   [junit4]   2> 7614206 T2950 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 7614206 T2951 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 7614206 T2952 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 7614206 T2953 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 7614206 T2954 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 7614208 T2950 oejsh.ContextHandler.doStart Started 
o.e.j.s.ServletContextHandler@a070b0e{/solr,null,AVAILABLE}
   [junit4]   2> 7614208 T2954 oejsh.ContextHandler.doStart Started 
o.e.j.s.ServletContextHandler@7cee6ad6{/solr,null,AVAILABLE}
   [junit4]   2> 7614209 T2952 oejsh.ContextHandler.doStart Started 
o.e.j.s.ServletContextHandler@3e18cbc1{/solr,null,AVAILABLE}
   [junit4]   2> 7614209 T2953 oejsh.ContextHandler.doStart Started 
o.e.j.s.ServletContextHandler@36039ac2{/solr,null,AVAILABLE}
   [junit4]   2> 7614209 T2952 oejs.AbstractConnector.doStart Started 
ServerConnector@46f625af{HTTP/1.1}{127.0.0.1:46170}
   [junit4]   2> 7614209 T2951 oejsh.ContextHandler.doStart Started 
o.e.j.s.ServletContextHandler@7d2958e0{/solr,null,AVAILABLE}
   [junit4]   2> 7614209 T2951 oejs.AbstractConnector.doStart Started 
ServerConnector@3b7bd74{HTTP/1.1}{127.0.0.1:46383}
   [junit4]   2> 7614210 T2951 oejs.Server.doStart Started @7615114ms
   [junit4]   2> 7614210 T2951 oascse.JettySolrRunner$1.lifeCycleStarted Jetty 
properties: {hostContext=/solr, hostPort=46383}
   [junit4]   2> 7614210 T2951 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 7614210 T2951 oasc.SolrResourceLoader.<init> new 
SolrResourceLoader for directory: 
'/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestMiniSolrCloudCluster
 E4D72758D4844D12-001/tempDir-002/'
   [junit4]   2> 7614209 T2954 oejs.AbstractConnector.doStart Started 
ServerConnector@5e1fccbf{HTTP/1.1}{127.0.0.1:36869}
   [junit4]   2> 7614209 T2950 oejs.AbstractConnector.doStart Started 
ServerConnector@66662422{HTTP/1.1}{127.0.0.1:42431}
   [junit4]   2> 7614210 T2954 oejs.Server.doStart Started @7615115ms
   [junit4]   2> 7614209 T2953 oejs.AbstractConnector.doStart Started 
ServerConnector@3545404d{HTTP/1.1}{127.0.0.1:50681}
   [junit4]   2> 7614209 T2952 oejs.Server.doStart Started @7615113ms
   [junit4]   2> 7614211 T2953 oejs.Server.doStart Started @7615115ms
   [junit4]   2> 7614211 T2954 oascse.JettySolrRunner$1.lifeCycleStarted Jetty 
properties: {hostContext=/solr, hostPort=36869}
   [junit4]   2> 7614211 T2950 oejs.Server.doStart Started @7615115ms
   [junit4]   2> 7614211 T2953 oascse.JettySolrRunner$1.lifeCycleStarted Jetty 
properties: {hostContext=/solr, hostPort=50681}
   [junit4]   2> 7614211 T2952 oascse.JettySolrRunner$1.lifeCycleStarted Jetty 
properties: {hostContext=/solr, hostPort=46170}
   [junit4]   2> 7614211 T2953 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 7614211 T2954 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 7614212 T2952 oass.SolrDispatchFilter.init 
SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 7614211 T2950 oascse.JettySolrRunner$1.lifeCycleStarted Jetty 
properties: {hostContext=/solr, hostPort=42431}
   [junit4]   2> 7614212 T2952 oasc.SolrResou

[...truncated too long message...]

[junit4]   2>   at 
org.apache.solr.core.CoreContainer.load(CoreContainer.java:248)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:172)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:149)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:138)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:852)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1526)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1560)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1252)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1101)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1060)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:375)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:336)
   [junit4]   2>        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
   [junit4]   2>        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:385)
   [junit4]   2>        at 
org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:246)
   [junit4]   2>        at 
org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:216)
   [junit4]   2>        at 
org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:200)
   [junit4]   2>        at 
org.apache.solr.cloud.TestMiniSolrCloudCluster.testCollectionCreateSearchDelete(TestMiniSolrCloudCluster.java:101)
   [junit4]   2>        at 
org.apache.solr.cloud.TestMiniSolrCloudCluster.testBasics(TestMiniSolrCloudCluster.java:77)
   [junit4]   2>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {}, 
docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, 
locale=ko_KR, timezone=America/Yakutat
   [junit4]   2> NOTE: Linux 3.13.0-49-generic amd64/Oracle Corporation 
1.8.0_45 (64-bit)/cpus=12,threads=1,free=380238984,total=508887040
   [junit4]   2> NOTE: All tests run in this JVM: [TestExactStatsCache, 
TestBulkSchemaConcurrent, HighlighterMaxOffsetTest, DateRangeFieldTest, 
TestCloudManagedSchema, TestSolrDeletionPolicy1, LeaderElectionIntegrationTest, 
TestDFRSimilarityFactory, TermsComponentTest, DateFieldTest, 
TestManagedSchemaDynamicFieldResource, TestOrdValues, ZkNodePropsTest, 
TestImplicitCoreProperties, SolrCoreTest, TestRealTimeGet, BlockCacheTest, 
TestReplicationHandler, DateMathParserTest, TestStandardQParsers, 
TestArbitraryIndexDir, TestConfigOverlay, MBeansHandlerTest, 
FastVectorHighlighterTest, ChaosMonkeyNothingIsSafeTest, TestHashQParserPlugin, 
TestSolrIndexConfig, SuggesterWFSTTest, TestComplexPhraseQParserPlugin, 
TestConfig, DistributedFacetPivotWhiteBoxTest, HdfsSyncSliceTest, 
SyncSliceTest, SolrCloudExampleTest, MultiTermTest, 
TestDefaultSearchFieldResource, RecoveryAfterSoftCommitTest, 
TestRemoteStreaming, TestSolr4Spatial2, TestUpdate, MinimalSchemaTest, 
TestMiniSolrCloudCluster, TestQuerySenderListener]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestQuerySenderListener -Dtests.seed=E4D72758D4844D12 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ko_KR 
-Dtests.timezone=America/Yakutat -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J0 | TestQuerySenderListener (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Some resources were 
not closed, shutdown, or released.
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([E4D72758D4844D12]:0)
   [junit4]    >        at 
org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:234)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [447/485] on J0 in 0.29s, 3 tests, 1 failure <<< FAILURES!

[...truncated 126 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:526: The following 
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following 
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:61: The following 
error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The 
following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:229: The 
following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:512: The 
following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1415: 
The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:973: 
There were test failures: 485 suites, 1945 tests, 2 suite-level errors, 1 
error, 57 ignored (25 assumptions)

Total time: 146 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to