Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/196/

4 tests failed.
FAILED:  
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationStartStop

Error Message:
Timeout while trying to assert number of documents @ target_collection

Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ 
target_collection
        at 
__randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:3B55C4154A256656]:0)
        at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
        at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationStartStop(CdcrReplicationDistributedZkTest.java:173)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: expected:<10> but was:<6>
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
        ... 42 more


FAILED:  org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test

Error Message:
Timeout waiting for all live and active

Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
        at 
__randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:30C2255CF0018027]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:309)
        at 
org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:127)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets

Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2

Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode 
for /collections/withconfigset2
        at 
__randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:95E855DCA9D957D3]:0)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: 
DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/3)={
   "replicationFactor":"2",   "shards":{     "shard1":{       
"range":"80000000-d554ffff",       "state":"active",       "replicas":{}},     
"shard2":{       "range":"d5550000-2aa9ffff",       "state":"active",       
"replicas":{}},     "shard3":{       "range":"2aaa0000-7fffffff",       
"state":"active",       "replicas":{}}},   "router":{"name":"compositeId"},   
"maxShardsPerNode":"2",   "autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: 
DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/3)={
  "replicationFactor":"2",
  "shards":{
    "shard1":{
      "range":"80000000-d554ffff",
      "state":"active",
      "replicas":{}},
    "shard2":{
      "range":"d5550000-2aa9ffff",
      "state":"active",
      "replicas":{}},
    "shard3":{
      "range":"2aaa0000-7fffffff",
      "state":"active",
      "replicas":{}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false"}
        at 
__randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:F0E36E3258CEC24A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11876 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/init-core-data-001
   [junit4]   2> 1187228 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1187229 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001
   [junit4]   2> 1187229 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1187230 INFO  (Thread-1588) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1187230 INFO  (Thread-1588) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1187330 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.s.c.ZkTestServer start zk server on port:51213
   [junit4]   2> 1187338 INFO  (jetty-launcher-711-thread-1) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1187339 INFO  (jetty-launcher-711-thread-3) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1187338 INFO  (jetty-launcher-711-thread-2) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1187343 INFO  (jetty-launcher-711-thread-4) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1187345 INFO  (jetty-launcher-711-thread-4) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@5c2461d{/solr,null,AVAILABLE}
   [junit4]   2> 1187351 INFO  (jetty-launcher-711-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@9f6cf6c{/solr,null,AVAILABLE}
   [junit4]   2> 1187355 INFO  (jetty-launcher-711-thread-1) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@f8160a5{HTTP/1.1,[http/1.1]}{127.0.0.1:45888}
   [junit4]   2> 1187356 INFO  (jetty-launcher-711-thread-1) [    ] 
o.e.j.s.Server Started @1191337ms
   [junit4]   2> 1187356 INFO  (jetty-launcher-711-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=45888}
   [junit4]   2> 1187356 ERROR (jetty-launcher-711-thread-1) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1187356 INFO  (jetty-launcher-711-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
6.4.0
   [junit4]   2> 1187356 INFO  (jetty-launcher-711-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1187356 INFO  (jetty-launcher-711-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1187356 INFO  (jetty-launcher-711-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-11T19:45:55.207Z
   [junit4]   2> 1187357 INFO  (jetty-launcher-711-thread-4) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@4947e949{HTTP/1.1,[http/1.1]}{127.0.0.1:36630}
   [junit4]   2> 1187357 INFO  (jetty-launcher-711-thread-4) [    ] 
o.e.j.s.Server Started @1191338ms
   [junit4]   2> 1187357 INFO  (jetty-launcher-711-thread-4) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=36630}
   [junit4]   2> 1187357 ERROR (jetty-launcher-711-thread-4) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1187357 INFO  (jetty-launcher-711-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
6.4.0
   [junit4]   2> 1187357 INFO  (jetty-launcher-711-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1187357 INFO  (jetty-launcher-711-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1187357 INFO  (jetty-launcher-711-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-11T19:45:55.208Z
   [junit4]   2> 1187362 INFO  (jetty-launcher-711-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@7082d83e{/solr,null,AVAILABLE}
   [junit4]   2> 1187362 INFO  (jetty-launcher-711-thread-3) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@72458de8{HTTP/1.1,[http/1.1]}{127.0.0.1:45902}
   [junit4]   2> 1187362 INFO  (jetty-launcher-711-thread-3) [    ] 
o.e.j.s.Server Started @1191344ms
   [junit4]   2> 1187362 INFO  (jetty-launcher-711-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=45902}
   [junit4]   2> 1187363 ERROR (jetty-launcher-711-thread-3) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1187363 INFO  (jetty-launcher-711-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
6.4.0
   [junit4]   2> 1187363 INFO  (jetty-launcher-711-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1187363 INFO  (jetty-launcher-711-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1187363 INFO  (jetty-launcher-711-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-11T19:45:55.214Z
   [junit4]   2> 1187368 INFO  (jetty-launcher-711-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@642cbee5{/solr,null,AVAILABLE}
   [junit4]   2> 1187368 INFO  (jetty-launcher-711-thread-2) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@7e1b3b73{HTTP/1.1,[http/1.1]}{127.0.0.1:52143}
   [junit4]   2> 1187368 INFO  (jetty-launcher-711-thread-2) [    ] 
o.e.j.s.Server Started @1191350ms
   [junit4]   2> 1187368 INFO  (jetty-launcher-711-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=52143}
   [junit4]   2> 1187368 ERROR (jetty-launcher-711-thread-2) [    ] 
o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1187368 INFO  (jetty-launcher-711-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
6.4.0
   [junit4]   2> 1187369 INFO  (jetty-launcher-711-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1187369 INFO  (jetty-launcher-711-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1187369 INFO  (jetty-launcher-711-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2016-11-11T19:45:55.220Z
   [junit4]   2> 1187371 INFO  (jetty-launcher-711-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1187378 INFO  (jetty-launcher-711-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1187385 INFO  (jetty-launcher-711-thread-3) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1187386 INFO  (jetty-launcher-711-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51213/solr
   [junit4]   2> 1187387 INFO  (jetty-launcher-711-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1187394 INFO  (jetty-launcher-711-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1187398 INFO  (jetty-launcher-711-thread-1) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1187398 INFO  (jetty-launcher-711-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51213/solr
   [junit4]   2> 1187401 INFO  (jetty-launcher-711-thread-4) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1187401 INFO  (jetty-launcher-711-thread-2) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1187402 INFO  (jetty-launcher-711-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51213/solr
   [junit4]   2> 1187403 INFO  (jetty-launcher-711-thread-4) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51213/solr
   [junit4]   2> 1187450 INFO  (jetty-launcher-711-thread-2) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:52143_solr
   [junit4]   2> 1187454 INFO  (jetty-launcher-711-thread-3) 
[n:127.0.0.1:45902_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:45902_solr
   [junit4]   2> 1187455 INFO  (jetty-launcher-711-thread-4) 
[n:127.0.0.1:36630_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:36630_solr
   [junit4]   2> 1187455 INFO  (jetty-launcher-711-thread-1) 
[n:127.0.0.1:45888_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:45888_solr
   [junit4]   2> 1187455 INFO  (jetty-launcher-711-thread-2) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.Overseer Overseer 
(id=96920768026312715-127.0.0.1:52143_solr-n_0000000000) starting
   [junit4]   2> 1187471 INFO  
(zkCallback-729-thread-1-processing-n:127.0.0.1:45902_solr) 
[n:127.0.0.1:45902_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 1187471 INFO  (jetty-launcher-711-thread-4) 
[n:127.0.0.1:36630_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node4/.
   [junit4]   2> 1187471 INFO  
(zkCallback-732-thread-1-processing-n:127.0.0.1:36630_solr) 
[n:127.0.0.1:36630_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 1187472 INFO  
(zkCallback-731-thread-1-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 1187472 INFO  
(zkCallback-730-thread-1-processing-n:127.0.0.1:45888_solr) 
[n:127.0.0.1:45888_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 1187481 INFO  (jetty-launcher-711-thread-2) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:52143_solr
   [junit4]   2> 1187483 INFO  
(zkCallback-730-thread-1-processing-n:127.0.0.1:45888_solr) 
[n:127.0.0.1:45888_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 1187483 INFO  
(zkCallback-732-thread-1-processing-n:127.0.0.1:36630_solr) 
[n:127.0.0.1:36630_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 1187483 INFO  
(zkCallback-731-thread-1-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 1187489 INFO  (jetty-launcher-711-thread-1) 
[n:127.0.0.1:45888_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node1/.
   [junit4]   2> 1187492 INFO  (jetty-launcher-711-thread-2) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node2/.
   [junit4]   2> 1187498 INFO  
(zkCallback-729-thread-1-processing-n:127.0.0.1:45902_solr) 
[n:127.0.0.1:45902_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 1187527 INFO  (jetty-launcher-711-thread-3) 
[n:127.0.0.1:45902_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node3/.
   [junit4]   2> 1187534 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1188107 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your 
platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1188780 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1188966 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via 
org.mortbay.log.Slf4jLog
   [junit4]   2> 1188992 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1189064 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 1189116 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_49480_hdfs____.a2omvj/webapp
   [junit4]   2> 1189869 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49480
   [junit4]   2> 1190777 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1190787 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 1190802 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_42718_datanode____.igor7d/webapp
   [junit4]   2> 1191232 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42718
   [junit4]   2> 1191702 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1191705 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 1191723 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_45598_datanode____.ihfcfa/webapp
   [junit4]   2> 1192073 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45598
   [junit4]   2> 1193206 INFO  (IPC Server handler 5 on 37446) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-f08d4d89-0367-4ff9-83f2-140475a4c65c node 
DatanodeRegistration(127.0.0.1:36702, 
datanodeUuid=1bf096c0-60fd-4e67-9f6f-f0ecf3732489, infoPort=54272, 
infoSecurePort=0, ipcPort=58312, 
storageInfo=lv=-56;cid=testClusterID;nsid=919641284;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 3 msecs
   [junit4]   2> 1193206 INFO  (IPC Server handler 6 on 37446) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-b4025d97-671c-49b1-a857-af62f730cb9d node 
DatanodeRegistration(127.0.0.1:45133, 
datanodeUuid=9dea7d69-f9e8-49d4-8582-792b2d73bf77, infoPort=37922, 
infoSecurePort=0, ipcPort=34614, 
storageInfo=lv=-56;cid=testClusterID;nsid=919641284;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 1 msecs
   [junit4]   2> 1193207 INFO  (IPC Server handler 5 on 37446) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-b7b2adcd-7dc2-4d4f-ac0a-e2e6c80efbc7 node 
DatanodeRegistration(127.0.0.1:36702, 
datanodeUuid=1bf096c0-60fd-4e67-9f6f-f0ecf3732489, infoPort=54272, 
infoSecurePort=0, ipcPort=58312, 
storageInfo=lv=-56;cid=testClusterID;nsid=919641284;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 1193210 INFO  (IPC Server handler 6 on 37446) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-cfc90d9b-1fbd-46f8-ba06-ee243be158b7 node 
DatanodeRegistration(127.0.0.1:45133, 
datanodeUuid=9dea7d69-f9e8-49d4-8582-792b2d73bf77, infoPort=37922, 
infoSecurePort=0, ipcPort=34614, 
storageInfo=lv=-56;cid=testClusterID;nsid=919641284;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 1193594 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testBadActionNames-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testBadActionNames
   [junit4]   2> 1193597 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testBadActionNames-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1193604 ERROR (qtp1903229075-5581) [n:127.0.0.1:36630_solr    
] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Unknown 
action: BADACTION
   [junit4]   2>        at 
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:203)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2>        at 
org.eclipse.jetty.server.Server.handle(Server.java:518)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2>        at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2>        at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1193605 INFO  (qtp1903229075-5581) [n:127.0.0.1:36630_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={name=badactioncollection&action=BADACTION&numShards=2&wt=javabin&version=2}
 status=400 QTime=0
   [junit4]   2> 1193605 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testBadActionNames-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testBadActionNames
   [junit4]   2> 1193672 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testSpecificConfigsets
   [junit4]   2> 1193674 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1193688 INFO  (qtp1903229075-5583) [n:127.0.0.1:36630_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=1&collection.configName=conf2&name=withconfigset2&action=CREATE&numShards=1&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1193718 INFO  
(OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.CreateCollectionCmd Create collection 
withconfigset2
   [junit4]   2> 1193836 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=conf2&newCollection=true&name=withconfigset2_shard1_replica1&action=CREATE&numShards=1&collection=withconfigset2&shard=shard1&wt=javabin&version=2
   [junit4]   2> 1193942 INFO  
(zkCallback-729-thread-1-processing-n:127.0.0.1:45902_solr) 
[n:127.0.0.1:45902_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/withconfigset2/state.json] for collection [withconfigset2] 
has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1194859 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1  x:withconfigset2_shard1_replica1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.4.0
   [junit4]   2> 1194867 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1  x:withconfigset2_shard1_replica1] 
o.a.s.s.IndexSchema [withconfigset2_shard1_replica1] Schema name=minimal
   [junit4]   2> 1194870 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1  x:withconfigset2_shard1_replica1] 
o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1194872 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1  x:withconfigset2_shard1_replica1] 
o.a.s.c.CoreContainer Creating SolrCore 'withconfigset2_shard1_replica1' using 
configuration from collection withconfigset2
   [junit4]   2> 1194886 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37446/data
   [junit4]   2> 1194886 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1194886 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] 
o.a.s.c.SolrCore [[withconfigset2_shard1_replica1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node3/withconfigset2_shard1_replica1],
 dataDir=[hdfs://localhost:37446/data/withconfigset2/core_node1/data/]
   [junit4]   2> 1194886 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8538112
   [junit4]   2> 1194887 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] 
o.a.s.c.SolrCore [withconfigset2_shard1_replica1]  CLOSING SolrCore 
org.apache.solr.core.SolrCore@5384e9c9
   [junit4]   2> 1194887 ERROR (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] 
o.a.s.c.SolrCore Error while closing
   [junit4]   2> java.lang.NullPointerException
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.close(SolrCore.java:1348)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:892)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2>        at 
org.eclipse.jetty.server.Server.handle(Server.java:518)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2>        at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2>        at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1194888 ERROR (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] 
o.a.s.c.CoreContainer Error creating core [withconfigset2_shard1_replica1]: 
Unrecognized lockType: native
   [junit4]   2> org.apache.solr.common.SolrException: Unrecognized lockType: 
native
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:900)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2>        at 
org.eclipse.jetty.server.Server.handle(Server.java:518)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2>        at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2>        at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unrecognized 
lockType: native
   [junit4]   2>        at 
org.apache.solr.core.HdfsDirectoryFactory.createLockFactory(HdfsDirectoryFactory.java:178)
   [junit4]   2>        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:434)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:841)
   [junit4]   2>        ... 35 more
   [junit4]   2> 1194890 ERROR (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] 
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error 
CREATEing SolrCore 'withconfigset2_shard1_replica1': Unable to create core 
[withconfigset2_shard1_replica1] Caused by: Unrecognized lockType: native
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:813)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
   [junit4]   2>        at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
   [junit4]   2>        at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
   [junit4]   2>        at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
   [junit4]   2>        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
   [junit4]   2>        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
   [junit4]   2>        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
   [junit4]   2>        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2>        at 
org.eclipse.jetty.server.Server.handle(Server.java:518)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
   [junit4]   2>        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
   [junit4]   2>        at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2>        at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2>        at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]   2>        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unable to 
create core [withconfigset2_shard1_replica1]
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:855)
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
   [junit4]   2>        ... 32 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unrecognized 
lockType: native
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:900)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
   [junit4]   2>        at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
   [junit4]   2>        ... 33 more
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unrecognized 
lockType: native
   [junit4]   2>        at 
org.apache.solr.core.HdfsDirectoryFactory.createLockFactory(HdfsDirectoryFactory.java:178)
   [junit4]   2>        at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:434)
   [junit4]   2>        at 
org.apache.solr.core.SolrCore.<init>(SolrCore.java:841)
   [junit4]   2>        ... 35 more
   [junit4]   2> 
   [junit4]   2> 1194890 INFO  (qtp64220878-5594) [n:127.0.0.1:45902_solr 
c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=conf2&newCollection=true&name=withconfigset2_shard1_replica1&action=CREATE&numShards=1&collection=withconfigset2&shard=shard1&wt=javabin&version=2}
 status=400 QTime=1056
   [junit4]   2> 1194891 ERROR 
(OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.OverseerCollectionMessageHandler Error 
from shard: http://127.0.0.1:45902/solr
   [junit4]   2> 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:45902/solr: Error CREATEing SolrCore 
'withconfigset2_shard1_replica1': Unable to create core 
[withconfigset2_shard1_replica1] Caused by: Unrecognized lockType: native
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:593)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
   [junit4]   2>        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
   [junit4]   2>        at 
org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]   2>        at 
org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:196)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1194895 ERROR 
(OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.OverseerCollectionMessageHandler Cleaning 
up collection [withconfigset2].
   [junit4]   2> 1194896 INFO  
(OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing 
Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 1194898 INFO  (qtp64220878-5596) [n:127.0.0.1:45902_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={deleteInstanceDir=true&core=withconfigset2_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2}
 status=0 QTime=0
   [junit4]   2> 1195601 INFO  
(OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.CreateCollectionCmd Cleaned up artifacts 
for failed create collection for [withconfigset2]
   [junit4]   2> 1195605 INFO  (qtp1903229075-5583) [n:127.0.0.1:36630_solr    
] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
30 seconds. Check all shard replicas
   [junit4]   2> 1195700 INFO  
(OverseerCollectionConfigSetProcessor-96920768026312715-127.0.0.1:52143_solr-n_0000000000)
 [n:127.0.0.1:52143_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 1225610 ERROR (qtp1903229075-5583) [n:127.0.0.1:36630_solr    
] o.a.s.h.a.CollectionsHandler Timed out waiting for new collection's replicas 
to become ACTIVE with timeout=30
   [junit4]   2> 1225611 INFO  (qtp1903229075-5583) [n:127.0.0.1:36630_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={replicationFactor=1&collection.configName=conf2&name=withconfigset2&action=CREATE&numShards=1&wt=javabin&version=2}
 status=0 QTime=31922
   [junit4]   2> 1225612 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testSpecificConfigsets
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=HdfsCollectionsAPIDistributedZkTest 
-Dtests.method=testSpecificConfigsets -Dtests.seed=B8961A865EFDEDDF 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=ms-MY -Dtests.timezone=Asia/Kabul -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   32.0s J0 | 
HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets <<<
   [junit4]    > Throwable #1: 
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode 
for /collections/withconfigset2
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:95E855DCA9D957D3]:0)
   [junit4]    >        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]    >        at 
org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]    >        at 
org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
   [junit4]    >        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]    >        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
   [junit4]    >        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1225692 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
   [junit4]   2> 1225694 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1225697 INFO  (qtp1903229075-5584) [n:127.0.0.1:36630_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1225700 INFO  
(OverseerThreadFactory-3109-thread-2-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.CreateCollectionCmd Create collection 
collection
   [junit4]   2> 1225700 WARN  
(OverseerThreadFactory-3109-thread-2-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.CreateCollectionCmd Specified 
replicationFactor of 10 on collection collection is higher than or equal to the 
number of Solr instances currently live or live and part of your 
createNodeSet(4). It's unusual to run two replica of the same slice on the same 
Solr-instance.
   [junit4]   2> 1225701 ERROR 
(OverseerThreadFactory-3109-thread-2-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.OverseerCollectionMessageHandler 
Collection: collection operation: create 
failed:org.apache.solr.common.SolrException: Cannot create collection 
collection. Value of maxShardsPerNode is 1, and the number of nodes currently 
live or live and part of your createNodeSet is 4. This allows a maximum of 4 to 
be created. Value of numShards is 2 and value of replicationFactor is 10. This 
requires 20 shards to be created (higher than the allowed number)
   [junit4]   2>        at 
org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:148)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1225704 INFO  (qtp1903229075-5584) [n:127.0.0.1:36630_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2}
 status=400 QTime=6
   [junit4]   2> 1225704 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
   [junit4]   2> 1225788 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testMissingNumShards-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testMissingNumShards
   [junit4]   2> 1225789 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testMissingNumShards-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1225793 INFO  (qtp1903229075-5586) [n:127.0.0.1:36630_solr    
] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1225798 INFO  
(OverseerThreadFactory-3109-thread-3-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.CreateCollectionCmd Create collection 
acollection
   [junit4]   2> 1225799 ERROR 
(OverseerThreadFactory-3109-thread-3-processing-n:127.0.0.1:52143_solr) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.OverseerCollectionMessageHandler 
Collection: acollection operation: create 
failed:org.apache.solr.common.SolrException: numShards is a required param 
(when using CompositeId router).
   [junit4]   2>        at 
org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:105)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
   [junit4]   2>        at 
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1225802 INFO  
(OverseerCollectionConfigSetProcessor-96920768026312715-127.0.0.1:52143_solr-n_0000000000)
 [n:127.0.0.1:52143_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 1225808 INFO  (qtp1903229075-5586) [n:127.0.0.1:36630_solr    
] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&wt=javabin&version=2}
 status=400 QTime=14
   [junit4]   2> 1225808 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testMissingNumShards-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testMissingNumShards
   [junit4]   2> 1225900 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testNoCollectionSpecified-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Starting testNoCollectionSpecified
   [junit4]   2> 1225901 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.testNoCollectionSpecified-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1225905 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr    
] o.a.s.h.a.CoreAdminOperation core create command 
dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003&collection.configName=conf&name=corewithnocollection&action=CREATE&numShards=1&collection=&wt=javabin&version=2
   [junit4]   2> 1225908 INFO  
(OverseerStateUpdate-96920768026312715-127.0.0.1:52143_solr-n_0000000000) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 1226919 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection   x:corewithnocollection] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 6.4.0
   [junit4]   2> 1226947 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection   x:corewithnocollection] o.a.s.s.IndexSchema 
[corewithnocollection] Schema name=minimal
   [junit4]   2> 1226950 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection   x:corewithnocollection] o.a.s.s.IndexSchema Loaded 
schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1226951 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection   x:corewithnocollection] o.a.s.c.CoreContainer Creating 
SolrCore 'corewithnocollection' using configuration from collection 
corewithnocollection
   [junit4]   2> 1226957 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37446/data
   [junit4]   2> 1226958 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1226958 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.SolrCore [[corewithnocollection] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node4/corewithnocollection],
 
dataDir=[hdfs://localhost:37446/data/corewithnocollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/]
   [junit4]   2> 1226959 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:37446/data/corewithnocollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/snapshot_metadata
   [junit4]   2> 1226978 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 1226979 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 1226979 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
   [junit4]   2> 1227805 INFO  
(OverseerCollectionConfigSetProcessor-96920768026312715-127.0.0.1:52143_solr-n_0000000000)
 [n:127.0.0.1:52143_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 1228647 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1228656 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:37446/data/corewithnocollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003
   [junit4]   2> 1228694 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:37446/data/corewithnocollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/index
   [junit4]   2> 1228702 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 1228702 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 1228709 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1229027 INFO  (IPC Server handler 4 on 37446) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36702 is 
added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-b4025d97-671c-49b1-a857-af62f730cb9d:NORMAL:127.0.0.1:45133|RBW],
 
ReplicaUC[[DISK]DS-b7b2adcd-7dc2-4d4f-ac0a-e2e6c80efbc7:NORMAL:127.0.0.1:36702|RBW]]}
 size 71
   [junit4]   2> 1229037 INFO  (IPC Server handler 7 on 37446) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45133 is 
added to blk_1073741825_1001 size 71
   [junit4]   2> 1229480 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1229481 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1229481 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 1229498 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1229498 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1229595 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@34ab42d[corewithnocollection] main]
   [junit4]   2> 1229596 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf
   [junit4]   2> 1229596 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1229599 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1229599 INFO  
(searcherExecutor-3117-thread-1-processing-n:127.0.0.1:36630_solr 
x:corewithnocollection s:shard1 c:corewithnocollection r:core_node1) 
[n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 
x:corewithnocollection] o.a.s.c.SolrCore [corewithnocollection] Registered new 
searcher Searcher@34ab42d[corewithnocollection] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1229599 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1550732332839731200
   [junit4]   2> 1229612 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1229612 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1229612 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:36630/solr/corewithnocollection/
   [junit4]   2> 1229612 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1229612 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.SyncStrategy http://127.0.0.1:36630/solr/corewithnocollection/ has no 
replicas
   [junit4]   2> 1229625 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:36630/solr/corewithnocollection/ shard1
   [junit4]   2> 1229778 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1229780 INFO  (qtp1903229075-5585) [n:127.0.0.1:36630_solr 
c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003&collection.configName=conf&name=corewithnocollection&action=CREATE&numShards=1&collection=&wt=javabin&version=2}
 status=0 QTime=3875
   [junit4]   2> 1229783 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr    
] o.a.s.h.a.CoreAdminOperation core create command 
dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003&collection.configName=conf&name=corewithnocollection2&action=CREATE&numShards=1&wt=javabin&version=2
   [junit4]   2> 1229787 INFO  
(OverseerStateUpdate-96920768026312715-127.0.0.1:52143_solr-n_0000000000) 
[n:127.0.0.1:52143_solr    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 1230804 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2   x:corewithnocollection2] o.a.s.c.SolrConfig Using 
Lucene MatchVersion: 6.4.0
   [junit4]   2> 1230813 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2   x:corewithnocollection2] o.a.s.s.IndexSchema 
[corewithnocollection2] Schema name=minimal
   [junit4]   2> 1230816 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2   x:corewithnocollection2] o.a.s.s.IndexSchema Loaded 
schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1230817 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2   x:corewithnocollection2] o.a.s.c.CoreContainer 
Creating SolrCore 'corewithnocollection2' using configuration from collection 
corewithnocollection2
   [junit4]   2> 1230818 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37446/data
   [junit4]   2> 1230818 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1230818 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.SolrCore [[corewithnocollection2] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node4/corewithnocollection2],
 
dataDir=[hdfs://localhost:37446/data/corewithnocollection2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/]
   [junit4]   2> 1230818 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:37446/data/corewithnocollection2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/snapshot_metadata
   [junit4]   2> 1230828 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 1230828 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 1230854 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1230856 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:37446/data/corewithnocollection2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003
   [junit4]   2> 1230923 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:37446/data/corewithnocollection2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/index
   [junit4]   2> 1230931 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 1230931 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 1230937 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1231014 INFO  (IPC Server handler 5 on 37446) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45133 is 
added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-f08d4d89-0367-4ff9-83f2-140475a4c65c:NORMAL:127.0.0.1:36702|RBW],
 
ReplicaUC[[DISK]DS-b4025d97-671c-49b1-a857-af62f730cb9d:NORMAL:127.0.0.1:45133|RBW]]}
 size 0
   [junit4]   2> 1231015 INFO  (IPC Server handler 5 on 37446) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36702 is 
added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-f08d4d89-0367-4ff9-83f2-140475a4c65c:NORMAL:127.0.0.1:36702|RBW],
 
ReplicaUC[[DISK]DS-b4025d97-671c-49b1-a857-af62f730cb9d:NORMAL:127.0.0.1:45133|RBW]]}
 size 0
   [junit4]   2> 1231059 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1231059 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1231059 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] 
o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
   [junit4]   2> 1231084 INFO  (qtp1903229075-5572) [n:127.0.0.1:36630_solr 
c:corewithnocollection2 s:shard1 r:core_node1 x:corewithn

[...truncated too long message...]

[B8961A865EFDEDDF]) [    ] o.e.j.s.ServerConnector Stopped 
ServerConnector@710f30a4{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 4528214 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationStartStop-seed#[B8961A865EFDEDDF])
 [    ] o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@56ee4cf9{/,null,UNAVAILABLE}
   [junit4]   2> 4528215 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationStartStop-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.SolrTestCaseJ4 ###Ending testReplicationStartStop
   [junit4]   2> 4528216 INFO  
(TEST-CdcrReplicationDistributedZkTest.testReplicationStartStop-seed#[B8961A865EFDEDDF])
 [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:52193 52193
   [junit4]   2> 4528322 INFO  (Thread-112192) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:52193 52193
   [junit4]   2> 4528323 WARN  (Thread-112192) [    ] o.a.s.c.ZkTestServer 
Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        26      /solr/aliases.json
   [junit4]   2>        16      /solr/configs/conf1
   [junit4]   2>        8       /solr/collections/tmp_collection/state.json
   [junit4]   2>        8       /solr/security.json
   [junit4]   2>        3       
/solr/collections/target_collection/leaders/shard2/leader
   [junit4]   2>        3       
/solr/collections/target_collection/leaders/shard1/leader
   [junit4]   2>        2       
/solr/collections/source_collection/leaders/shard1/leader
   [junit4]   2>        2       
/solr/collections/source_collection/leaders/shard2/leader
   [junit4]   2>        2       
/solr/collections/tmp_collection/leaders/shard2/leader
   [junit4]   2>        2       
/solr/collections/tmp_collection/leaders/shard1/leader
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        26      /solr/clusterprops.json
   [junit4]   2>        26      /solr/clusterstate.json
   [junit4]   2>        8       
/solr/collections/tmp_collection/cdcr/state/process
   [junit4]   2>        8       
/solr/collections/tmp_collection/cdcr/state/buffer
   [junit4]   2>        6       
/solr/collections/target_collection/cdcr/state/buffer
   [junit4]   2>        6       
/solr/collections/target_collection/cdcr/state/process
   [junit4]   2>        5       
/solr/collections/source_collection/cdcr/state/process
   [junit4]   2>        4       /solr/collections/source_collection/state.json
   [junit4]   2>        4       
/solr/collections/source_collection/cdcr/state/buffer
   [junit4]   2>        4       /solr/collections/target_collection/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        175     /solr/overseer/collection-queue-work
   [junit4]   2>        63      /solr/overseer/queue
   [junit4]   2>        26      /solr/live_nodes
   [junit4]   2>        26      /solr/collections
   [junit4]   2>        21      /solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=CdcrReplicationDistributedZkTest 
-Dtests.method=testReplicationStartStop -Dtests.seed=B8961A865EFDEDDF 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=es-MX -Dtests.timezone=US/Hawaii -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE  188s J0 | 
CdcrReplicationDistributedZkTest.testReplicationStartStop <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Timeout while trying 
to assert number of documents @ target_collection
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:3B55C4154A256656]:0)
   [junit4]    >        at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
   [junit4]    >        at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationStartStop(CdcrReplicationDistributedZkTest.java:173)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.AssertionError: expected:<10> but was:<6>
   [junit4]    >        at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
   [junit4]    >        ... 42 more
   [junit4]   2> 4528325 INFO  
(SUITE-CdcrReplicationDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [    ] 
o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_B8961A865EFDEDDF-001
   [junit4]   2> Nov 11, 2016 8:41:36 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
{_version_=PostingsFormat(name=LuceneVarGapDocFreqInterval), 
id=PostingsFormat(name=Asserting)}, docValues:{}, maxPointsInLeafNode=632, 
maxMBSortInHeap=6.508491660832644, 
sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=es-MX, 
timezone=US/Hawaii
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_102 (64-bit)/cpus=4,threads=1,free=220688288,total=522190848
   [junit4]   2> NOTE: All tests run in this JVM: 
[CloudExitableDirectoryReaderTest, CacheHeaderTest, 
DistribDocExpirationUpdateProcessorTest, NumericFieldsTest, TestLazyCores, 
FullHLLTest, PeerSyncTest, MergeStrategyTest, TestInitQParser, 
TestSchemaResource, DistributedExpandComponentTest, TestSolrCoreProperties, 
TestBinaryResponseWriter, TriLevelCompositeIdRoutingTest, RequiredFieldsTest, 
UniqFieldsUpdateProcessorFactoryTest, QueryResultKeyTest, TestMacros, 
TestExclusionRuleCollectionAccess, TestPHPSerializedResponseWriter, 
TestReplicationHandler, TestRandomDVFaceting, FastVectorHighlighterTest, 
GraphQueryTest, SSLMigrationTest, DeleteLastCustomShardedReplicaTest, 
TestPerFieldSimilarity, TestConfigSetImmutable, TestSchemaManager, TestNRTOpen, 
TestReplicationHandlerBackup, PathHierarchyTokenizerFactoryTest, PolyFieldTest, 
UUIDFieldTest, AnalyticsMergeStrategyTest, ReturnFieldsTest, 
TestExactStatsCache, TestBlobHandler, MultiThreadedOCPTest, 
SuggestComponentTest, AsyncCallRequestStatusResponseTest, 
CollectionTooManyReplicasTest, TestRandomFaceting, 
TestPerFieldSimilarityClassic, LeaderElectionContextKeyTest, 
AnalysisAfterCoreReloadTest, TestSubQueryTransformerDistrib, 
HdfsCollectionsAPIDistributedZkTest, TestFaceting, HdfsBasicDistributedZkTest, 
ConcurrentDeleteAndCreateCollectionTest, FileBasedSpellCheckerTest, 
TestCrossCoreJoin, TestHdfsUpdateLog, TestRandomRequestDistribution, 
RuleEngineTest, HighlighterMaxOffsetTest, DisMaxRequestHandlerTest, 
TestRemoteStreaming, TestDocumentBuilder, TestSolrFieldCacheMBean, 
ActionThrottleTest, TestClusterProperties, RecoveryAfterSoftCommitTest, 
TestDocSet, DocumentAnalysisRequestHandlerTest, TestManagedSchema, 
TestFilteredDocIdSet, OverseerStatusTest, CursorPagingTest, TestJmxIntegration, 
TestOrdValues, TestRecoveryHdfs, DistributedVersionInfoTest, SolrCoreTest, 
TestShortCircuitedRequests, RegexBoostProcessorTest, BlockCacheTest, 
TestFieldCacheWithThreads, TestSolrConfigHandler, AtomicUpdatesTest, 
CurrencyFieldOpenExchangeTest, DistributedSuggestComponentTest, 
TestSolrDeletionPolicy2, TestOmitPositions, TestCloudPivotFacet, 
TestDistributedSearch, TestPivotHelperCode, CheckHdfsIndexTest, 
QueryElevationComponentTest, TestFoldingMultitermQuery, 
TestRuleBasedAuthorizationPlugin, TestComponentsName, 
TestRandomCollapseQParserPlugin, TestConfigSetsAPIExclusivity, 
AlternateDirectoryTest, TestQuerySenderNoQuery, ZkNodePropsTest, 
LeaderElectionIntegrationTest, TestBinaryResponseWriter, ClusterStateTest, 
HdfsRecoveryZkTest, DirectoryFactoryTest, TestRequestForwarding, 
TestMiniSolrCloudClusterSSL, SimpleFacetsTest, TestSolr4Spatial, 
SolrCmdDistributorTest, BadIndexSchemaTest, TestSort, TestFiltering, 
TestFunctionQuery, BasicFunctionalityTest, DirectUpdateHandlerTest, 
TestBadConfig, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, 
TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, SolrRequestParserTest, 
DocValuesTest, TestStressLucene, TestCSVLoader, WordBreakSolrSpellCheckerTest, 
SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, 
TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, 
TestAtomicUpdateErrorCases, QueryEqualityTest, DefaultValueUpdateProcessorTest, 
SortByFunctionTest, DocValuesMultiTest, SolrInfoMBeanTest, 
XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, 
LukeRequestHandlerTest, StandardRequestHandlerTest, TestWriterPerf, 
DirectSolrSpellCheckerTest, TestQueryTypes, PrimitiveFieldTypeTest, 
RequestHandlersTest, TestValueSourceCache, TermVectorComponentTest, 
TestIndexingPerformance, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, 
IndexSchemaTest, TestCollationField, UpdateRequestProcessorFactoryTest, 
MBeansHandlerTest, TestAnalyzedSuggestions, BinaryUpdateRequestHandlerTest, 
TestPartialUpdateDeduplication, SearchHandlerTest, HighlighterConfigTest, 
SOLR749Test, TestSolrIndexConfig, CopyFieldTest, SolrIndexConfigTest, 
TestStressRecovery, PreAnalyzedFieldTest, TestSystemIdResolver, DateFieldTest, 
SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, TestLRUCache, 
TestUtils, CircularListTest, TestRTGBase, SolrTestCaseJ4Test, 
TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, 
TestSimpleTrackingShardHandler, TestTolerantSearch, 
TestEmbeddedSolrServerConstructors, TestJettySolrRunner, 
BaseCdcrDistributedZkTest, CdcrReplicationDistributedZkTest]
   [junit4] Completed [501/651 (3!)] on J0 in 1676.63s, 11 tests, 1 failure <<< 
FAILURES!

[...truncated 61103 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to