Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/841/

1 tests failed.
FAILED:  
org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=63513, 
name=coreZkRegister-4791-thread-1, state=RUNNABLE, 
group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=63513, name=coreZkRegister-4791-thread-1, 
state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([EAFA91E638D68228]:0)
        at 
org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2126)
        at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:433)
        at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
        at 
org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
        at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11075 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/init-core-data-001
   [junit4]   2> 2564342 INFO  
(SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[EAFA91E638D68228]-worker)
 [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system 
property: /m_ovg/ey
   [junit4]   2> 2564344 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2564344 INFO  (Thread-54620) [    ] o.a.s.c.ZkTestServer 
client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2564344 INFO  (Thread-54620) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 2564444 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.ZkTestServer start zk server on port:58570
   [junit4]   2> 2564444 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2564445 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2564447 INFO  (zkCallback-2554-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@3b1539b4 
name:ZooKeeperConnection Watcher:127.0.0.1:58570 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2564447 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2564447 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2564447 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2564450 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2564450 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2564451 INFO  (zkCallback-2555-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@2d4756df 
name:ZooKeeperConnection Watcher:127.0.0.1:58570/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2564451 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2564451 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2564451 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2564453 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2564454 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2564455 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2564456 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 2564456 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2564458 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 2564458 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2564460 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2564460 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2564462 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 2564462 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2564463 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 2564463 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2564464 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 2564465 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2564466 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2564466 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2564467 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2564468 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2564469 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2564469 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2564470 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2564471 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2564472 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 2564473 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2564780 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1
   [junit4]   2> 2564781 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2564783 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@66e4c6c3{/m_ovg/ey,null,AVAILABLE}
   [junit4]   2> 2564785 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.ServerConnector Started 
ServerConnector@3219d2e3{HTTP/1.1}{127.0.0.1:51724}
   [junit4]   2> 2564785 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.Server Started @2567525ms
   [junit4]   2> 2564785 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/tempDir-001/control/data,
 hostContext=/m_ovg/ey, hostPort=51724, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores}
   [junit4]   2> 2564785 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 2564785 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/'
   [junit4]   2> 2564786 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr 
(NoInitialContextEx)
   [junit4]   2> 2564786 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not 
find system property or JNDI)
   [junit4]   2> 2564801 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2564801 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2564802 INFO  (zkCallback-2556-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@fa646db name:ZooKeeperConnection 
Watcher:127.0.0.1:58570/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 2564803 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2564803 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2564804 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 2564804 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/solr.xml
   [junit4]   2> 2564808 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores
   [junit4]   2> 2564808 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.CoreContainer New CoreContainer 1048032233
   [junit4]   2> 2564808 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/]
   [junit4]   2> 2564809 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/lib
   [junit4]   2> 2564809 WARN  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/lib).
   [junit4]   2> 2564816 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 
90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 
20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 
2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : 
false,useRetries : false,
   [junit4]   2> 2564817 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with 
params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2564817 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2564817 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2564817 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58570/solr
   [junit4]   2> 2564817 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2564817 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2564818 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2564819 INFO  (zkCallback-2558-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6cbddf53 
name:ZooKeeperConnection Watcher:127.0.0.1:58570 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2564819 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2564819 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2564821 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.ConnectionManager Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 2564821 INFO  
(zkCallback-2559-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@57814907 
name:ZooKeeperConnection Watcher:127.0.0.1:58570/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2564821 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.ConnectionManager Client is 
connected to ZooKeeper
   [junit4]   2> 2564822 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/queue
   [junit4]   2> 2564824 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 2564826 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 2564828 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 2564829 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 2564831 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2564832 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/aliases.json
   [junit4]   2> 2564833 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/clusterstate.json
   [junit4]   2> 2564834 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/security.json
   [junit4]   2> 2564835 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.ZkController Register node as live 
in ZooKeeper:/live_nodes/127.0.0.1:51724_m_ovg%2Fey
   [junit4]   2> 2564835 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:51724_m_ovg%2Fey
   [junit4]   2> 2564837 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect
   [junit4]   2> 2564838 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 2564839 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2564839 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.LeaderElector Joined leadership 
election with path: 
/overseer_elect/election/94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000
   [junit4]   2> 2564839 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.OverseerElectionContext I am going 
to be the leader 127.0.0.1:51724_m_ovg%2Fey
   [junit4]   2> 2564839 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/leader
   [junit4]   2> 2564840 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.Overseer Overseer 
(id=94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) starting
   [junit4]   2> 2564842 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/queue-work
   [junit4]   2> 2564846 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.OverseerAutoReplicaFailoverThread 
Starting OverseerAutoReplicaFailoverThread 
autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2564847 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.ZkStateReader Updating cluster 
state from ZooKeeper... 
   [junit4]   2> 2564847 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.Overseer Starting to work on the 
main queue
   [junit4]   2> 2564847 INFO  
(OverseerCollectionConfigSetProcessor-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000)
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.OverseerTaskProcessor Process 
current queue of overseer operations
   [junit4]   2> 2564859 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.CoreContainer Security conf doesn't 
exist. Skipping setup for authorization module.
   [junit4]   2> 2564859 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.CoreContainer No authentication 
plugin used.
   [junit4]   2> 2564860 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.CoresLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores
   [junit4]   2> 2564861 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1,
 collection=control_collection, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2564861 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.CoresLocator Found core collection1 
in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/
   [junit4]   2> 2564861 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.CoresLocator Found 1 core 
definitions
   [junit4]   2> 2564862 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 2564862 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.s.SolrDispatchFilter 
SolrDispatchFilter.init() done
   [junit4]   2> 2564862 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2564862 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.ZkController numShards not found on descriptor - reading it from system 
property
   [junit4]   2> 2564863 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.Overseer processMessage: queueSize: 
1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51724/m_ovg/ey";,
   [junit4]   2>   "node_name":"127.0.0.1:51724_m_ovg%2Fey",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2564864 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ReplicaMutator Update state 
numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51724/m_ovg/ey";,
   [junit4]   2>   "node_name":"127.0.0.1:51724_m_ovg%2Fey",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2564864 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2564864 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ClusterStateMutator building a new 
cName: control_collection
   [junit4]   2> 2564865 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ReplicaMutator Assigning new node 
to shard shard=shard1
   [junit4]   2> 2564866 INFO  
(zkCallback-2559-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.ZkStateReader A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2565864 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2565864 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2565865 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2565865 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 2565866 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 2565866 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/'
   [junit4]   2> 2565866 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2565866 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 2565882 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2565886 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2565891 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 2565908 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2565910 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 2565914 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2566008 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Initialized with 
rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2566015 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2566016 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2566024 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2566027 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2566029 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 2566029 WARN  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2566030 WARN  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2566030 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 2566030 WARN  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 2566030 WARN  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 2566031 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 2566031 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection control_collection
   [junit4]   2> 2566032 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2566032 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/],
 dataDir=[null]
   [junit4]   2> 2566032 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr 
mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2fbf099b
   [junit4]   2> 2566033 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/data
   [junit4]   2> 2566033 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore New index directory detected: old=null 
new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2566033 WARN  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/data/index'
 doesn't exist. Creating new index...
   [junit4]   2> 2566033 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/data/index
   [junit4]   2> 2566033 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=43, maxMergeAtOnceExplicit=23, maxMergedSegmentMB=94.7294921875, 
floorSegmentMB=1.4091796875, forceMergeDeletesPctAllowed=29.161398557175048, 
segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.17260107164071414
   [junit4]   2> 2566034 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>        
commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@54704b07 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ae46c8e; 
maxCacheMB=0.3974609375 
maxMergeSizeMB=0.712890625)),segFN=segments_1,generation=1}
   [junit4]   2> 2566034 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2566035 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup 
for core collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/data/
   [junit4]   2> 2566035 WARN  
(OldIndexDirectoryCleanupThreadForCore-collection1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.DirectoryFactory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/control-001/cores/collection1/data/
 does not point to a valid data directory; skipping clean-up of old index 
directories.
   [junit4]   2> 2566038 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2566039 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "dedupe"
   [junit4]   2> 2566039 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2566039 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2566039 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2566039 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2566039 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2566039 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 2566040 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as 
default, creating implicit default
   [junit4]   2> 2566040 WARN  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler 
{type = requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 2566042 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2566043 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2566044 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2566045 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2566048 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/dump,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 2566048 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2566049 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 2566049 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= 
defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 
numVersionBuckets=65536
   [junit4]   2> 2566050 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 2566050 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 2566050 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=34, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=11.947265625, 
floorSegmentMB=0.427734375, forceMergeDeletesPctAllowed=10.790841346503457, 
segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2566051 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>        
commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@54704b07 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ae46c8e; 
maxCacheMB=0.3974609375 
maxMergeSizeMB=0.712890625)),segFN=segments_1,generation=1}
   [junit4]   2> 2566051 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 2566051 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@2a5a77cb[collection1] 
main
   [junit4]   2> 2566051 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 2566052 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection 
configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2566052 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based 
storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2566052 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO 
with znodeBase: /configs/conf1
   [junit4]   2> 2566052 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2566052 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2566053 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 2566053 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2566053 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2566053 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2566054 INFO  
(searcherExecutor-4754-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher 
Searcher@2a5a77cb[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2566054 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed 
version buckets
   [junit4]   2> 2566054 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 
65536 version buckets from index
   [junit4]   2> 2566054 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed 
version bucket highest value from index
   [junit4]   2> 2566054 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent 
updates, using new clock 1516826643176357888
   [junit4]   2> 2566056 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with 
highest version 1516826643176357888
   [junit4]   2> 2566056 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2566057 INFO  
(coreLoadExecutor-4753-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2566057 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController Register replica - core:collection1 
address:http://127.0.0.1:51724/m_ovg/ey collection:control_collection 
shard:shard1
   [junit4]   2> 2566058 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.SolrZkClient makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2566061 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: 
/collections/control_collection/leader_elect/shard1/election/94801665086521348-core_node1-n_0000000000
   [junit4]   2> 2566062 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process 
for shard shard1
   [junit4]   2> 2566063 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to 
continue.
   [junit4]   2> 2566063 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try 
and sync
   [junit4]   2> 2566063 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SyncStrategy Sync replicas to 
http://127.0.0.1:51724/m_ovg/ey/collection1/
   [junit4]   2> 2566063 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2566063 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.Overseer processMessage: queueSize: 
1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2566063 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:51724/m_ovg/ey/collection1/ has no replicas
   [junit4]   2> 2566064 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:51724/m_ovg/ey/collection1/ shard1
   [junit4]   2> 2566064 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.SolrZkClient makePath: 
/collections/control_collection/leaders/shard1
   [junit4]   2> 2566066 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader 
registration node
   [junit4]   2> 2566069 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.Overseer processMessage: queueSize: 
1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:51724/m_ovg/ey";,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2566172 INFO  
(zkCallback-2559-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.ZkStateReader A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2566220 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController We are 
http://127.0.0.1:51724/m_ovg/ey/collection1/ and leader is 
http://127.0.0.1:51724/m_ovg/ey/collection1/
   [junit4]   2> 2566220 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 
baseURL=http://127.0.0.1:51724/m_ovg/ey
   [junit4]   2> 2566220 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2566220 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2566220 INFO  
(coreZkRegister-4747-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey 
x:collection1 s:shard1 c:control_collection r:core_node1) 
[n:127.0.0.1:51724_m_ovg%2Fey c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading 
it from system property
   [junit4]   2> 2566222 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.Overseer processMessage: queueSize: 
1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51724/m_ovg/ey";,
   [junit4]   2>   "node_name":"127.0.0.1:51724_m_ovg%2Fey",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 2566223 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ReplicaMutator Update state 
numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51724/m_ovg/ey";,
   [junit4]   2>   "node_name":"127.0.0.1:51724_m_ovg%2Fey",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2566325 INFO  
(zkCallback-2559-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.ZkStateReader A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2566363 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2566364 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2566365 INFO  (zkCallback-2561-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7595647e 
name:ZooKeeperConnection Watcher:127.0.0.1:58570/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2566365 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2566366 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2566366 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2566368 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause 
connection loss:false
   [junit4]   2> 2566368 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with 
stateFormat=2
   [junit4]   2> 2566368 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2566369 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2566369 INFO  (zkCallback-2562-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@5e8f4a22 
name:ZooKeeperConnection Watcher:127.0.0.1:58570/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2566369 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2566370 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2566372 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.Overseer processMessage: queueSize: 
1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 2566372 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ClusterStateMutator building a new 
cName: collection1
   [junit4]   2> 2566475 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ZkStateWriter going to 
create_collection /collections/collection1/state.json
   [junit4]   2> 2566735 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/cores/collection1
   [junit4]   2> 2566736 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001
   [junit4]   2> 2566737 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 2566739 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@3217a7ce{/m_ovg/ey,null,AVAILABLE}
   [junit4]   2> 2566739 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.ServerConnector Started 
ServerConnector@2d7b005c{HTTP/1.1}{127.0.0.1:34427}
   [junit4]   2> 2566739 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.Server Started @2569480ms
   [junit4]   2> 2566739 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/tempDir-001/jetty1,
 solrconfig=solrconfig.xml, hostContext=/m_ovg/ey, hostPort=34427, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/cores}
   [junit4]   2> 2566740 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 2566740 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/'
   [junit4]   2> 2566740 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr 
(NoInitialContextEx)
   [junit4]   2> 2566740 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not 
find system property or JNDI)
   [junit4]   2> 2566758 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2566759 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2566760 INFO  (zkCallback-2563-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@3799738f 
name:ZooKeeperConnection Watcher:127.0.0.1:58570/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2566760 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2566760 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2566761 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 2566761 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/solr.xml
   [junit4]   2> 2566766 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/cores
   [junit4]   2> 2566766 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.CoreContainer New CoreContainer 1457516883
   [junit4]   2> 2566766 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/]
   [junit4]   2> 2566766 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/lib
   [junit4]   2> 2566767 WARN  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/lib).
   [junit4]   2> 2566775 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 
90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 
20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 
2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : 
false,useRetries : false,
   [junit4]   2> 2566776 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with 
params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2566776 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2566777 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2566777 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58570/solr
   [junit4]   2> 2566777 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2566777 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2566777 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2566778 INFO  (zkCallback-2565-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@3dbfaae4 
name:ZooKeeperConnection Watcher:127.0.0.1:58570 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2566779 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2566779 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2566781 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.c.ConnectionManager Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 2566781 INFO  
(zkCallback-2566-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@778d1116 
name:ZooKeeperConnection Watcher:127.0.0.1:58570/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2566781 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.c.ConnectionManager Client is 
connected to ZooKeeper
   [junit4]   2> 2566787 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.c.ZkStateReader Updating cluster 
state from ZooKeeper... 
   [junit4]   2> 2567793 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.ZkController Register node as live 
in ZooKeeper:/live_nodes/127.0.0.1:34427_m_ovg%2Fey
   [junit4]   2> 2567794 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:34427_m_ovg%2Fey
   [junit4]   2> 2567796 INFO  
(zkCallback-2566-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.c.ZkStateReader A live node change: 
WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has 
occurred - updating... (live nodes size: 1)
   [junit4]   2> 2567796 INFO  
(zkCallback-2559-thread-1-processing-n:127.0.0.1:51724_m_ovg%2Fey) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.c.ZkStateReader A live node change: 
WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has 
occurred - updating... (live nodes size: 1)
   [junit4]   2> 2567797 INFO  (zkCallback-2561-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live 
nodes size: 1)
   [junit4]   2> 2567799 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2567799 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.LeaderElector Joined leadership 
election with path: 
/overseer_elect/election/94801665086521353-127.0.0.1:34427_m_ovg%2Fey-n_0000000001
   [junit4]   2> 2567800 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.LeaderElector Watching path 
/overseer_elect/election/94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000
 to know if I could be the leader
   [junit4]   2> 2567811 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.CoreContainer Security conf doesn't 
exist. Skipping setup for authorization module.
   [junit4]   2> 2567811 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.CoreContainer No authentication 
plugin used.
   [junit4]   2> 2567811 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.CoresLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/cores
   [junit4]   2> 2567812 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/cores/collection1,
 collection=collection1, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2567812 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.CoresLocator Found core collection1 
in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001/shard-1-001/cores/collection1/
   [junit4]   2> 2567812 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.CoresLocator Found 1 core 
definitions
   [junit4]   2> 2567814 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0
   [junit4]   2> 2567814 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.s.SolrDispatchFilter 
SolrDispatchFilter.init() done
   [junit4]   2> 2567814 INFO  
(coreLoadExecutor-4764-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey c:collection1   x:collection1] 
o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2567814 INFO  
(coreLoadExecutor-4764-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey c:collection1   x:collection1] 
o.a.s.c.ZkController numShards not found on descriptor - reading it from system 
property
   [junit4]   2> 2567816 INFO  
(coreLoadExecutor-4764-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey c:collection1   x:collection1] 
o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 2567816 INFO  
(coreLoadExecutor-4764-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey c:collection1   x:collection1] 
o.a.s.c.c.ZkStateReader addZkWatch collection1
   [junit4]   2> 2567816 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.Overseer processMessage: queueSize: 
1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34427/m_ovg/ey";,
   [junit4]   2>   "node_name":"127.0.0.1:34427_m_ovg%2Fey",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 2567816 INFO  
(coreLoadExecutor-4764-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey c:collection1   x:collection1] 
o.a.s.c.c.ZkStateReader Add data for collection1 ver 0 
   [junit4]   2> 2567816 INFO  
(coreLoadExecutor-4764-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey c:collection1   x:collection1] 
o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2567816 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ReplicaMutator Update state 
numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:34427/m_ovg/ey";,
   [junit4]   2>   "node_name":"127.0.0.1:34427_m_ovg%2Fey",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2567816 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ReplicaMutator Collection already 
exists with numShards=1
   [junit4]   2> 2567817 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ReplicaMutator Assigning new node 
to shard shard=shard1
   [junit4]   2> 2567919 INFO  
(OverseerStateUpdate-94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000) 
[n:127.0.0.1:51724_m_ovg%2Fey    ] o.a.s.c.o.ZkStateWriter going to 
update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 2567919 INFO  
(zkCallback-2566-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.c.ZkStateReader A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json for collection collection1 has 
occurred - updating... (live nodes size: 2)
   [junit4]   2> 2567920 INFO  
(zkCallback-2566-thread-1-processing-n:127.0.0.1:34427_m_ovg%2Fey) 
[n:127.0.0.1:34427_m_ovg%2Fey    ] o.a.s.c.c.ZkStateReader Updating data for 
collectio

[...truncated too long message...]

nit4]   2>      at 
org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:388)
   [junit4]   2>        at 
org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:269)
   [junit4]   2>        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2>        at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2>        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2>        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2927937 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.ServerConnector Stopped 
ServerConnector@149e4a9b{HTTP/1.1}{127.0.0.1:44435}
   [junit4]   2> 2927937 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [    ] o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@571ba103{/m_ovg/ey,null,UNAVAILABLE}
   [junit4]   2> 2927939 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[EAFA91E638D68228])
 [n:127.0.0.1:44435_m_ovg%2Fey c:collection1 s:shard1 r:core_node2 
x:collection1] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58570 58570
   [junit4]   2> 2928149 INFO  (Thread-54620) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:58570 58570
   [junit4]   2> 2928151 WARN  (Thread-54620) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        6       /solr/aliases.json
   [junit4]   2>        5       /solr/security.json
   [junit4]   2>        5       /solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        6       /solr/clusterstate.json
   [junit4]   2>        4       /solr/collections/collection1/state.json
   [junit4]   2>        3       
/solr/overseer_elect/election/94801665086521348-127.0.0.1:51724_m_ovg%2Fey-n_0000000000
   [junit4]   2>        3       /solr/collections/all_in_lir/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        6       /solr/collections
   [junit4]   2>        4       /solr/live_nodes
   [junit4]   2>        3       /solr/overseer/queue
   [junit4]   2>        3       /solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest 
-Dtests.method=testRestartWithAllInLIR -Dtests.seed=EAFA91E638D68228 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=es_PE -Dtests.timezone=MIT -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR    364s J0 | 
LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes 
recoverying - waited for 330 seconds
   [junit4]    >        at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    >        at 
org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)Throwable #2: 
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=63513, name=coreZkRegister-4791-thread-1, 
state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([EAFA91E638D68228]:0)
   [junit4]    >        at 
org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2126)
   [junit4]    >        at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:433)
   [junit4]    >        at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    >        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    >        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    >        at 
org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    >        at 
org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    >        at 
org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    >        at 
org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    >        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    >        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    >        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2928161 INFO  
(SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[EAFA91E638D68228]-worker)
 [n:127.0.0.1:44435_m_ovg%2Fey c:collection1 s:shard1 r:core_node2 
x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_EAFA91E638D68228-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene60): {}, 
docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, 
locale=es_PE, timezone=MIT
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 
1.8.0_45 (64-bit)/cpus=4,threads=1,free=191435672,total=529006592
   [junit4]   2> NOTE: All tests run in this JVM: [TestLeaderElectionZkExpiry, 
FileUtilsTest, ResponseLogComponentTest, ExternalFileFieldSortTest, TestSolrJ, 
TestExpandComponent, TestMergePolicyConfig, TestReplicationHandler, 
DistributedSpellCheckComponentTest, TestDistributedSearch, 
BigEndianAscendingWordSerializerTest, TestJoin, DataDrivenBlockJoinTest, 
TestHighFrequencyDictionaryFactory, PingRequestHandlerTest, TestCSVLoader, 
RankQueryTest, TestSolrConfigHandler, TestDocBasedVersionConstraints, 
TestFunctionQuery, PeerSyncTest, HdfsChaosMonkeyNothingIsSafeTest, 
SyncSliceTest, CursorPagingTest, TestRangeQuery, TestSolrConfigHandlerCloud, 
VersionInfoTest, TestIBSimilarityFactory, OutputWriterTest, HttpPartitionTest, 
EnumFieldTest, OverseerStatusTest, TestPKIAuthenticationPlugin, 
TestSolrConfigHandlerConcurrent, DateFieldTest, TestArbitraryIndexDir, 
UUIDUpdateProcessorFallbackTest, HdfsThreadLeakTest, OverseerRolesTest, 
TestBlobHandler, ChangedSchemaMergeTest, TestSolr4Spatial2, 
CdcrReplicationDistributedZkTest, CachingDirectoryFactoryTest, 
TestInfoStreamLogging, TestAtomicUpdateErrorCases, TestRestManager, 
BasicZkTest, QueryElevationComponentTest, SpellingQueryConverterTest, 
CloneFieldUpdateProcessorFactoryTest, TestBadConfig, 
DistributedFacetPivotLargeTest, TestConfigSetsAPIExclusivity, 
DistribJoinFromCollectionTest, OpenExchangeRatesOrgProviderTest, 
TestValueSourceCache, TestFoldingMultitermQuery, TestConfig, 
TestMiniSolrCloudCluster, TestSolrXml, CurrencyFieldOpenExchangeTest, 
SmileWriterTest, ExitableDirectoryReaderTest, XsltUpdateRequestHandlerTest, 
CoreMergeIndexesAdminHandlerTest, TestAuthorizationFramework, 
PathHierarchyTokenizerFactoryTest, DistributedQueryElevationComponentTest, 
BadCopyFieldTest, LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [415/554] on J0 in 363.84s, 1 test, 1 error <<< FAILURES!

[...truncated 446 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:782:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:719:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:516:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1432:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:987:
 There were test failures: 554 suites (8 ignored), 2182 tests, 1 error, 31 
ignored (18 assumptions) [seed: EAFA91E638D68228]

Total time: 195 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to