Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21505/
Java: 32bit/jdk1.8.0_162 -client -XX:+UseParallelGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.search.join.TestCloudNestedDocsSort

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.search.join.TestCloudNestedDocsSort:     1) Thread[id=8035, 
name=qtp10177888-8035, state=TIMED_WAITING, group=TGRP-TestCloudNestedDocsSort] 
        at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.search.join.TestCloudNestedDocsSort: 
   1) Thread[id=8035, name=qtp10177888-8035, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([DB3A8ADC0CE6F582]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.search.join.TestCloudNestedDocsSort

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=8035, name=qtp10177888-8035, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=8035, name=qtp10177888-8035, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([DB3A8ADC0CE6F582]:0)




Build Log:
[...truncated 12441 lines...]
   [junit4] Suite: org.apache.solr.search.join.TestCloudNestedDocsSort
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/init-core-data-001
   [junit4]   2> 423391 WARN  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=8 numCloses=8
   [junit4]   2> 423391 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 423393 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 423393 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 423394 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001
   [junit4]   2> 423394 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 423395 INFO  (Thread-3143) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 423395 INFO  (Thread-3143) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 423396 ERROR (Thread-3143) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 423495 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.c.ZkTestServer start zk server on port:46143
   [junit4]   2> 423497 INFO  (zkConnectionManagerCallback-2669-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423504 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 423504 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 423511 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 423513 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 423513 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 423516 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 423516 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 423516 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 423517 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 423517 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 423517 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 423517 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@f55887{/solr,null,AVAILABLE}
   [junit4]   2> 423517 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 423517 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 423517 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 423517 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@1a5d40f{/solr,null,AVAILABLE}
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@c3699f{SSL,[ssl, 
http/1.1]}{127.0.0.1:43285}
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.e.j.s.Server Started @424878ms
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=43285}
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@160892e{SSL,[ssl, 
http/1.1]}{127.0.0.1:39563}
   [junit4]   2> 423518 ERROR (jetty-launcher-2666-thread-3) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.e.j.s.Server Started @424878ms
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T17:05:18.581Z
   [junit4]   2> 423518 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=39563}
   [junit4]   2> 423519 ERROR (jetty-launcher-2666-thread-5) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 423519 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 423519 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 423519 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 423519 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T17:05:18.582Z
   [junit4]   2> 423524 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 423524 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 423524 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 423524 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@16b31a9{/solr,null,AVAILABLE}
   [junit4]   2> 423528 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@84b4bd{/solr,null,AVAILABLE}
   [junit4]   2> 423528 INFO  (zkConnectionManagerCallback-2671-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423529 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 423529 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@16966ec{SSL,[ssl, 
http/1.1]}{127.0.0.1:38775}
   [junit4]   2> 423529 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.e.j.s.Server Started @424889ms
   [junit4]   2> 423529 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=38775}
   [junit4]   2> 423529 ERROR (jetty-launcher-2666-thread-4) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 423529 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 423529 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 423529 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 423529 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T17:05:18.592Z
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@56cbd7{SSL,[ssl, 
http/1.1]}{127.0.0.1:36479}
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.e.j.s.Server Started @424900ms
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=36479}
   [junit4]   2> 423540 ERROR (jetty-launcher-2666-thread-2) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 423540 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T17:05:18.603Z
   [junit4]   2> 423541 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@148fcc0{/solr,null,AVAILABLE}
   [junit4]   2> 423542 INFO  (zkConnectionManagerCallback-2675-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423559 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.e.j.s.AbstractConnector Started ServerConnector@1a5714d{SSL,[ssl, 
http/1.1]}{127.0.0.1:35869}
   [junit4]   2> 423560 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.e.j.s.Server Started @424920ms
   [junit4]   2> 423560 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=35869}
   [junit4]   2> 423560 ERROR (jetty-launcher-2666-thread-1) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 423560 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 
8.0.0
   [junit4]   2> 423560 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 423560 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 423559 INFO  (zkConnectionManagerCallback-2673-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423560 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 423560 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T17:05:18.623Z
   [junit4]   2> 423563 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 423565 INFO  (zkConnectionManagerCallback-2677-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423565 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 423574 INFO  (zkConnectionManagerCallback-2679-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423575 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 423577 INFO  (jetty-launcher-2666-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46143/solr
   [junit4]   2> 423579 INFO  (zkConnectionManagerCallback-2684-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423580 INFO  (jetty-launcher-2666-thread-4) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46143/solr
   [junit4]   2> 423581 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 
0x10076f6b1f00006, likely client has closed socket
   [junit4]   2> 423586 INFO  (zkConnectionManagerCallback-2688-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423590 INFO  
(zkConnectionManagerCallback-2691-thread-1-processing-n:127.0.0.1:38775_solr) 
[n:127.0.0.1:38775_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423593 INFO  
(zkConnectionManagerCallback-2693-thread-1-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423595 INFO  (jetty-launcher-2666-thread-5) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46143/solr
   [junit4]   2> 423595 INFO  (jetty-launcher-2666-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46143/solr
   [junit4]   2> 423606 INFO  (zkConnectionManagerCallback-2697-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423607 INFO  (zkConnectionManagerCallback-2701-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423612 INFO  (jetty-launcher-2666-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46143/solr
   [junit4]   2> 423618 INFO  
(zkConnectionManagerCallback-2703-thread-1-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423624 INFO  (zkConnectionManagerCallback-2709-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423635 INFO  
(zkConnectionManagerCallback-2706-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423636 INFO  (jetty-launcher-2666-thread-5) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 423638 INFO  (jetty-launcher-2666-thread-5) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:39563_solr
   [junit4]   2> 423640 INFO  (jetty-launcher-2666-thread-5) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.Overseer Overseer 
(id=72188395957256204-127.0.0.1:39563_solr-n_0000000000) starting
   [junit4]   2> 423643 INFO  
(zkConnectionManagerCallback-2712-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 423647 INFO  (jetty-launcher-2666-thread-2) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 423648 INFO  (jetty-launcher-2666-thread-2) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:36479_solr
   [junit4]   2> 423655 INFO  (jetty-launcher-2666-thread-1) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 423660 INFO  (jetty-launcher-2666-thread-1) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 423664 INFO  (jetty-launcher-2666-thread-1) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:35869_solr
   [junit4]   2> 423671 INFO  (jetty-launcher-2666-thread-5) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:39563_solr
   [junit4]   2> 423691 INFO  
(OverseerStateUpdate-72188395957256204-127.0.0.1:39563_solr-n_0000000000) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 423708 INFO  
(zkCallback-2711-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (1) -> (3)
   [junit4]   2> 423708 INFO  
(zkCallback-2705-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 423749 INFO  (jetty-launcher-2666-thread-3) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 423752 INFO  (jetty-launcher-2666-thread-3) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 423761 INFO  (jetty-launcher-2666-thread-3) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:43285_solr
   [junit4]   2> 423779 INFO  
(zkCallback-2702-thread-1-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 423780 INFO  
(zkCallback-2705-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 423780 INFO  
(zkCallback-2711-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 423797 INFO  (jetty-launcher-2666-thread-4) 
[n:127.0.0.1:38775_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (4)
   [junit4]   2> 423801 INFO  (jetty-launcher-2666-thread-4) 
[n:127.0.0.1:38775_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 423814 INFO  
(zkCallback-2692-thread-1-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (4)
   [junit4]   2> 423816 INFO  (jetty-launcher-2666-thread-4) 
[n:127.0.0.1:38775_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:38775_solr
   [junit4]   2> 423817 INFO  
(zkCallback-2702-thread-1-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 423819 INFO  
(zkCallback-2711-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 423819 INFO  
(zkCallback-2705-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 423820 INFO  
(zkCallback-2692-thread-1-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 423847 INFO  
(zkCallback-2690-thread-1-processing-n:127.0.0.1:38775_solr) 
[n:127.0.0.1:38775_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (4) -> (5)
   [junit4]   2> 423873 INFO  (jetty-launcher-2666-thread-5) 
[n:127.0.0.1:39563_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39563.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 423893 INFO  (jetty-launcher-2666-thread-5) 
[n:127.0.0.1:39563_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39563.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 423894 INFO  (jetty-launcher-2666-thread-5) 
[n:127.0.0.1:39563_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39563.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 423897 INFO  (jetty-launcher-2666-thread-5) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node5/.
   [junit4]   2> 423968 INFO  (jetty-launcher-2666-thread-2) 
[n:127.0.0.1:36479_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36479.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 423994 INFO  (jetty-launcher-2666-thread-2) 
[n:127.0.0.1:36479_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36479.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 423995 INFO  (jetty-launcher-2666-thread-2) 
[n:127.0.0.1:36479_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36479.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 423997 INFO  (jetty-launcher-2666-thread-2) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node2/.
   [junit4]   2> 424082 INFO  (jetty-launcher-2666-thread-4) 
[n:127.0.0.1:38775_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_38775.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 424084 INFO  (jetty-launcher-2666-thread-1) 
[n:127.0.0.1:35869_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_35869.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 424092 INFO  (jetty-launcher-2666-thread-4) 
[n:127.0.0.1:38775_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_38775.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 424093 INFO  (jetty-launcher-2666-thread-4) 
[n:127.0.0.1:38775_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_38775.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 424094 INFO  (jetty-launcher-2666-thread-4) 
[n:127.0.0.1:38775_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node4/.
   [junit4]   2> 424098 INFO  (jetty-launcher-2666-thread-1) 
[n:127.0.0.1:35869_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_35869.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 424099 INFO  (jetty-launcher-2666-thread-1) 
[n:127.0.0.1:35869_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_35869.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 424100 INFO  (jetty-launcher-2666-thread-1) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node1/.
   [junit4]   2> 424126 INFO  (jetty-launcher-2666-thread-3) 
[n:127.0.0.1:43285_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_43285.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 424137 INFO  (jetty-launcher-2666-thread-3) 
[n:127.0.0.1:43285_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_43285.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 424137 INFO  (jetty-launcher-2666-thread-3) 
[n:127.0.0.1:43285_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_43285.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 424139 INFO  (jetty-launcher-2666-thread-3) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node3/.
   [junit4]   2> 424267 INFO  (zkConnectionManagerCallback-2721-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 424270 INFO  (zkConnectionManagerCallback-2725-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 424271 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 424271 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46143/solr ready
   [junit4]   2> 424508 INFO  (qtp24143628-8039) [n:127.0.0.1:36479_solr    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=2&property.config=solrconfig-minimal.xml&property.schema=schema.xml&collection.configName=solrCloudCollectionConfig&name=collection1&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 424509 INFO  
(OverseerThreadFactory-1824-thread-1-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection 
collection1
   [junit4]   2> 424614 INFO  
(OverseerStateUpdate-72188395957256204-127.0.0.1:39563_solr-n_0000000000) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:35869/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 424616 INFO  
(OverseerStateUpdate-72188395957256204-127.0.0.1:39563_solr-n_0000000000) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:36479/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 424617 INFO  
(OverseerStateUpdate-72188395957256204-127.0.0.1:39563_solr-n_0000000000) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:39563/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 424619 INFO  
(OverseerStateUpdate-72188395957256204-127.0.0.1:39563_solr-n_0000000000) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:43285/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 424842 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=collection1&version=2&replicaType=NRT&property.config=solrconfig-minimal.xml&property.schema=schema.xml&coreNodeName=core_node5&name=collection1_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 424851 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 424906 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=collection1&version=2&replicaType=NRT&property.config=solrconfig-minimal.xml&property.schema=schema.xml&coreNodeName=core_node7&name=collection1_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 424906 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 424908 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=collection1&version=2&replicaType=NRT&property.config=solrconfig-minimal.xml&property.schema=schema.xml&coreNodeName=core_node8&name=collection1_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 424909 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=collection1&version=2&replicaType=NRT&property.config=solrconfig-minimal.xml&property.schema=schema.xml&coreNodeName=core_node3&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 424909 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 424909 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 425019 INFO  
(zkCallback-2702-thread-1-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 425019 INFO  
(zkCallback-2711-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 425019 INFO  
(zkCallback-2692-thread-1-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 425020 INFO  
(zkCallback-2711-thread-2-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 425019 INFO  
(zkCallback-2705-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 425020 INFO  
(zkCallback-2705-thread-2-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 425020 INFO  
(zkCallback-2692-thread-2-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 425020 INFO  
(zkCallback-2702-thread-2-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 425872 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 425914 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.s.IndexSchema [collection1_shard1_replica_n2] Schema name=test
   [junit4]   2> 425942 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 425949 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 425950 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 425968 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.s.IndexSchema [collection1_shard2_replica_n6] Schema name=test
   [junit4]   2> 425971 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=test
   [junit4]   2> 425971 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.s.IndexSchema [collection1_shard2_replica_n4] Schema name=test
   [junit4]   2> 426124 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 426149 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 426159 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 426164 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 426185 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n2' using 
configuration from collection collection1, trusted=true
   [junit4]   2> 426186 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36479.solr.core.collection1.shard1.replica_n2' (registry 
'solr.core.collection1.shard1.replica_n2') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 426186 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 426186 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.SolrCore [[collection1_shard1_replica_n2] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node2/collection1_shard1_replica_n2],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node2/./collection1_shard1_replica_n2/data/]
   [junit4]   2> 426191 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n4' using 
configuration from collection collection1, trusted=true
   [junit4]   2> 426192 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39563.solr.core.collection1.shard2.replica_n4' (registry 
'solr.core.collection1.shard2.replica_n4') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 426192 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 426192 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.c.SolrCore [[collection1_shard2_replica_n4] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node5/collection1_shard2_replica_n4],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node5/./collection1_shard2_replica_n4/data/]
   [junit4]   2> 426193 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=40, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=56.12890625, 
floorSegmentMB=0.6416015625, forceMergeDeletesPctAllowed=9.297839922902698, 
segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.48047945090854716
   [junit4]   2> 426194 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n6' using 
configuration from collection collection1, trusted=true
   [junit4]   2> 426194 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using 
configuration from collection collection1, trusted=true
   [junit4]   2> 426194 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_43285.solr.core.collection1.shard2.replica_n6' (registry 
'solr.core.collection1.shard2.replica_n6') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 426195 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_35869.solr.core.collection1.shard1.replica_n1' (registry 
'solr.core.collection1.shard1.replica_n1') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@108b7b4
   [junit4]   2> 426195 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 426195 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 426195 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.SolrCore [[collection1_shard2_replica_n6] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node3/collection1_shard2_replica_n6],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node3/./collection1_shard2_replica_n6/data/]
   [junit4]   2> 426195 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node1/collection1_shard1_replica_n1],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.join.TestCloudNestedDocsSort_DB3A8ADC0CE6F582-001/tempDir-001/node1/./collection1_shard1_replica_n1/data/]
   [junit4]   2> 426199 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=40, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=56.12890625, 
floorSegmentMB=0.6416015625, forceMergeDeletesPctAllowed=9.297839922902698, 
segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.48047945090854716
   [junit4]   2> 426200 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=40, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=56.12890625, 
floorSegmentMB=0.6416015625, forceMergeDeletesPctAllowed=9.297839922902698, 
segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.48047945090854716
   [junit4]   2> 426201 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=40, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=56.12890625, 
floorSegmentMB=0.6416015625, forceMergeDeletesPctAllowed=9.297839922902698, 
segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.48047945090854716
   [junit4]   2> 426293 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 426293 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 426295 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: 
minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.23714892635251755]
   [junit4]   2> 426296 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@1c42cbc[collection1_shard2_replica_n6] main]
   [junit4]   2> 426297 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 426297 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 426298 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 426320 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for 
shard shard2: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 426323 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 426323 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 426325 INFO  
(searcherExecutor-1837-thread-1-processing-n:127.0.0.1:43285_solr 
x:collection1_shard2_replica_n6 s:shard2 c:collection1 r:core_node8) 
[n:127.0.0.1:43285_solr c:collection1 s:shard2 r:core_node8 
x:collection1_shard2_replica_n6] o.a.s.c.SolrCore 
[collection1_shard2_replica_n6] Registered new searcher 
Searcher@1c42cbc[collection1_shard2_replica_n6] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 426325 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: 
minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.23714892635251755]
   [junit4]   2> 426325 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 426325 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 426325 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@236a97[collection1_shard1_replica_n2] main]
   [junit4]   2> 426327 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: 
minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.23714892635251755]
   [junit4]   2> 426327 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 426327 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 426327 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 426327 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@d2e85[collection1_shard1_replica_n1] main]
   [junit4]   2> 426328 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 426328 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 426329 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 426329 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 426330 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 426330 INFO  
(searcherExecutor-1835-thread-1-processing-n:127.0.0.1:36479_solr 
x:collection1_shard1_replica_n2 s:shard1 c:collection1 r:core_node5) 
[n:127.0.0.1:36479_solr c:collection1 s:shard1 r:core_node5 
x:collection1_shard1_replica_n2] o.a.s.c.SolrCore 
[collection1_shard1_replica_n2] Registered new searcher 
Searcher@236a97[collection1_shard1_replica_n2] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 426330 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: 
minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.23714892635251755]
   [junit4]   2> 426332 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.c.ZkShardTerms Failed to save terms, version is not match, retrying
   [junit4]   2> 426335 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 426335 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 426335 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.SyncStrategy Sync replicas to 
https://127.0.0.1:36479/solr/collection1_shard1_replica_n2/
   [junit4]   2> 426335 ERROR (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.SyncStrategy No UpdateLog found - cannot sync
   [junit4]   2> 426336 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we 
can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 426336 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 426336 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@20ca08[collection1_shard2_replica_n4] main]
   [junit4]   2> 426337 INFO  
(searcherExecutor-1838-thread-1-processing-n:127.0.0.1:35869_solr 
x:collection1_shard1_replica_n1 s:shard1 c:collection1 r:core_node3) 
[n:127.0.0.1:35869_solr c:collection1 s:shard1 r:core_node3 
x:collection1_shard1_replica_n1] o.a.s.c.SolrCore 
[collection1_shard1_replica_n1] Registered new searcher 
Searcher@d2e85[collection1_shard1_replica_n1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 426337 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 426337 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 426338 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 426339 INFO  
(searcherExecutor-1836-thread-1-processing-n:127.0.0.1:39563_solr 
x:collection1_shard2_replica_n4 s:shard2 c:collection1 r:core_node7) 
[n:127.0.0.1:39563_solr c:collection1 s:shard2 r:core_node7 
x:collection1_shard2_replica_n4] o.a.s.c.SolrCore 
[collection1_shard2_replica_n4] Registered new searcher 
Searcher@20ca08[collection1_shard2_replica_n4] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 426341 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:36479/solr/collection1_shard1_replica_n2/ shard1
   [junit4]   2> 426443 INFO  
(zkCallback-2702-thread-2-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426443 INFO  
(zkCallback-2705-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426443 INFO  
(zkCallback-2711-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426443 INFO  
(zkCallback-2702-thread-1-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426443 INFO  
(zkCallback-2705-thread-2-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426443 INFO  
(zkCallback-2692-thread-2-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426443 INFO  
(zkCallback-2711-thread-2-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426443 INFO  
(zkCallback-2692-thread-1-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426492 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 426600 INFO  
(zkCallback-2702-thread-1-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426600 INFO  
(zkCallback-2692-thread-1-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426600 INFO  
(zkCallback-2702-thread-2-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426600 INFO  
(zkCallback-2705-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426600 INFO  
(zkCallback-2711-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426600 INFO  
(zkCallback-2711-thread-2-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426600 INFO  
(zkCallback-2705-thread-2-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426600 INFO  
(zkCallback-2692-thread-2-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426777 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=collection1&version=2&replicaType=NRT&property.config=solrconfig-minimal.xml&property.schema=schema.xml&coreNodeName=core_node5&name=collection1_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin}
 status=0 QTime=1935
   [junit4]   2> 426820 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 426820 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 426821 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.SyncStrategy Sync replicas to 
https://127.0.0.1:43285/solr/collection1_shard2_replica_n6/
   [junit4]   2> 426821 ERROR (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.SyncStrategy No UpdateLog found - cannot sync
   [junit4]   2> 426822 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we 
can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 426822 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext Found all replicas participating in 
election, clear LIR
   [junit4]   2> 426824 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:43285/solr/collection1_shard2_replica_n6/ shard2
   [junit4]   2> 426925 INFO  
(zkCallback-2702-thread-2-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426925 INFO  
(zkCallback-2711-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426925 INFO  
(zkCallback-2702-thread-1-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426925 INFO  
(zkCallback-2711-thread-2-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426925 INFO  
(zkCallback-2692-thread-1-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426925 INFO  
(zkCallback-2705-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426925 INFO  
(zkCallback-2692-thread-2-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426925 INFO  
(zkCallback-2705-thread-2-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 426974 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 426976 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=collection1&version=2&replicaType=NRT&property.config=solrconfig-minimal.xml&property.schema=schema.xml&coreNodeName=core_node8&name=collection1_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin}
 status=0 QTime=2067
   [junit4]   2> 427077 INFO  
(zkCallback-2711-thread-2-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427077 INFO  
(zkCallback-2702-thread-1-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427077 INFO  
(zkCallback-2692-thread-1-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427077 INFO  
(zkCallback-2692-thread-2-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427077 INFO  
(zkCallback-2711-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427077 INFO  
(zkCallback-2705-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427077 INFO  
(zkCallback-2705-thread-2-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427077 INFO  
(zkCallback-2702-thread-2-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427341 INFO  (qtp10177888-8051) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=collection1&version=2&replicaType=NRT&property.config=solrconfig-minimal.xml&property.schema=schema.xml&coreNodeName=core_node3&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin}
 status=0 QTime=2432
   [junit4]   2> 427344 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores 
params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=collection1&version=2&replicaType=NRT&property.config=solrconfig-minimal.xml&property.schema=schema.xml&coreNodeName=core_node7&name=collection1_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin}
 status=0 QTime=2438
   [junit4]   2> 427348 INFO  (qtp24143628-8039) [n:127.0.0.1:36479_solr    ] 
o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 
30 seconds. Check all shard replicas
   [junit4]   2> 427445 INFO  
(zkCallback-2711-thread-1-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427445 INFO  
(zkCallback-2705-thread-2-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427445 INFO  
(zkCallback-2711-thread-2-processing-n:127.0.0.1:35869_solr) 
[n:127.0.0.1:35869_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427445 INFO  
(zkCallback-2705-thread-1-processing-n:127.0.0.1:36479_solr) 
[n:127.0.0.1:36479_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427445 INFO  
(zkCallback-2692-thread-1-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427445 INFO  
(zkCallback-2692-thread-2-processing-n:127.0.0.1:43285_solr) 
[n:127.0.0.1:43285_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427445 INFO  
(zkCallback-2702-thread-3-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 427445 INFO  
(zkCallback-2702-thread-2-processing-n:127.0.0.1:39563_solr) 
[n:127.0.0.1:39563_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/collection1/state.json] for collection [collection1] has 
occurred - updating... (live nodes size: [5])
   [junit4]   2> 428348 INFO  (qtp24143628-8039) [n:127.0.0.1:36479_solr    ] 
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections 
params={replicationFactor=2&property.config=solrconfig-minimal.xml&property.schema=schema.xml&collection.configName=solrCloudCollectionConfig&name=collection1&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2}
 status=0 QTime=3840
   [junit4]   2> 428349 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: 
collection1 failOnTimeout:true timeout (sec):30
   [junit4]   1> -
   [junit4]   1> replica:core_node3 rstate:active live:true
   [junit4]   1> replica:core_node5 rstate:active live:true
   [junit4]   1> replica:core_node7 rstate:active live:true
   [junit4]   1> replica:core_node8 rstate:active live:true
   [junit4]   1> no one is recoverying
   [junit4]   2> 428350 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 428455 INFO  (qtp10177888-8062) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1]  
webapp=/solr path=/update 
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:36479/solr/collection1_shard1_replica_n2/&wt=javabin&version=2}{add=[1321058528,
 -1058693297]} 0 26
   [junit4]   2> 428455 INFO  (qtp24143628-8031) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2]  
webapp=/solr path=/update params={wt=javabin&version=2}{add=[1321058528, 
-1058693297]} 0 95
   [junit4]   2> 428511 INFO  
(OverseerCollectionConfigSetProcessor-72188395957256204-127.0.0.1:39563_solr-n_0000000000)
 [n:127.0.0.1:39563_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: 
/overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may 
have disconnected from ZooKeeper
   [junit4]   2> 428607 INFO  (qtp8648848-8045) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n4]  
webapp=/solr path=/update 
params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:43285/solr/collection1_shard2_replica_n6/&wt=javabin&version=2}{add=[2032101441,
 1904804780, -2082306277, -1751658774, -1415156358, -601735152, -1616844440, 
-1759885443, 109378109, 2135447242]} 0 133
   [junit4]   2> 428607 INFO  (qtp31823412-8026) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6]  
webapp=/solr path=/update params={wt=javabin&version=2}{add=[2032101441, 
1904804780, -2082306277, -1751658774, -1415156358, -601735152, -1616844440, 
-1759885443, 109378109, 2135447242]} 0 241
   [junit4]   2> 428623 INFO  (qtp8648848-8044) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 428627 INFO  (qtp8648848-8044) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.u.SolrIndexWriter Calling setCommitData with 
IW:org.apache.solr.update.SolrIndexWriter@16b4e7d commitCommandVersion:0
   [junit4]   2> 428629 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 428629 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.u.SolrIndexWriter Calling setCommitData with 
IW:org.apache.solr.update.SolrIndexWriter@18eef85 commitCommandVersion:0
   [junit4]   2> 428630 INFO  (qtp31823412-8023) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 428631 INFO  (qtp31823412-8023) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.SolrIndexWriter Calling setCommitData with 
IW:org.apache.solr.update.SolrIndexWriter@1dae429 commitCommandVersion:0
   [junit4]   2> 428632 INFO  (qtp10177888-8062) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.u.DirectUpdateHandler2 start 
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 428632 INFO  (qtp10177888-8062) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.u.SolrIndexWriter Calling setCommitData with 
IW:org.apache.solr.update.SolrIndexWriter@751421 commitCommandVersion:0
   [junit4]   2> 428655 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@385dba[collection1_shard1_replica_n2] main]
   [junit4]   2> 428657 INFO  (qtp10177888-8062) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@e7f43e[collection1_shard1_replica_n1] main]
   [junit4]   2> 428659 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 428660 INFO  (qtp10177888-8062) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 428664 INFO  
(searcherExecutor-1838-thread-1-processing-n:127.0.0.1:35869_solr 
x:collection1_shard1_replica_n1 s:shard1 c:collection1 r:core_node3) 
[n:127.0.0.1:35869_solr c:collection1 s:shard1 r:core_node3 
x:collection1_shard1_replica_n1] o.a.s.c.SolrCore 
[collection1_shard1_replica_n1] Registered new searcher 
Searcher@e7f43e[collection1_shard1_replica_n1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C149)))}
   [junit4]   2> 428664 INFO  (qtp10177888-8062) [n:127.0.0.1:35869_solr 
c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1]  
webapp=/solr path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:43285/solr/collection1_shard2_replica_n6/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 32
   [junit4]   2> 428664 INFO  
(searcherExecutor-1835-thread-1-processing-n:127.0.0.1:36479_solr 
x:collection1_shard1_replica_n2 s:shard1 c:collection1 r:core_node5) 
[n:127.0.0.1:36479_solr c:collection1 s:shard1 r:core_node5 
x:collection1_shard1_replica_n2] o.a.s.c.SolrCore 
[collection1_shard1_replica_n2] Registered new searcher 
Searcher@385dba[collection1_shard1_replica_n2] 
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C149)))}
   [junit4]   2> 428666 INFO  (qtp24143628-8016) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2]  
webapp=/solr path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:43285/solr/collection1_shard2_replica_n6/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 37
   [junit4]   2> 428670 INFO  (qtp8648848-8044) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@1544a8d[collection1_shard2_replica_n4] main]
   [junit4]   2> 428670 INFO  (qtp31823412-8023) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@1a1a445[collection1_shard2_replica_n6] main]
   [junit4]   2> 428671 INFO  (qtp8648848-8044) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 428671 INFO  (qtp31823412-8023) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 428675 INFO  
(searcherExecutor-1837-thread-1-processing-n:127.0.0.1:43285_solr 
x:collection1_shard2_replica_n6 s:shard2 c:collection1 r:core_node8) 
[n:127.0.0.1:43285_solr c:collection1 s:shard2 r:core_node8 
x:collection1_shard2_replica_n6] o.a.s.c.SolrCore 
[collection1_shard2_replica_n6] Registered new searcher 
Searcher@1a1a445[collection1_shard2_replica_n6] 
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C508)
 Uninverting(_1(8.0.0):C224)))}
   [junit4]   2> 428675 INFO  
(searcherExecutor-1836-thread-1-processing-n:127.0.0.1:39563_solr 
x:collection1_shard2_replica_n4 s:shard2 c:collection1 r:core_node7) 
[n:127.0.0.1:39563_solr c:collection1 s:shard2 r:core_node7 
x:collection1_shard2_replica_n4] o.a.s.c.SolrCore 
[collection1_shard2_replica_n4] Registered new searcher 
Searcher@1544a8d[collection1_shard2_replica_n4] 
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C508)
 Uninverting(_1(8.0.0):C224)))}
   [junit4]   2> 428675 INFO  (qtp31823412-8023) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6]  
webapp=/solr path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:43285/solr/collection1_shard2_replica_n6/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 45
   [junit4]   2> 428675 INFO  (qtp8648848-8044) [n:127.0.0.1:39563_solr 
c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n4]  
webapp=/solr path=/update 
params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:43285/solr/collection1_shard2_replica_n6/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=}
 0 52
   [junit4]   2> 428676 INFO  (qtp31823412-8027) [n:127.0.0.1:43285_solr 
c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6] 
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6]  
webapp=/solr path=/update 
params={_stateVer_=collection1:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=}
 0 66
   [junit4]   2> 428719 INFO  
(TEST-TestCloudNestedDocsSort.test-seed#[DB3A8ADC0CE6F582]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 428740 INFO  (qtp24143628-8018) [n:127.0.0.1:36479_solr 
c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2] 
o.a.s.c.S.Request [collection1_shard1_replica_n2]  webapp=/solr path=/select 
params={df=text&distrib=false&_stateVer_=collection1:7&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&sort=val_s1+asc,+parent_id_s1+desc&shard.url=https://127.0.0.1:36479/solr/collection1_shard1_replica_n2/|https://127.0.0.1:35869/solr/collection1_shard1_replica_n1/&rows=881&version=2&q=%2Btype_s:child^%3D0+%2BparentFilter_s:(hnky+32cd+86kz)^%3D0+%2BchildFilter_s:(32cd+6oz3+a913+eiqc+1m)^%3D0&NOW=1519232723784&isShard=true&wt=javabin}
 hit

[...truncated too long message...]

hreadPool QueuedThreadPool@qtp21629779{STOPPING,8<=8<=10000,i=0,q=1} Couldn't 
stop Thread[qtp21629779-65,5,TGRP-TestCloudNestedDocsSort]
   [junit4]   2> 43574 ERROR 
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper 
server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 43575 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1:36653 36653
   [junit4]   2> 43598 INFO  (Thread-1) [    ] o.a.s.c.ZkTestServer connecting 
to 127.0.0.1:36653 36653
   [junit4]   2> 43598 WARN  (Thread-1) [    ] o.a.s.c.ZkTestServer Watch limit 
violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        6       /solr/aliases.json
   [junit4]   2>        5       /solr/collections/collection1/terms/shard2
   [junit4]   2>        5       /solr/security.json
   [junit4]   2>        5       /solr/collections/collection1/terms/shard1
   [junit4]   2>        4       /solr/configs/solrCloudCollectionConfig
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        40      /solr/collections/collection1/state.json
   [junit4]   2>        6       /solr/clusterstate.json
   [junit4]   2>        6       /solr/clusterprops.json
   [junit4]   2>        3       
/solr/overseer_elect/election/72188608133529612-127.0.0.1:42825_solr-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        6       /solr/live_nodes
   [junit4]   2>        6       /solr/collections
   [junit4]   2> 
   [junit4]   2> 43599 INFO  
(SUITE-TestCloudNestedDocsSort-seed#[DB3A8ADC0CE6F582]-worker) [    ] 
o.a.s.SolrTestCaseJ4 ------------------------------------------------------- 
Done waiting for tracked resources to be released
   [junit4]   2> Feb 21, 2018 5:59:59 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked 
thread(s).
   [junit4]   2> Feb 21, 2018 6:00:19 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at 
org.apache.solr.search.join.TestCloudNestedDocsSort: 
   [junit4]   2>    1) Thread[id=65, name=qtp21629779-65, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Feb 21, 2018 6:00:19 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=65, name=qtp21629779-65, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]
   [junit4]   2> Feb 21, 2018 6:00:22 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be 
terminated:
   [junit4]   2>    1) Thread[id=65, name=qtp21629779-65, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{childFilter_s=PostingsFormat(name=MockRandom), 
parentTie_s1=Lucene50(blocksize=128), type_s=PostingsFormat(name=MockRandom), 
parentFilter_s=FSTOrd50, multiDefault=FSTOrd50, 
parent_id_s1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))),
 _root_=PostingsFormat(name=MockRandom), val_s1=FSTOrd50, 
id=Lucene50(blocksize=128)}, 
docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), 
range_facet_i_dv=DocValuesFormat(name=Direct), 
intDvoDefault=DocValuesFormat(name=Memory), 
timestamp=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=891, 
maxMBSortInHeap=5.718724386902019, 
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@389567),
 locale=de, timezone=Europe/Brussels
   [junit4]   2> NOTE: Linux 4.13.0-32-generic i386/Oracle Corporation 
1.8.0_162 (32-bit)/cpus=8,threads=2,free=124268216,total=237502464
   [junit4]   2> NOTE: All tests run in this JVM: [TestCloudNestedDocsSort]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestCloudNestedDocsSort -Dtests.seed=DB3A8ADC0CE6F582 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de 
-Dtests.timezone=Europe/Brussels -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestCloudNestedDocsSort (suite) <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.search.join.TestCloudNestedDocsSort: 
   [junit4]    >    1) Thread[id=65, name=qtp21629779-65, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([DB3A8ADC0CE6F582]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=65, name=qtp21629779-65, state=TIMED_WAITING, 
group=TGRP-TestCloudNestedDocsSort]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([DB3A8ADC0CE6F582]:0)
   [junit4] Completed [5/5 (1!)] on J1 in 67.51s, 1 test, 2 errors <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1560: 
The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1085: 
There were test failures: 5 suites, 5 tests, 2 suite-level errors [seed: 
DB3A8ADC0CE6F582]

Total time: 1 minute 9 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   1/5 failed: org.apache.solr.search.join.TestCloudNestedDocsSort
[repro] git checkout a9f0272380438df88d29ed7c41572136f999f8db
HEAD is now at a9f0272... SOLR-12006: Add a '*_t' and '*_t_sort' dynamic field 
for single valued text fields
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
+ rm -f /tmp/tmp.AJiRsVdb85
Archiving artifacts
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
<Git Blamer> Using GitBlamer to create author and commit information for all 
warnings.
<Git Blamer> GIT_COMMIT=a9f0272380438df88d29ed7c41572136f999f8db, 
workspace=/var/lib/jenkins/workspace/Lucene-Solr-master-Linux
[WARNINGS] Computing warning deltas based on reference build #21504
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to