Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/1398/
Java: 32bit/jdk1.8.0_162 -client -XX:+UseParallelGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.search.join.BlockJoinFacetDistribTest

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.search.join.BlockJoinFacetDistribTest:     1) Thread[id=11161, 
name=qtp515070-11161, state=TIMED_WAITING, 
group=TGRP-BlockJoinFacetDistribTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.search.join.BlockJoinFacetDistribTest: 
   1) Thread[id=11161, name=qtp515070-11161, state=TIMED_WAITING, 
group=TGRP-BlockJoinFacetDistribTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([FE37B6B8804C4D10]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.search.join.BlockJoinFacetDistribTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=11161, name=qtp515070-11161, state=TIMED_WAITING, 
group=TGRP-BlockJoinFacetDistribTest]         at sun.misc.Unsafe.park(Native 
Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) 
        at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=11161, name=qtp515070-11161, state=TIMED_WAITING, 
group=TGRP-BlockJoinFacetDistribTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([FE37B6B8804C4D10]:0)


FAILED:  org.apache.lucene.index.TestStressNRT.test

Error Message:
MockDirectoryWrapper: cannot close: there are still 22 open files: {_9f.cfs=1, 
_9p.fdt=1, _9p_Asserting_0.tim=1, _9o.fdt=1, _9n.fdt=1, _9t.fdt=1, 
_9p_Asserting_0.doc=1, _9k_Asserting_0.tim=1, _9n_Asserting_0.tim=1, _9u.cfs=1, 
_9q_Asserting_0.tim=1, _9t_Asserting_0.tim=1, _9k.fdt=1, _9o_Asserting_0.doc=1, 
_9k_Asserting_0.doc=1, _9n_Asserting_0.doc=1, _9s.cfs=1, _9q_Asserting_0.doc=1, 
_9t_Asserting_0.doc=1, _9q.fdt=1, _96.cfs=1, _9o_Asserting_0.tim=1}

Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still 
22 open files: {_9f.cfs=1, _9p.fdt=1, _9p_Asserting_0.tim=1, _9o.fdt=1, 
_9n.fdt=1, _9t.fdt=1, _9p_Asserting_0.doc=1, _9k_Asserting_0.tim=1, 
_9n_Asserting_0.tim=1, _9u.cfs=1, _9q_Asserting_0.tim=1, _9t_Asserting_0.tim=1, 
_9k.fdt=1, _9o_Asserting_0.doc=1, _9k_Asserting_0.doc=1, _9n_Asserting_0.doc=1, 
_9s.cfs=1, _9q_Asserting_0.doc=1, _9t_Asserting_0.doc=1, _9q.fdt=1, _96.cfs=1, 
_9o_Asserting_0.tim=1}
        at 
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
        at org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:398)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _9k_Asserting_0.tim
        at 
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
        at 
org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
        at 
org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:138)
        at 
org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:445)
        at 
org.apache.lucene.codecs.asserting.AssertingPostingsFormat.fieldsProducer(AssertingPostingsFormat.java:58)
        at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:292)
        at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
        at 
org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:112)
        at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78)
        at 
org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:208)
        at 
org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:258)
        at 
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:105)
        at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:490)
        at 
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:293)
        at 
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:268)
        at 
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:258)
        at 
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
        at org.apache.lucene.index.TestStressNRT$1.run(TestStressNRT.java:160)




Build Log:
[...truncated 1111 lines...]
   [junit4] Suite: org.apache.lucene.index.TestStressNRT
   [junit4]   1> READER4: FAILED: unexpected exception
   [junit4]   1> java.lang.AssertionError: No documents or tombstones found for 
id 148, expected at least -55 
reader=StandardDirectoryReader(segments_1a:975:nrt _9f(7.3.0):c25/21:delGen=1 
_96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 _9k(7.3.0):C61/54:delGen=2 
_9o(7.3.0):C24/16:delGen=1 _9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 
_9s(7.3.0):c69/33:delGen=1 _9u(7.3.0):c55/26:delGen=1 
_9t(7.3.0):C71/37:delGen=1)
   [junit4]   1>        at org.junit.Assert.fail(Assert.java:93)
   [junit4]   1>        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:348)
   [junit4]   1> READER0: FAILED: unexpected exception
   [junit4]   1> java.lang.AssertionError: No documents or tombstones found for 
id 64, expected at least -61 reader=StandardDirectoryReader(segments_1a:975:nrt 
_9f(7.3.0):c25/21:delGen=1 _96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 
_9k(7.3.0):C61/54:delGen=2 _9o(7.3.0):C24/16:delGen=1 
_9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 _9s(7.3.0):c69/33:delGen=1 
_9u(7.3.0):c55/26:delGen=1 _9t(7.3.0):C71/37:delGen=1)
   [junit4]   1>        at org.junit.Assert.fail(Assert.java:93)
   [junit4]   1>        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:348)
   [junit4]   2> feb 21, 2018 3:16:24 AM 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> ADVERTENCIA: Uncaught exception in thread: 
Thread[READER4,5,TGRP-TestStressNRT]
   [junit4]   2> java.lang.RuntimeException: java.lang.AssertionError: No 
documents or tombstones found for id 148, expected at least -55 
reader=StandardDirectoryReader(segments_1a:975:nrt _9f(7.3.0):c25/21:delGen=1 
_96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 _9k(7.3.0):C61/54:delGen=2 
_9o(7.3.0):C24/16:delGen=1 _9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 
_9s(7.3.0):c69/33:delGen=1 _9u(7.3.0):c55/26:delGen=1 
_9t(7.3.0):C71/37:delGen=1)
   [junit4]   2>        at 
__randomizedtesting.SeedInfo.seed([1F4589E2D2DD0A22]:0)
   [junit4]   2>        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:377)
   [junit4]   2> Caused by: java.lang.AssertionError: No documents or 
tombstones found for id 148, expected at least -55 
reader=StandardDirectoryReader(segments_1a:975:nrt _9f(7.3.0):c25/21:delGen=1 
_96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 _9k(7.3.0):C61/54:delGen=2 
_9o(7.3.0):C24/16:delGen=1 _9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 
_9s(7.3.0):c69/33:delGen=1 _9u(7.3.0):c55/26:delGen=1 
_9t(7.3.0):C71/37:delGen=1)
   [junit4]   2>        at org.junit.Assert.fail(Assert.java:93)
   [junit4]   2>        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:348)
   [junit4]   2> 
   [junit4]   2> feb 21, 2018 3:16:24 AM 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> ADVERTENCIA: Uncaught exception in thread: 
Thread[READER0,5,TGRP-TestStressNRT]
   [junit4]   2> java.lang.RuntimeException: java.lang.AssertionError: No 
documents or tombstones found for id 64, expected at least -61 
reader=StandardDirectoryReader(segments_1a:975:nrt _9f(7.3.0):c25/21:delGen=1 
_96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 _9k(7.3.0):C61/54:delGen=2 
_9o(7.3.0):C24/16:delGen=1 _9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 
_9s(7.3.0):c69/33:delGen=1 _9u(7.3.0):c55/26:delGen=1 
_9t(7.3.0):C71/37:delGen=1)
   [junit4]   2>        at 
__randomizedtesting.SeedInfo.seed([1F4589E2D2DD0A22]:0)
   [junit4]   2>        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:377)
   [junit4]   2> Caused by: java.lang.AssertionError: No documents or 
tombstones found for id 64, expected at least -61 
reader=StandardDirectoryReader(segments_1a:975:nrt _9f(7.3.0):c25/21:delGen=1 
_96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 _9k(7.3.0):C61/54:delGen=2 
_9o(7.3.0):C24/16:delGen=1 _9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 
_9s(7.3.0):c69/33:delGen=1 _9u(7.3.0):c55/26:delGen=1 
_9t(7.3.0):C71/37:delGen=1)
   [junit4]   2>        at org.junit.Assert.fail(Assert.java:93)
   [junit4]   2>        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:348)
   [junit4]   2> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestStressNRT 
-Dtests.method=test -Dtests.seed=1F4589E2D2DD0A22 -Dtests.multiplier=3 
-Dtests.slow=true -Dtests.locale=es-DO -Dtests.timezone=Europe/Budapest 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   1.36s J2 | TestStressNRT.test <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: 
MockDirectoryWrapper: cannot close: there are still 22 open files: {_9f.cfs=1, 
_9p.fdt=1, _9p_Asserting_0.tim=1, _9o.fdt=1, _9n.fdt=1, _9t.fdt=1, 
_9p_Asserting_0.doc=1, _9k_Asserting_0.tim=1, _9n_Asserting_0.tim=1, _9u.cfs=1, 
_9q_Asserting_0.tim=1, _9t_Asserting_0.tim=1, _9k.fdt=1, _9o_Asserting_0.doc=1, 
_9k_Asserting_0.doc=1, _9n_Asserting_0.doc=1, _9s.cfs=1, _9q_Asserting_0.doc=1, 
_9t_Asserting_0.doc=1, _9q.fdt=1, _96.cfs=1, _9o_Asserting_0.tim=1}
   [junit4]    >        at 
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
   [junit4]    >        at 
org.apache.lucene.index.TestStressNRT.test(TestStressNRT.java:398)
   [junit4]    >        at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.lang.RuntimeException: unclosed IndexInput: 
_9k_Asserting_0.tim
   [junit4]    >        at 
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:732)
   [junit4]    >        at 
org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:776)
   [junit4]    >        at 
org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:138)
   [junit4]    >        at 
org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:445)
   [junit4]    >        at 
org.apache.lucene.codecs.asserting.AssertingPostingsFormat.fieldsProducer(AssertingPostingsFormat.java:58)
   [junit4]    >        at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:292)
   [junit4]    >        at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
   [junit4]    >        at 
org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:112)
   [junit4]    >        at 
org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78)
   [junit4]    >        at 
org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:208)
   [junit4]    >        at 
org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:258)
   [junit4]    >        at 
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:105)
   [junit4]    >        at 
org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:490)
   [junit4]    >        at 
org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:293)
   [junit4]    >        at 
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:268)
   [junit4]    >        at 
org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:258)
   [junit4]    >        at 
org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
   [junit4]    >        at 
org.apache.lucene.index.TestStressNRT$1.run(TestStressNRT.java:160)Throwable 
#2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an 
uncaught exception in thread: Thread[id=3397, name=READER4, state=RUNNABLE, 
group=TGRP-TestStressNRT]
   [junit4]    > Caused by: java.lang.RuntimeException: 
java.lang.AssertionError: No documents or tombstones found for id 148, expected 
at least -55 reader=StandardDirectoryReader(segments_1a:975:nrt 
_9f(7.3.0):c25/21:delGen=1 _96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 
_9k(7.3.0):C61/54:delGen=2 _9o(7.3.0):C24/16:delGen=1 
_9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 _9s(7.3.0):c69/33:delGen=1 
_9u(7.3.0):c55/26:delGen=1 _9t(7.3.0):C71/37:delGen=1)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([1F4589E2D2DD0A22]:0)
   [junit4]    >        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:377)
   [junit4]    > Caused by: java.lang.AssertionError: No documents or 
tombstones found for id 148, expected at least -55 
reader=StandardDirectoryReader(segments_1a:975:nrt _9f(7.3.0):c25/21:delGen=1 
_96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 _9k(7.3.0):C61/54:delGen=2 
_9o(7.3.0):C24/16:delGen=1 _9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 
_9s(7.3.0):c69/33:delGen=1 _9u(7.3.0):c55/26:delGen=1 
_9t(7.3.0):C71/37:delGen=1)
   [junit4]    >        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:348)Throwable 
#3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an 
uncaught exception in thread: Thread[id=3393, name=READER0, state=RUNNABLE, 
group=TGRP-TestStressNRT]
   [junit4]    > Caused by: java.lang.RuntimeException: 
java.lang.AssertionError: No documents or tombstones found for id 64, expected 
at least -61 reader=StandardDirectoryReader(segments_1a:975:nrt 
_9f(7.3.0):c25/21:delGen=1 _96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 
_9k(7.3.0):C61/54:delGen=2 _9o(7.3.0):C24/16:delGen=1 
_9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 _9s(7.3.0):c69/33:delGen=1 
_9u(7.3.0):c55/26:delGen=1 _9t(7.3.0):C71/37:delGen=1)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([1F4589E2D2DD0A22]:0)
   [junit4]    >        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:377)
   [junit4]    > Caused by: java.lang.AssertionError: No documents or 
tombstones found for id 64, expected at least -61 
reader=StandardDirectoryReader(segments_1a:975:nrt _9f(7.3.0):c25/21:delGen=1 
_96(7.3.0):c83/82:delGen=4 _9q(7.3.0):C96/63 _9k(7.3.0):C61/54:delGen=2 
_9o(7.3.0):C24/16:delGen=1 _9n(7.3.0):C43/35:delGen=1 _9p(7.3.0):C3/2 
_9s(7.3.0):c69/33:delGen=1 _9u(7.3.0):c55/26:delGen=1 
_9t(7.3.0):C71/37:delGen=1)
   [junit4]    >        at 
org.apache.lucene.index.TestStressNRT$2.run(TestStressNRT.java:348)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), 
sim=RandomSimilarity(queryNorm=false): {id=DFR I(F)2}, locale=es-DO, 
timezone=Europe/Budapest
   [junit4]   2> NOTE: Linux 4.13.0-32-generic i386/Oracle Corporation 
1.8.0_162 (32-bit)/cpus=8,threads=1,free=128265712,total=387973120
   [junit4]   2> NOTE: All tests run in this JVM: [TestSimilarity2, 
Test2BNumericDocValues, TestTopDocsCollector, TestFixedLengthBytesRefArray, 
TestIndexWriterLockRelease, TestAttributeSource, TestRollingBuffer, 
TestSearchForDuplicates, TestSameTokenSamePosition, TestStressIndexing2, 
TestIndexingSequenceNumbers, TestFastCompressionMode, TestCloseableThreadLocal, 
TestPerFieldPostingsFormat2, TestIndexWriterReader, TestIndexSearcher, 
TestScorerPerf, TestPositiveScoresOnlyCollector, 
TestSimpleExplanationsOfNonMatches, TestLucene60PointsFormat, TestSpans, 
TestFutureObjects, TestUsageTrackingFilterCachingPolicy, TestMinShouldMatch2, 
TestShardSearching, TestFieldsReader, TestFilterWeight, TestFilterDirectory, 
TestCustomTermFreq, TestSynonymQuery, TestFilterDirectoryReader, 
TestTermRangeQuery, TestSpanNearQuery, TestOrdinalMap, TestBlockPostingsFormat, 
TestFieldInfos, TestRollingUpdates, TestDocValuesRewriteMethod, 
TestReaderClosed, TestRegexpQuery, TestIntroSelector, 
TestAllFilesDetectTruncation, TestDateSort, TestOfflineSorter, 
TestReaderWrapperDVTypeCheck, TestConjunctions, TestCollectionUtil, 
TestSearchWithThreads, TestMultiMMap, TestMultiFields, 
TestConsistentFieldNumbers, TestSimpleFSDirectory, TestLongRange, 
TestOperations, TestTermVectors, TestIndexCommit, TestSpanTermQuery, 
TestRoaringDocIdSet, TestAssertions, TestSimilarityBase, TestIndexWriterCommit, 
TestCachingTokenFilter, TestBoolean2ScorerSupplier, TestPackedInts, 
TestNRTReaderWithThreads, TestGraphTokenStreamFiniteStrings, 
TestDirectoryReader, TestMultiset, TestHighCompressionMode, 
TestPayloadsOnVectors, TestSpanBoostQuery, TestStressDeletes, TestStopFilter, 
TestManyFields, TestSetOnce, TestCompiledAutomaton, TestPrefixQuery, 
TestForUtil, TestIndexWriter, TestPerFieldDocValuesFormat, 
TestIndexWriterMerging, TestFSTs, TestBytesStore, TestAddIndexes, 
TestIndexWriterExceptions, TestSearcherManager, TestDeletionPolicy, 
TestBooleanOr, TestBasics, TestSloppyPhraseQuery, TestIndexWriterDelete, 
TestIndexWriterMergePolicy, TestNRTThreads, TestDuelingCodecs, 
TestPersistentSnapshotDeletionPolicy, TestStressAdvance, TestNorms, 
TestStandardAnalyzer, TestTieredMergePolicy, TestArrayUtil, 
TestBufferedIndexInput, TestWeakIdentityMap, TestComplexExplanations, 
TestRegexpRandom2, TestSegmentReader, TestTransactions, 
TestMultiThreadTermVectors, TestSegmentTermDocs, TestSimpleSearchEquivalence, 
TestHugeRamFile, TestTermsEnum2, TestDeterminizeLexicon, TestSpanExplanations, 
TestRAMDirectory, TestBagOfPostings, TestStressNRT]
   [junit4] Completed [268/466 (1!)] on J2 in 1.37s, 1 test, 1 error <<< 
FAILURES!

[...truncated 12041 lines...]
   [junit4] Suite: org.apache.solr.search.join.BlockJoinFacetDistribTest
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/init-core-data-001
   [junit4]   2> 1471027 WARN  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=27 numCloses=27
   [junit4]   2> 1471028 INFO  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1471029 INFO  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1471029 INFO  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1471030 INFO  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.c.MiniSolrCloudCluster Starting cluster of 6 servers in 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001
   [junit4]   2> 1471030 INFO  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1471030 INFO  (Thread-3133) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1471030 INFO  (Thread-3133) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1471032 ERROR (Thread-3133) [    ] o.a.z.s.ZooKeeperServer 
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action 
on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1471130 INFO  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.c.ZkTestServer start zk server on port:39885
   [junit4]   2> 1471133 INFO  (zkConnectionManagerCallback-2127-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471137 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1471139 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1471139 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1471140 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1471140 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1471141 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 
2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1471141 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1471141 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1471141 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1471141 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1471141 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1471142 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1471142 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@1ad6af4{/solr,null,AVAILABLE}
   [junit4]   2> 1471142 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1471142 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@67373b{/solr,null,AVAILABLE}
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@b0f535{HTTP/1.1,[http/1.1]}{127.0.0.1:33421}
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@3a97d{/solr,null,AVAILABLE}
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.e.j.s.Server Started @1473421ms
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=33421}
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@17779ff{HTTP/1.1,[http/1.1]}{127.0.0.1:39175}
   [junit4]   2> 1471143 ERROR (jetty-launcher-2124-thread-2) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@ab5510{HTTP/1.1,[http/1.1]}{127.0.0.1:36437}
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.3.0
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T02:59:20.107Z
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.e.j.s.Server Started @1473421ms
   [junit4]   2> 1471143 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.e.j.s.Server Started @1473421ms
   [junit4]   2> 1471144 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=36437}
   [junit4]   2> 1471144 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=39175}
   [junit4]   2> 1471144 ERROR (jetty-launcher-2124-thread-3) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1471144 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.3.0
   [junit4]   2> 1471144 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1471144 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1471145 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T02:59:20.109Z
   [junit4]   2> 1471145 INFO  (zkConnectionManagerCallback-2129-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471144 ERROR (jetty-launcher-2124-thread-1) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1471145 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.3.0
   [junit4]   2> 1471145 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1471145 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1471145 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T02:59:20.109Z
   [junit4]   2> 1471146 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1471146 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1471146 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1471146 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1471146 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@9b2a51{/solr,null,AVAILABLE}
   [junit4]   2> 1471147 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@c91788{HTTP/1.1,[http/1.1]}{127.0.0.1:33111}
   [junit4]   2> 1471147 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.e.j.s.Server Started @1473425ms
   [junit4]   2> 1471147 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=33111}
   [junit4]   2> 1471147 ERROR (jetty-launcher-2124-thread-4) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1471148 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.3.0
   [junit4]   2> 1471148 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1471148 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1471148 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T02:59:20.112Z
   [junit4]   2> 1471151 INFO  (zkConnectionManagerCallback-2131-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471151 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1471153 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1471154 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1471154 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1471155 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1471155 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1471155 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1471155 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@299387{/solr,null,AVAILABLE}
   [junit4]   2> 1471155 INFO  (zkConnectionManagerCallback-2133-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471156 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@5bb40c{HTTP/1.1,[http/1.1]}{127.0.0.1:33283}
   [junit4]   2> 1471156 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.e.j.s.Server Started @1473434ms
   [junit4]   2> 1471156 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=33283}
   [junit4]   2> 1471156 ERROR (jetty-launcher-2124-thread-6) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1471156 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.3.0
   [junit4]   2> 1471156 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1471156 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1471156 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1471156 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T02:59:20.120Z
   [junit4]   2> 1471157 INFO  (zkConnectionManagerCallback-2135-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471157 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@cac48a{/solr,null,AVAILABLE}
   [junit4]   2> 1471158 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1471158 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.e.j.s.AbstractConnector Started 
ServerConnector@df5ae9{HTTP/1.1,[http/1.1]}{127.0.0.1:34115}
   [junit4]   2> 1471158 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.e.j.s.Server Started @1473436ms
   [junit4]   2> 1471158 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=34115}
   [junit4]   2> 1471159 ERROR (jetty-launcher-2124-thread-5) [    ] 
o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be 
missing or incomplete.
   [junit4]   2> 1471159 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 
7.3.0
   [junit4]   2> 1471159 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1471159 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1471159 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 
2018-02-21T02:59:20.123Z
   [junit4]   2> 1471160 INFO  (zkConnectionManagerCallback-2138-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471160 INFO  (jetty-launcher-2124-thread-2) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39885/solr
   [junit4]   2> 1471160 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1471161 INFO  (zkConnectionManagerCallback-2140-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471161 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 
0x10073f02eb00003, likely client has closed socket
   [junit4]   2> 1471163 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1471164 INFO  (jetty-launcher-2124-thread-3) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39885/solr
   [junit4]   2> 1471166 INFO  (zkConnectionManagerCallback-2143-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471175 INFO  (jetty-launcher-2124-thread-4) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39885/solr
   [junit4]   2> 1471175 INFO  (jetty-launcher-2124-thread-1) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39885/solr
   [junit4]   2> 1471178 INFO  (zkConnectionManagerCallback-2148-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471182 INFO  (jetty-launcher-2124-thread-6) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39885/solr
   [junit4]   2> 1471182 INFO  
(zkConnectionManagerCallback-2153-thread-1-processing-n:127.0.0.1:33421_solr) 
[n:127.0.0.1:33421_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471183 INFO  (zkConnectionManagerCallback-2155-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471183 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 
0x10073f02eb00008, likely client has closed socket
   [junit4]   2> 1471183 INFO  (jetty-launcher-2124-thread-5) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39885/solr
   [junit4]   2> 1471184 INFO  (zkConnectionManagerCallback-2158-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471185 INFO  (zkConnectionManagerCallback-2165-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471185 INFO  (zkConnectionManagerCallback-2161-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471186 INFO  
(zkConnectionManagerCallback-2167-thread-1-processing-n:127.0.0.1:36437_solr) 
[n:127.0.0.1:36437_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471186 INFO  
(zkConnectionManagerCallback-2169-thread-1-processing-n:127.0.0.1:33111_solr) 
[n:127.0.0.1:33111_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471189 INFO  
(zkConnectionManagerCallback-2173-thread-1-processing-n:127.0.0.1:33283_solr) 
[n:127.0.0.1:33283_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471190 INFO  
(zkConnectionManagerCallback-2172-thread-1-processing-n:127.0.0.1:39175_solr) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471192 INFO  
(zkConnectionManagerCallback-2175-thread-1-processing-n:127.0.0.1:34115_solr) 
[n:127.0.0.1:34115_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471330 INFO  (jetty-launcher-2124-thread-1) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1471330 INFO  (jetty-launcher-2124-thread-6) 
[n:127.0.0.1:33283_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1471330 INFO  (jetty-launcher-2124-thread-3) 
[n:127.0.0.1:36437_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1471330 INFO  (jetty-launcher-2124-thread-2) 
[n:127.0.0.1:33421_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1471330 INFO  (jetty-launcher-2124-thread-1) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:39175_solr
   [junit4]   2> 1471331 INFO  (jetty-launcher-2124-thread-3) 
[n:127.0.0.1:36437_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:36437_solr
   [junit4]   2> 1471331 INFO  (jetty-launcher-2124-thread-2) 
[n:127.0.0.1:33421_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:33421_solr
   [junit4]   2> 1471331 INFO  (jetty-launcher-2124-thread-6) 
[n:127.0.0.1:33283_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:33283_solr
   [junit4]   2> 1471332 INFO  (jetty-launcher-2124-thread-1) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.Overseer Overseer 
(id=72185069450559506-127.0.0.1:39175_solr-n_0000000000) starting
   [junit4]   2> 1471332 INFO  (jetty-launcher-2124-thread-5) 
[n:127.0.0.1:34115_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 1471338 INFO  (jetty-launcher-2124-thread-4) 
[n:127.0.0.1:33111_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1471339 INFO  (jetty-launcher-2124-thread-5) 
[n:127.0.0.1:34115_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1471341 INFO  (jetty-launcher-2124-thread-4) 
[n:127.0.0.1:33111_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:33111_solr
   [junit4]   2> 1471349 INFO  (jetty-launcher-2124-thread-5) 
[n:127.0.0.1:34115_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:34115_solr
   [junit4]   2> 1471396 INFO  
(zkCallback-2168-thread-1-processing-n:127.0.0.1:33111_solr) 
[n:127.0.0.1:33111_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (5)
   [junit4]   2> 1471396 INFO  
(zkCallback-2152-thread-1-processing-n:127.0.0.1:33421_solr) 
[n:127.0.0.1:33421_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (3)
   [junit4]   2> 1471396 INFO  
(zkCallback-2166-thread-1-processing-n:127.0.0.1:36437_solr) 
[n:127.0.0.1:36437_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (5)
   [junit4]   2> 1471396 INFO  
(zkCallback-2174-thread-1-processing-n:127.0.0.1:34115_solr) 
[n:127.0.0.1:34115_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (5)
   [junit4]   2> 1471396 INFO  
(zkCallback-2170-thread-1-processing-n:127.0.0.1:39175_solr) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (5)
   [junit4]   2> 1471406 INFO  
(zkCallback-2171-thread-1-processing-n:127.0.0.1:33283_solr) 
[n:127.0.0.1:33283_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (5)
   [junit4]   2> 1471407 INFO  (jetty-launcher-2124-thread-1) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:39175_solr
   [junit4]   2> 1471408 INFO  
(zkCallback-2152-thread-2-processing-n:127.0.0.1:33421_solr) 
[n:127.0.0.1:33421_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (3) -> (5)
   [junit4]   2> 1471409 INFO  
(zkCallback-2166-thread-1-processing-n:127.0.0.1:36437_solr) 
[n:127.0.0.1:36437_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (5) -> (6)
   [junit4]   2> 1471409 INFO  
(zkCallback-2174-thread-1-processing-n:127.0.0.1:34115_solr) 
[n:127.0.0.1:34115_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (5) -> (6)
   [junit4]   2> 1471409 INFO  
(zkCallback-2168-thread-1-processing-n:127.0.0.1:33111_solr) 
[n:127.0.0.1:33111_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (5) -> (6)
   [junit4]   2> 1471411 INFO  
(zkCallback-2171-thread-1-processing-n:127.0.0.1:33283_solr) 
[n:127.0.0.1:33283_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (5) -> (6)
   [junit4]   2> 1471411 INFO  
(zkCallback-2152-thread-2-processing-n:127.0.0.1:33421_solr) 
[n:127.0.0.1:33421_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (5) -> (6)
   [junit4]   2> 1471420 INFO  
(OverseerStateUpdate-72185069450559506-127.0.0.1:39175_solr-n_0000000000) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (5) -> (6)
   [junit4]   2> 1471495 INFO  (jetty-launcher-2124-thread-2) 
[n:127.0.0.1:33421_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33421.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471504 INFO  (jetty-launcher-2124-thread-3) 
[n:127.0.0.1:36437_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36437.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471525 INFO  (jetty-launcher-2124-thread-2) 
[n:127.0.0.1:33421_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33421.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471525 INFO  (jetty-launcher-2124-thread-2) 
[n:127.0.0.1:33421_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33421.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471527 INFO  (jetty-launcher-2124-thread-2) 
[n:127.0.0.1:33421_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node2/.
   [junit4]   2> 1471527 INFO  (jetty-launcher-2124-thread-3) 
[n:127.0.0.1:36437_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36437.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471528 INFO  (jetty-launcher-2124-thread-3) 
[n:127.0.0.1:36437_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36437.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471529 INFO  (jetty-launcher-2124-thread-3) 
[n:127.0.0.1:36437_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node3/.
   [junit4]   2> 1471537 INFO  (jetty-launcher-2124-thread-4) 
[n:127.0.0.1:33111_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33111.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471549 INFO  (jetty-launcher-2124-thread-4) 
[n:127.0.0.1:33111_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33111.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471550 INFO  (jetty-launcher-2124-thread-4) 
[n:127.0.0.1:33111_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33111.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471551 INFO  (jetty-launcher-2124-thread-4) 
[n:127.0.0.1:33111_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node4/.
   [junit4]   2> 1471585 INFO  (jetty-launcher-2124-thread-6) 
[n:127.0.0.1:33283_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33283.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471599 INFO  (jetty-launcher-2124-thread-6) 
[n:127.0.0.1:33283_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33283.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471599 INFO  (jetty-launcher-2124-thread-6) 
[n:127.0.0.1:33283_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33283.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471601 INFO  (jetty-launcher-2124-thread-6) 
[n:127.0.0.1:33283_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node6/.
   [junit4]   2> 1471633 INFO  (jetty-launcher-2124-thread-5) 
[n:127.0.0.1:34115_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_34115.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471647 INFO  (jetty-launcher-2124-thread-5) 
[n:127.0.0.1:34115_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_34115.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471648 INFO  (jetty-launcher-2124-thread-5) 
[n:127.0.0.1:34115_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_34115.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471649 INFO  (jetty-launcher-2124-thread-5) 
[n:127.0.0.1:34115_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node5/.
   [junit4]   2> 1471663 INFO  (jetty-launcher-2124-thread-1) 
[n:127.0.0.1:39175_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39175.solr.node' (registry 'solr.node') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471674 INFO  (jetty-launcher-2124-thread-1) 
[n:127.0.0.1:39175_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39175.solr.jvm' (registry 'solr.jvm') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471675 INFO  (jetty-launcher-2124-thread-1) 
[n:127.0.0.1:39175_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39175.solr.jetty' (registry 'solr.jetty') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1471676 INFO  (jetty-launcher-2124-thread-1) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core 
definitions underneath 
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node1/.
   [junit4]   2> 1471932 INFO  (zkConnectionManagerCallback-2189-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471933 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] 
o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 
0x10073f02eb00013, likely client has closed socket
   [junit4]   2> 1471934 INFO  (zkConnectionManagerCallback-2193-thread-1) [    
] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1471935 INFO  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
   [junit4]   2> 1471936 INFO  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39885/solr ready
   [junit4]   2> 1472138 INFO  (qtp22501017-11124) [n:127.0.0.1:33111_solr    ] 
o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params 
replicationFactor=2&property.config=solrconfig-blockjoinfacetcomponent.xml&property.schema=schema-blockjoinfacetcomponent.xml&collection.configName=solrCloudCollectionConfig&name=facetcollection&nrtReplicas=2&action=CREATE&numShards=3&wt=javabin&version=2
 and sendToOCPQueue=true
   [junit4]   2> 1472140 INFO  
(OverseerThreadFactory-3505-thread-1-processing-n:127.0.0.1:39175_solr) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection 
facetcollection
   [junit4]   2> 1472246 INFO  
(OverseerStateUpdate-72185069450559506-127.0.0.1:39175_solr-n_0000000000) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"facetcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"facetcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33283/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1472247 INFO  
(OverseerStateUpdate-72185069450559506-127.0.0.1:39175_solr-n_0000000000) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"facetcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"facetcollection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:39175/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1472250 INFO  
(OverseerStateUpdate-72185069450559506-127.0.0.1:39175_solr-n_0000000000) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"facetcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"facetcollection_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36437/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1472251 INFO  
(OverseerStateUpdate-72185069450559506-127.0.0.1:39175_solr-n_0000000000) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"facetcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"facetcollection_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33421/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1472253 INFO  
(OverseerStateUpdate-72185069450559506-127.0.0.1:39175_solr-n_0000000000) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"facetcollection",
   [junit4]   2>   "shard":"shard3",
   [junit4]   2>   "core":"facetcollection_shard3_replica_n8",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:34115/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1472254 INFO  
(OverseerStateUpdate-72185069450559506-127.0.0.1:39175_solr-n_0000000000) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"facetcollection",
   [junit4]   2>   "shard":"shard3",
   [junit4]   2>   "core":"facetcollection_shard3_replica_n10",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33111/solr";,
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1472458 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=facetcollection&version=2&replicaType=NRT&property.config=solrconfig-blockjoinfacetcomponent.xml&property.schema=schema-blockjoinfacetcomponent.xml&coreNodeName=core_node12&name=facetcollection_shard3_replica_n10&action=CREATE&numShards=3&shard=shard3&wt=javabin
   [junit4]   2> 1472458 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=facetcollection&version=2&replicaType=NRT&property.config=solrconfig-blockjoinfacetcomponent.xml&property.schema=schema-blockjoinfacetcomponent.xml&coreNodeName=core_node11&name=facetcollection_shard3_replica_n8&action=CREATE&numShards=3&shard=shard3&wt=javabin
   [junit4]   2> 1472458 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=facetcollection&version=2&replicaType=NRT&property.config=solrconfig-blockjoinfacetcomponent.xml&property.schema=schema-blockjoinfacetcomponent.xml&coreNodeName=core_node9&name=facetcollection_shard2_replica_n6&action=CREATE&numShards=3&shard=shard2&wt=javabin
   [junit4]   2> 1472459 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=facetcollection&version=2&replicaType=NRT&property.config=solrconfig-blockjoinfacetcomponent.xml&property.schema=schema-blockjoinfacetcomponent.xml&coreNodeName=core_node5&name=facetcollection_shard1_replica_n2&action=CREATE&numShards=3&shard=shard1&wt=javabin
   [junit4]   2> 1472458 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=facetcollection&version=2&replicaType=NRT&property.config=solrconfig-blockjoinfacetcomponent.xml&property.schema=schema-blockjoinfacetcomponent.xml&coreNodeName=core_node3&name=facetcollection_shard1_replica_n1&action=CREATE&numShards=3&shard=shard1&wt=javabin
   [junit4]   2> 1472459 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 1472459 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 1472459 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 1472459 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 1472459 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 1472471 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr    ] 
o.a.s.h.a.CoreAdminOperation core create command 
qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=facetcollection&version=2&replicaType=NRT&property.config=solrconfig-blockjoinfacetcomponent.xml&property.schema=schema-blockjoinfacetcomponent.xml&coreNodeName=core_node7&name=facetcollection_shard2_replica_n4&action=CREATE&numShards=3&shard=shard2&wt=javabin
   [junit4]   2> 1472471 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr    ] 
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 
transient cores
   [junit4]   2> 1472574 INFO  
(zkCallback-2152-thread-2-processing-n:127.0.0.1:33421_solr) 
[n:127.0.0.1:33421_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472574 INFO  
(zkCallback-2174-thread-1-processing-n:127.0.0.1:34115_solr) 
[n:127.0.0.1:34115_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472574 INFO  
(zkCallback-2152-thread-1-processing-n:127.0.0.1:33421_solr) 
[n:127.0.0.1:33421_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472574 INFO  
(zkCallback-2170-thread-1-processing-n:127.0.0.1:39175_solr) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472574 INFO  
(zkCallback-2168-thread-1-processing-n:127.0.0.1:33111_solr) 
[n:127.0.0.1:33111_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472575 INFO  
(zkCallback-2170-thread-2-processing-n:127.0.0.1:39175_solr) 
[n:127.0.0.1:39175_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472574 INFO  
(zkCallback-2166-thread-1-processing-n:127.0.0.1:36437_solr) 
[n:127.0.0.1:36437_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472575 INFO  
(zkCallback-2168-thread-2-processing-n:127.0.0.1:33111_solr) 
[n:127.0.0.1:33111_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472574 INFO  
(zkCallback-2171-thread-1-processing-n:127.0.0.1:33283_solr) 
[n:127.0.0.1:33283_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472575 INFO  
(zkCallback-2166-thread-2-processing-n:127.0.0.1:36437_solr) 
[n:127.0.0.1:36437_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472575 INFO  
(zkCallback-2171-thread-2-processing-n:127.0.0.1:33283_solr) 
[n:127.0.0.1:33283_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1472575 INFO  
(zkCallback-2174-thread-2-processing-n:127.0.0.1:34115_solr) 
[n:127.0.0.1:34115_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/collections/facetcollection/state.json] for collection [facetcollection] 
has occurred - updating... (live nodes size: [6])
   [junit4]   2> 1473484 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1473501 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1473502 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1473503 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1473505 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.s.IndexSchema [facetcollection_shard3_replica_n8] Schema name=test
   [junit4]   2> 1473508 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1473510 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1473510 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.c.CoreContainer Creating SolrCore 'facetcollection_shard3_replica_n8' 
using configuration from collection facetcollection, trusted=true
   [junit4]   2> 1473511 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_34115.solr.core.facetcollection.shard3.replica_n8' (registry 
'solr.core.facetcollection.shard3.replica_n8') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1473511 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1473511 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.c.SolrCore [[facetcollection_shard3_replica_n8] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node5/facetcollection_shard3_replica_n8],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node5/./facetcollection_shard3_replica_n8/data/]
   [junit4]   2> 1473512 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
   [junit4]   2> 1473514 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.s.IndexSchema [facetcollection_shard2_replica_n6] Schema name=test
   [junit4]   2> 1473517 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1473517 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.c.CoreContainer Creating SolrCore 'facetcollection_shard2_replica_n6' 
using configuration from collection facetcollection, trusted=true
   [junit4]   2> 1473518 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33421.solr.core.facetcollection.shard2.replica_n6' (registry 
'solr.core.facetcollection.shard2.replica_n6') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1473518 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1473518 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.c.SolrCore [[facetcollection_shard2_replica_n6] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node2/facetcollection_shard2_replica_n6],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node2/./facetcollection_shard2_replica_n6/data/]
   [junit4]   2> 1473520 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.s.IndexSchema [facetcollection_shard2_replica_n4] Schema name=test
   [junit4]   2> 1473568 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.s.IndexSchema [facetcollection_shard1_replica_n2] Schema name=test
   [junit4]   2> 1473570 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.s.IndexSchema [facetcollection_shard1_replica_n1] Schema name=test
   [junit4]   2> 1473574 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1473574 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.c.CoreContainer Creating SolrCore 'facetcollection_shard1_replica_n1' 
using configuration from collection facetcollection, trusted=true
   [junit4]   2> 1473574 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33283.solr.core.facetcollection.shard1.replica_n1' (registry 
'solr.core.facetcollection.shard1.replica_n1') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1473574 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1473575 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.c.SolrCore [[facetcollection_shard1_replica_n1] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node6/facetcollection_shard1_replica_n1],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node6/./facetcollection_shard1_replica_n1/data/]
   [junit4]   2> 1473576 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1473576 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.c.CoreContainer Creating SolrCore 'facetcollection_shard2_replica_n4' 
using configuration from collection facetcollection, trusted=true
   [junit4]   2> 1473577 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_36437.solr.core.facetcollection.shard2.replica_n4' (registry 
'solr.core.facetcollection.shard2.replica_n4') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1473577 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1473577 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.c.SolrCore [[facetcollection_shard2_replica_n4] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node3/facetcollection_shard2_replica_n4],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node3/./facetcollection_shard2_replica_n4/data/]
   [junit4]   2> 1473590 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.s.IndexSchema [facetcollection_shard3_replica_n10] Schema name=test
   [junit4]   2> 1473593 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1473594 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.c.CoreContainer Creating SolrCore 'facetcollection_shard3_replica_n10' 
using configuration from collection facetcollection, trusted=true
   [junit4]   2> 1473594 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_33111.solr.core.facetcollection.shard3.replica_n10' (registry 
'solr.core.facetcollection.shard3.replica_n10') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1473594 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1473594 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.c.SolrCore [[facetcollection_shard3_replica_n10] ] Opening new SolrCore 
at 
[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node4/facetcollection_shard3_replica_n10],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node4/./facetcollection_shard3_replica_n10/data/]
   [junit4]   2> 1473599 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1473599 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.c.CoreContainer Creating SolrCore 'facetcollection_shard1_replica_n2' 
using configuration from collection facetcollection, trusted=true
   [junit4]   2> 1473599 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.m.r.SolrJmxReporter JMX monitoring for 
'solr_39175.solr.core.facetcollection.shard1.replica_n2' (registry 
'solr.core.facetcollection.shard1.replica_n2') enabled at server: 
com.sun.jmx.mbeanserver.JmxMBeanServer@1a72d53
   [junit4]   2> 1473600 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1473600 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.c.SolrCore [[facetcollection_shard1_replica_n2] ] Opening new SolrCore at 
[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node1/facetcollection_shard1_replica_n2],
 
dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.search.join.BlockJoinFacetDistribTest_FE37B6B8804C4D10-001/tempDir-001/node1/./facetcollection_shard1_replica_n2/data/]
   [junit4]   2> 1473847 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 1473848 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1473848 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 1473848 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1473848 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 1473848 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1473851 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1473851 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1473851 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1473851 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 1473851 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1473851 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1473851 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1473851 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 1473851 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1473853 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1473853 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1473853 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1473853 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1473854 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.UpdateLog
   [junit4]   2> 1473854 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1473855 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@14f7ace[facetcollection_shard2_replica_n6] main]
   [junit4]   2> 1473855 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1473855 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1473855 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@1de4ef9[facetcollection_shard3_replica_n10] main]
   [junit4]   2> 1473857 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 1473857 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1473858 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@c39b05[facetcollection_shard1_replica_n1] main]
   [junit4]   2> 1473858 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 1473851 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1473858 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1473858 INFO  (qtp1989294-11105) [n:127.0.0.1:33421_solr 
c:facetcollection s:shard2 r:core_node9 x:facetcollection_shard2_replica_n6] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1592977745848041472
   [junit4]   2> 1473859 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 1473859 INFO  (qtp515070-11132) [n:127.0.0.1:33283_solr 
c:facetcollection s:shard1 r:core_node3 x:facetcollection_shard1_replica_n1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1473855 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@b32fc6[facetcollection_shard1_replica_n2] main]
   [junit4]   2> 1473861 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/solrCloudCollectionConfig
   [junit4]   2> 1473861 INFO  (qtp30210226-11123) [n:127.0.0.1:34115_solr 
c:facetcollection s:shard3 r:core_node11 x:facetcollection_shard3_replica_n8] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@1416f90[facetcollection_shard3_replica_n8] main]
   [junit4]   2> 1473861 INFO  
(searcherExecutor-3522-thread-1-processing-n:127.0.0.1:33111_solr 
x:facetcollection_shard3_replica_n10 s:shard3 c:facetcollection r:core_node12) 
[n:127.0.0.1:33111_solr c:facetcollection s:shard3 r:core_node12 
x:facetcollection_shard3_replica_n10] o.a.s.c.SolrCore 
[facetcollection_shard3_replica_n10] Registered new searcher 
Searcher@1de4ef9[facetcollection_shard3_replica_n10] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1473861 INFO  (qtp22501017-11146) [n:127.0.0.1:33111_solr 
c:facetcollection s:shard3 r:core_node12 x:facetcollection_shard3_replica_n10] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1592977745851187200
   [junit4]   2> 1473861 INFO  
(searcherExecutor-3519-thread-1-processing-n:127.0.0.1:33421_solr 
x:facetcollection_shard2_replica_n6 s:shard2 c:facetcollection r:core_node9) 
[n:127.0.0.1:33421_solr c:facetcollection s:shard2 r:core_node9 
x:facetcollection_shard2_replica_n6] o.a.s.c.SolrCore 
[facetcollection_shard2_replica_n6] Registered new searcher 
Searcher@14f7ace[facetcollection_shard2_replica_n6] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1473861 INFO  (qtp1550259-11104) [n:127.0.0.1:39175_solr 
c:facetcollection s:shard1 r:core_node5 x:facetcollection_shard1_replica_n2] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1473861 INFO  (qtp3267762-11115) [n:127.0.0.1:36437_solr 
c:facetcollection s:shard2 r:core_node7 x:facetcollection_shard2_replica_n4] 
o.a.s.s.SolrIndexSearcher Opening 
[Searcher@183c494[facetcollection_shard2_replica_n4] main]
   [junit4]   2> 1473861 INFO  
(searcherExecutor-3520-thread-1-processing-n:127.0.0.1:33283_solr 
x:facetcollection_shard1_replica_n1 s:shard1 c:facetcollection r:core_node3) 
[n:127.0.0.1:3

[...truncated too long message...]

terprops.json
   [junit4]   2>        6       /solr/collections/facetcollection/terms/shard2
   [junit4]   2>        6       /solr/security.json
   [junit4]   2>        6       /solr/configs/solrCloudCollectionConfig
   [junit4]   2>        5       /solr/collections/facetcollection/terms/shard3
   [junit4]   2>        5       /solr/collections/facetcollection/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        43      /solr/collections/facetcollection/state.json
   [junit4]   2>        7       /solr/clusterstate.json
   [junit4]   2>        3       
/solr/overseer_elect/election/72185069450559506-127.0.0.1:39175_solr-n_0000000000
   [junit4]   2>        2       
/solr/overseer_elect/election/72185069450559502-127.0.0.1:36437_solr-n_0000000002
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        7       /solr/live_nodes
   [junit4]   2>        7       /solr/collections
   [junit4]   2> 
   [junit4]   2> 1511162 INFO  
(SUITE-BlockJoinFacetDistribTest-seed#[FE37B6B8804C4D10]-worker) [    ] 
o.a.s.SolrTestCaseJ4 ------------------------------------------------------- 
Done waiting for tracked resources to be released
   [junit4]   2> Feb 21, 2018 3:00:00 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> Feb 21, 2018 3:00:20 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at 
org.apache.solr.search.join.BlockJoinFacetDistribTest: 
   [junit4]   2>    1) Thread[id=11161, name=qtp515070-11161, 
state=TIMED_WAITING, group=TGRP-BlockJoinFacetDistribTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Feb 21, 2018 3:00:20 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=11161, name=qtp515070-11161, 
state=TIMED_WAITING, group=TGRP-BlockJoinFacetDistribTest]
   [junit4]   2> Feb 21, 2018 3:00:23 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be 
terminated:
   [junit4]   2>    1) Thread[id=11161, name=qtp515070-11161, 
state=TIMED_WAITING, group=TGRP-BlockJoinFacetDistribTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{type_s=PostingsFormat(name=MockRandom), COLOR_s=FSTOrd50, SIZE_s=FSTOrd50, 
_root_=PostingsFormat(name=MockRandom), BRAND_s=FSTOrd50, id=FSTOrd50}, 
docValues:{}, maxPointsInLeafNode=496, maxMBSortInHeap=5.048635068415731, 
sim=RandomSimilarity(queryNorm=true): {}, locale=ar-MA, timezone=America/Belem
   [junit4]   2> NOTE: Linux 4.13.0-32-generic i386/Oracle Corporation 
1.8.0_162 (32-bit)/cpus=8,threads=2,free=213641784,total=400031744
   [junit4]   2> NOTE: All tests run in this JVM: [TestApiFramework, 
TestHdfsBackupRestoreCore, BlockDirectoryTest, TestInitParams, 
OverseerCollectionConfigSetProcessorTest, TestQuerySenderListener, 
TestDFRSimilarityFactory, BigEndianAscendingWordDeserializerTest, 
DirectSolrSpellCheckerTest, TestLMJelinekMercerSimilarityFactory, 
TestSolrDeletionPolicy2, HdfsRestartWhileUpdatingTest, ExplicitHLLTest, 
MultiTermTest, TestRealTimeGet, CloudMLTQParserTest, TestLFUCache, 
TestOverriddenPrefixQueryForCustomFieldType, TestDistribStateManager, 
PreAnalyzedFieldTest, DocValuesTest, TestSegmentSorting, VersionInfoTest, 
TestGeoJSONResponseWriter, TestShardHandlerFactory, NotRequiredUniqueKeyTest, 
TestInPlaceUpdatesDistrib, AddBlockUpdateTest, TestUtils, TestInitQParser, 
TestBinaryResponseWriter, MetricsConfigTest, TestSolrCloudWithHadoopAuthPlugin, 
TestAuthorizationFramework, TestEmbeddedSolrServerConstructors, TestXmlQParser, 
TestSolrCloudWithSecureImpersonation, CdcrVersionReplicationTest, 
DisMaxRequestHandlerTest, DistribCursorPagingTest, DistributedQueueTest, 
TestSolrXml, TestSkipOverseerOperations, SearchRateTriggerTest, 
TestRebalanceLeaders, TestCloudManagedSchema, SolrGraphiteReporterTest, 
ClusterStateTest, MetricTriggerTest, ReturnFieldsTest, 
TestDistribDocBasedVersion, ConnectionManagerTest, RecoveryZkTest, 
TestUseDocValuesAsStored2, DistributedSuggestComponentTest, 
TestHalfAndHalfDocValues, OverseerStatusTest, FileBasedSpellCheckerTest, 
RequestLoggingTest, BooleanFieldTest, HdfsBasicDistributedZkTest, 
ConcurrentDeleteAndCreateCollectionTest, GraphQueryTest, 
TriLevelCompositeIdRoutingTest, DocValuesMultiTest, TestQueryTypes, 
CachingDirectoryFactoryTest, URLClassifyProcessorTest, SpatialFilterTest, 
TestValueSourceCache, TestTolerantUpdateProcessorRandomCloud, 
ExecutePlanActionTest, SolrMetricReporterTest, MoreLikeThisHandlerTest, 
DistributedFacetPivotLongTailTest, DateMathParserTest, HLLUtilTest, 
NodeMutatorTest, TestTolerantSearch, HdfsTlogReplayBufferedWhileIndexingTest, 
CacheHeaderTest, DistanceUnitsTest, DeleteReplicaTest, TestDynamicLoading, 
RollingRestartTest, TestCoreDiscovery, TimeZoneUtilsTest, StressHdfsTest, 
TestClusterProperties, SolrSlf4jReporterTest, RecoveryAfterSoftCommitTest, 
ClassificationUpdateProcessorTest, TestFieldTypeCollectionResource, 
TestCollectionsAPIViaSolrCloudCluster, TestStandardQParsers, 
TestSha256AuthenticationProvider, TestMissingGroups, TestFieldCacheVsDocValues, 
RulesTest, DirectoryFactoryTest, LeaderElectionTest, TestCoreContainer, 
CdcrReplicationDistributedZkTest, TestExactSharedStatsCache, 
BlockJoinFacetDistribTest]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=BlockJoinFacetDistribTest -Dtests.seed=FE37B6B8804C4D10 
-Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar-MA 
-Dtests.timezone=America/Belem -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J2 | BlockJoinFacetDistribTest (suite) <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.search.join.BlockJoinFacetDistribTest: 
   [junit4]    >    1) Thread[id=11161, name=qtp515070-11161, 
state=TIMED_WAITING, group=TGRP-BlockJoinFacetDistribTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([FE37B6B8804C4D10]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=11161, name=qtp515070-11161, 
state=TIMED_WAITING, group=TGRP-BlockJoinFacetDistribTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([FE37B6B8804C4D10]:0)
   [junit4] Completed [314/777 (1!)] on J2 in 63.26s, 1 test, 2 errors <<< 
FAILURES!

[...truncated 45736 lines...]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to