[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 137 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/137/

5 tests failed.
FAILED:  org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataV1

Error Message:


Stack Trace:
java.lang.AssertionError
        at 
__randomizedtesting.SeedInfo.seed([3C08F93888ED514E:ED6847F1A8414217]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at 
org.apache.solr.cloud.AliasIntegrationTest.checkFooAndBarMeta(AliasIntegrationTest.java:283)
        at 
org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataV1(AliasIntegrationTest.java:251)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv

Error Message:
java.lang.RuntimeException: Error from server at 
http://127.0.0.1:35719/solr/test_col: Async exception during distributed 
update: Error from server at 
http://127.0.0.1:37533/solr/test_col_shard5_replica_n16: Server Error    
request: 
http://127.0.0.1:37533/solr/test_col_shard5_replica_n16/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A35719%2Fsolr%2Ftest_col_shard3_replica_n8%2F&wt=javabin&version=2
 Remote error message: Failed synchronous update on shard StdNode: 
http://127.0.0.1:34673/solr/test_col_shard5_replica_n14/ update: 
org.apache.solr.client.solrj.request.UpdateRequest@7449f6b0

Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from 
server at http://127.0.0.1:35719/solr/test_col: Async exception during 
distributed update: Error from server at 
http://127.0.0.1:37533/solr/test_col_shard5_replica_n16: Server Error



request: 
http://127.0.0.1:37533/solr/test_col_shard5_replica_n16/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A35719%2Fsolr%2Ftest_col_shard3_replica_n8%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: 
http://127.0.0.1:34673/solr/test_col_shard5_replica_n14/ update: 
org.apache.solr.client.solrj.request.UpdateRequest@7449f6b0
        at 
__randomizedtesting.SeedInfo.seed([3C08F93888ED514E:A1C9B7E02B06B5F]:0)
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:283)
        at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:195)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error from server at 
http://127.0.0.1:35719/solr/test_col: Async exception during distributed 
update: Error from server at 
http://127.0.0.1:37533/solr/test_col_shard5_replica_n16: Server Error



request: 
http://127.0.0.1:37533/solr/test_col_shard5_replica_n16/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A35719%2Fsolr%2Ftest_col_shard3_replica_n8%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: 
http://127.0.0.1:34673/solr/test_col_shard5_replica_n14/ update: 
org.apache.solr.client.solrj.request.UpdateRequest@7449f6b0
        at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:411)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        ... 1 more
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:35719/solr/test_col: Async exception during 
distributed update: Error from server at 
http://127.0.0.1:37533/solr/test_col_shard5_replica_n16: Server Error



request: 
http://127.0.0.1:37533/solr/test_col_shard5_replica_n16/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A35719%2Fsolr%2Ftest_col_shard3_replica_n8%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: 
http://127.0.0.1:34673/solr/test_col_shard5_replica_n14/ update: 
org.apache.solr.client.solrj.request.UpdateRequest@7449f6b0
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:370)
        at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:399)
        ... 6 more


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 8 in https://127.0.0.1:46795/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 8 in https://127.0.0.1:46795/solr
        at 
__randomizedtesting.SeedInfo.seed([3C08F93888ED514E:FDF88094A5BD9BE9]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNotNull(Assert.java:526)
        at 
org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:885)
        at 
org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:599)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitAfterFailedSplit

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
        at 
__randomizedtesting.SeedInfo.seed([3C08F93888ED514E:C5456A97B4981CC4]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at 
org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:282)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
        at 
__randomizedtesting.SeedInfo.seed([3C08F93888ED514E:6140E7B1472BF701]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12477 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
   [junit4]   2> Creating dataDir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestLargeCluster_3C08F93888ED514E-001/init-core-data-001
   [junit4]   2> 134394 WARN  
(SUITE-TestLargeCluster-seed#[3C08F93888ED514E]-worker) [    ] 
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=50 numCloses=50
   [junit4]   2> 134395 INFO  
(SUITE-TestLargeCluster-seed#[3C08F93888ED514E]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) 
w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 134397 INFO  
(SUITE-TestLargeCluster-seed#[3C08F93888ED514E]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 134397 INFO  
(SUITE-TestLargeCluster-seed#[3C08F93888ED514E]-worker) [    ] 
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: 
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 134560 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 134561 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 0
   [junit4]   2> 134561 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10043_solr, 
127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10057_solr, 
127.0.0.1:10049_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 
127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10060_solr, 
127.0.0.1:10005_solr, 127.0.0.1:10013_solr, 127.0.0.1:10021_solr, 
127.0.0.1:10058_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 
127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10027_solr, 
127.0.0.1:10022_solr, 127.0.0.1:10039_solr, 127.0.0.1:10033_solr, 
127.0.0.1:10025_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 
127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10036_solr, 
127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10041_solr, 
127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 
127.0.0.1:10014_solr, 127.0.0.1:10003_solr, 127.0.0.1:10011_solr, 
127.0.0.1:10001_solr, 127.0.0.1:10040_solr, 127.0.0.1:10054_solr, 
127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 
127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10007_solr, 
127.0.0.1:10015_solr, 127.0.0.1:10055_solr, 127.0.0.1:10069_solr, 
127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10062_solr, 
127.0.0.1:10070_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 
127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10061_solr, 
127.0.0.1:10067_solr, 127.0.0.1:10050_solr, 127.0.0.1:10002_solr, 
127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10045_solr, 
127.0.0.1:10037_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 
127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10048_solr, 
127.0.0.1:10056_solr, 127.0.0.1:10064_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 134561 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 134561 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 134596 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 134597 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 134597 DEBUG (ScheduledTrigger-152-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 134617 DEBUG (ScheduledTrigger-152-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 134642 INFO  
(TEST-TestLargeCluster.testAddNode-seed#[3C08F93888ED514E]) [    ] 
o.a.s.SolrTestCaseJ4 ###Starting testAddNode
   [junit4]   2> 134642 DEBUG 
(TEST-TestLargeCluster.testAddNode-seed#[3C08F93888ED514E]) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
1
   [junit4]   2> 134643 INFO  
(TEST-TestLargeCluster.testAddNode-seed#[3C08F93888ED514E]) [    ] 
o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing 
object cache...
   [junit4]   2> 134644 DEBUG 
(TEST-TestLargeCluster.testAddNode-seed#[3C08F93888ED514E]) [    ] 
o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 134649 WARN  (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are 
closed, exiting.
   [junit4]   2> 134714 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 134714 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode 
version 2
   [junit4]   2> 134715 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 
127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 
127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 
127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 
127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 
127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 
127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 
127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 
127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 
127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 
127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 
127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 
127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 
127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 
127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 
127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 
127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 
127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 
127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 
127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 
127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 
127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 
127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 
127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 
127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 
127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 
127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 
127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 
127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 
127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 
127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 
127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 
127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 
127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 134715 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 134715 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 134715 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 134715 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 134767 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 134787 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135050 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135075 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135095 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135111 DEBUG (simCloudManagerPool-154-thread-1) [    ] 
o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 135111 DEBUG (simCloudManagerPool-154-thread-1) [    ] 
o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 
3
   [junit4]   2> 135113 DEBUG (simCloudManagerPool-154-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 
127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 
127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 
127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 
127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 
127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 
127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 
127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 
127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 
127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 
127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 
127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 
127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 
127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 
127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 
127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 
127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 
127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 
127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 
127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 
127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 
127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 
127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 
127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 
127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 
127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 
127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 
127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 
127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 
127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 
127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 
127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 
127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 
127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 135131 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135167 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135168 DEBUG (simCloudManagerPool-154-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10082_solr, 
127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 
127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 
127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 
127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 
127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 
127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 
127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 
127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 
127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 
127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 
127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 
127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 
127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 
127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 
127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 
127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 
127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 
127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 
127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 
127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 
127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 
127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 
127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 
127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 
127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 
127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 
127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 
127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 
127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 
127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 
127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 
127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 
127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 135168 DEBUG (simCloudManagerPool-154-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated 
with properties: {event=nodeAdded, waitFor=5, actions=[{name=compute, 
class=org.apache.solr.cloud.autoscaling.ComputePlanAction}, {name=execute, 
class=org.apache.solr.cloud.autoscaling.ExecutePlanAction}], enabled=true}
   [junit4]   2> 135168 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 135169 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    
] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 135175 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135175 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135188 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135212 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135212 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135212 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135256 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135256 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135256 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135289 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135337 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135354 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135354 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135357 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135374 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135374 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135378 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135398 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135398 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135398 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135434 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135434 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135435 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135455 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135455 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135455 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135488 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135488 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135489 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135509 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135509 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135524 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135540 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135540 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135560 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135560 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135560 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135580 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135580 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135580 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135601 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135601 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135601 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135629 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135629 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135629 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135649 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135649 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135649 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135690 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135690 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135690 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135710 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135710 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135710 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135730 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135730 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135730 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135750 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135750 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135751 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135783 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135783 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135783 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135803 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135803 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135803 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135823 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135823 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135824 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135844 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135844 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135844 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135864 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135864 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135864 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135884 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135884 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135884 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135905 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135905 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135905 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135925 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135925 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135925 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135957 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135957 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135957 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135977 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135977 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135977 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 135998 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 135998 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 135998 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136026 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136026 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136026 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136046 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136046 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136046 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136066 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136067 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136067 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136092 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136092 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136092 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136112 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136112 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136112 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136164 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136164 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136164 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136192 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136192 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136192 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136212 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136212 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136212 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136233 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136233 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136233 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136261 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136261 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136261 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136289 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136289 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136290 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136310 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136310 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136310 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136330 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136330 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136330 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136350 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136350 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136350 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136390 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136391 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136391 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136411 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136411 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136411 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136431 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136431 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136431 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136451 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136451 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136451 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136471 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136471 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136472 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136504 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136504 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136504 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136528 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136528 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136528 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136569 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136569 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136569 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136605 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136605 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136605 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136642 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136642 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136642 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136678 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136678 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136678 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136715 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136715 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136715 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136751 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136751 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136751 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136787 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136788 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136788 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136824 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136824 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136824 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136860 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136861 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136861 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136897 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136897 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136897 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136917 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136917 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136917 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136950 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136950 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136950 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 136978 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 136978 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 136978 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137010 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137011 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137011 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137047 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137047 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137047 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137084 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137084 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137084 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137120 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137120 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137120 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137156 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137156 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137156 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137193 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137193 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137193 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137229 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137229 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137229 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137266 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137266 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137266 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137302 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137302 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137302 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137338 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137339 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137339 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137375 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137375 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137375 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137411 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137411 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137411 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137432 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137432 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137432 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137452 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137452 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137452 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137484 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137484 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137484 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137504 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137505 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137505 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137533 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137533 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137533 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137569 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137569 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137569 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137602 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137602 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137602 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137626 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137626 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137626 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137646 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137646 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137646 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137670 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137670 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137670 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137691 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137691 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137691 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137711 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137711 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137711 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137731 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137731 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137731 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137751 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137751 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137752 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137780 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137780 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137780 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137800 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137800 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137800 DEBUG (ScheduledTrigger-155-thread-1) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137820 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137820 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137820 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137857 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137857 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137857 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137893 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137893 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137893 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137914 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137914 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137914 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137934 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137934 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137934 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137954 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137954 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137954 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 137990 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 137991 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 137991 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138011 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138011 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138011 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138031 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138031 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138031 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138051 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138051 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138051 DEBUG (ScheduledTrigger-155-thread-3) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138072 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138072 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138072 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138092 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138092 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138092 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138112 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138112 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138112 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138132 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138132 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138132 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138153 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138153 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138153 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138173 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138173 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138173 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138193 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138193 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138193 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138213 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138213 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138213 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138234 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138234 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138234 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138254 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138254 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138254 DEBUG (ScheduledTrigger-155-thread-2) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138274 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138274 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 138274 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
   [junit4]   2> 138294 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with 
currently live nodes: 100
   [junit4]   2> 138294 DEBUG (ScheduledTrigger-155-thread-4) [    ] 
o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [j

[...truncated too long message...]

.0.0.1:34270_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread 
woken up but we are closed, exiting.
   [junit4]   2> 5334684 INFO  (jetty-closer-4795-thread-2) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@367a04b0{/solr,null,UNAVAILABLE}
   [junit4]   2> 5334896 INFO  (jetty-closer-4795-thread-2) [    ] 
o.e.j.s.session Stopped scavenging
   [junit4]   2> 5334924 INFO  (jetty-closer-4795-thread-1) [    ] 
o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@3a2a5db2{/solr,null,UNAVAILABLE}
   [junit4]   2> 5334924 INFO  (jetty-closer-4795-thread-1) [    ] 
o.e.j.s.session Stopped scavenging
   [junit4]   2> 5334960 ERROR 
(SUITE-TestTlogReplica-seed#[3C08F93888ED514E]-worker) [    ] 
o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper 
server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 5334976 INFO  
(SUITE-TestTlogReplica-seed#[3C08F93888ED514E]-worker) [    ] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1:36765 36765
   [junit4]   2> 5335094 INFO  (Thread-8733) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:36765 36765
   [junit4]   2> 5335095 WARN  (Thread-8733) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        33      /solr/configs/conf
   [junit4]   2>        26      
/solr/collections/tlog_replica_test_create_delete/terms/shard1
   [junit4]   2>        24      
/solr/collections/tlog_replica_test_create_delete/terms/shard2
   [junit4]   2>        12      
/solr/collections/tlog_replica_test_recovery/terms/shard1
   [junit4]   2>        11      
/solr/collections/tlog_replica_test_remove_leader/terms/shard1
   [junit4]   2>        10      /solr/aliases.json
   [junit4]   2>        9       
/solr/collections/tlog_replica_test_basic_leader_election/terms/shard1
   [junit4]   2>        9       /solr/security.json
   [junit4]   2>        8       
/solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/terms/shard1
   [junit4]   2>        8       
/solr/collections/tlog_replica_test_kill_leader/terms/shard1
   [junit4]   2>        8       
/solr/collections/tlog_replica_test_real_time_get/terms/shard1
   [junit4]   2>        8       
/solr/collections/tlog_replica_test_kill_tlog_replica/terms/shard1
   [junit4]   2>        8       
/solr/collections/tlog_replica_test_only_leader_indexes/terms/shard1
   [junit4]   2>        7       
/solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard1
   [junit4]   2>        7       
/solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard2
   [junit4]   2>        6       
/solr/collections/tlog_replica_test_delete_by_id/terms/shard1
   [junit4]   2>        2       
/solr/collections/tlog_replica_test_add_docs/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        10      
/solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
   [junit4]   2>        10      /solr/clusterprops.json
   [junit4]   2>        10      /solr/clusterstate.json
   [junit4]   2>        8       
/solr/collections/tlog_replica_test_recovery/state.json
   [junit4]   2>        5       
/solr/collections/tlog_replica_test_kill_leader/state.json
   [junit4]   2>        5       
/solr/collections/tlog_replica_test_basic_leader_election/state.json
   [junit4]   2>        4       
/solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/72184847799812113-core_node2-n_0000000000
   [junit4]   2>        4       
/solr/overseer_elect/election/72184847799812101-127.0.0.1:46795_solr-n_0000000000
   [junit4]   2>        4       
/solr/overseer_elect/election/72184847799812113-127.0.0.1:34270_solr-n_0000000004
   [junit4]   2>        4       
/solr/collections/tlog_replica_test_create_delete/state.json
   [junit4]   2>        4       
/solr/collections/tlog_replica_test_kill_tlog_replica/state.json
   [junit4]   2>        3       
/solr/collections/tlog_replica_test_remove_leader/state.json
   [junit4]   2>        3       
/solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
   [junit4]   2>        3       
/solr/collections/tlog_replica_test_real_time_get/state.json
   [junit4]   2>        3       
/solr/collections/tlog_replica_test_only_leader_indexes/state.json
   [junit4]   2>        3       
/solr/collections/tlog_replica_test_delete_by_id/state.json
   [junit4]   2>        2       
/solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72184847799812101-core_node5-n_0000000000
   [junit4]   2>        2       
/solr/collections/tlog_replica_test_kill_tlog_replica/leader_elect/shard1/election/72184847799812101-core_node4-n_0000000000
   [junit4]   2>        2       
/solr/collections/tlog_replica_test_add_docs/state.json
   [junit4]   2>        2       /solr/autoscaling.json
   [junit4]   2>        2       
/solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72184847799812101-core_node16-n_0000000000
   [junit4]   2>        2       
/solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72184847799812101-core_node9-n_0000000001
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        10      /solr/live_nodes
   [junit4]   2>        10      /solr/collections
   [junit4]   2>        2       /solr/overseer/queue
   [junit4]   2>        2       /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2>        2       /solr/overseer/queue-work
   [junit4]   2>        2       /solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestTlogReplica_3C08F93888ED514E-001
   [junit4]   2> Feb 01, 2018 12:19:02 PM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene70, 
sim=RandomSimilarity(queryNorm=true): {}, locale=ar-QA, 
timezone=America/Montserrat
   [junit4]   2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 
1.8.0_152 (64-bit)/cpus=4,threads=1,free=104608064,total=503840768
   [junit4]   2> NOTE: All tests run in this JVM: [TestMacros, 
TestCodecSupport, V2ApiIntegrationTest, TestBinaryResponseWriter, 
TestApiFramework, TestManagedResource, 
OverseerCollectionConfigSetProcessorTest, ParsingFieldUpdateProcessorsTest, 
TestConfigSetProperties, TestMultiValuedNumericRangeQuery, ConvertedLegacyTest, 
TestExactStatsCache, TestExpandComponent, TestCloudPseudoReturnFields, 
ClassificationUpdateProcessorFactoryTest, SharedFSAutoReplicaFailoverTest, 
TestInPlaceUpdatesStandalone, TestAtomicUpdateErrorCases, CoreAdminHandlerTest, 
TestRTimerTree, WrapperMergePolicyFactoryTest, TestSolrCoreParser, 
CollectionsAPISolrJTest, TestNumericTerms64, FileBasedSpellCheckerTest, 
PingRequestHandlerTest, TestCustomSort, SpellPossibilityIteratorTest, 
CircularListTest, BufferStoreTest, ShowFileRequestHandlerTest, 
TestInfoStreamLogging, TestLMJelinekMercerSimilarityFactory, 
TestNonDefinedSimilarityFactory, LIRRollingUpdatesTest, 
DistributedSuggestComponentTest, TestXmlQParserPlugin, FullHLLTest, 
IndexSchemaRuntimeFieldTest, ReplaceNodeTest, RollingRestartTest, 
TestDistribDocBasedVersion, CustomCollectionTest, TestUtils, 
TestCloudInspectUtil, TestManagedSynonymFilterFactory, TestUpdate, 
ChaosMonkeyNothingIsSafeWithPullReplicasTest, URLClassifyProcessorTest, 
TestFieldTypeCollectionResource, TestJsonFacets, TestLegacyFieldReuse, 
CursorMarkTest, DocValuesMultiTest, CacheHeaderTest, TestSolrDeletionPolicy2, 
CdcrBidirectionalTest, UniqFieldsUpdateProcessorFactoryTest, 
TestSolrCoreProperties, TestClusterStateMutator, ReplicaListTransformerTest, 
TestShortCircuitedRequests, TestStressCloudBlindAtomicUpdates, 
SimpleMLTQParserTest, DateRangeFieldTest, InfoHandlerTest, 
SolrCmdDistributorTest, MoreLikeThisHandlerTest, DistanceFunctionTest, 
RecoveryZkTest, TestRealTimeGet, TestLockTree, ConjunctionSolrSpellCheckerTest, 
TestPrepRecovery, TestHdfsBackupRestoreCore, PreAnalyzedFieldTest, 
TestConfigSetsAPIZkFailure, TestRequestStatusCollectionAPI, TestBlobHandler, 
TestSchemaResource, AnalysisAfterCoreReloadTest, TestCloudNestedDocsSort, 
TestCustomStream, TestDFRSimilarityFactory, TestDefaultStatsCache, 
AutoScalingHandlerTest, LeaderElectionIntegrationTest, 
StatelessScriptUpdateProcessorFactoryTest, 
VMParamsZkACLAndCredentialsProvidersTest, ClusterStateTest, 
ClusterStateUpdateTest, TimeZoneUtilsTest, SolrXmlInZkTest, 
HdfsUnloadDistributedZkTest, HdfsChaosMonkeyNothingIsSafeTest, 
DateMathParserTest, TestPushWriter, AddSchemaFieldsUpdateProcessorFactoryTest, 
TestCloudManagedSchema, CoreAdminCreateDiscoverTest, ConnectionManagerTest, 
TestRetrieveFieldsOptimizer, TestStressReorder, SearchHandlerTest, 
TestFieldCacheWithThreads, TolerantUpdateProcessorTest, 
BlobRepositoryCloudTest, TestIntervalFaceting, SolrGraphiteReporterTest, 
SystemLogListenerTest, TestSchemalessBufferedUpdates, 
ManagedSchemaRoundRobinCloudTest, DistributedFacetPivotLongTailTest, 
TestUtilizeNode, TestLazyCores, TestNumericTokenStream, TestJmxIntegration, 
TestCoreDiscovery, CollectionsAPIAsyncDistributedZkTest, 
DistribCursorPagingTest, SolrMetricReporterTest, 
TestLeaderElectionWithEmptyReplica, DeleteReplicaTest, 
TestDynamicFieldResource, TestSortByMinMaxFunction, 
TemplateUpdateProcessorTest, TestClusterStateProvider, TestValueSourceCache, 
FieldAnalysisRequestHandlerTest, ReturnFieldsTest, JsonLoaderTest, 
TestQuerySenderListener, TestQuerySenderNoQuery, ResponseLogComponentTest, 
TestSearchPerf, MinimalSchemaTest, DirectSolrConnectionTest, 
NotRequiredUniqueKeyTest, TestCharFilters, SynonymTokenizerTest, 
EchoParamsTest, ResourceLoaderTest, TestFastOutputStream, 
ChaosMonkeyNothingIsSafeTest, TestTolerantSearch, 
TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, 
ConnectionReuseTest, AliasIntegrationTest, CreateCollectionCleanupTest, 
DeleteShardTest, DistributedVersionInfoTest, HttpPartitionTest, 
MoveReplicaTest, MultiThreadedOCPTest, RemoteQueryErrorTest, 
ReplicationFactorTest, RestartWhileUpdatingTest, TestRebalanceLeaders, 
TestTlogReplica]
   [junit4] Completed [558/773 (5!)] on J2 in 136.90s, 13 tests, 1 failure <<< 
FAILURES!

[...truncated 48867 lines...]

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org