See <http://sierranevada.servebeer.com:8080/job/slow-io-beasting/4882/>
------------------------------------------
[...truncated 13995 lines...]
[junit4:junit4] 2> 690920 T20 oazs.SessionTrackerImpl.run SessionTrackerImpl
exited loop!
[junit4:junit4] 2> 693166 T19 oazs.NIOServerCnxn$Factory.run WARNING Ignoring
unexpected runtime exception java.nio.channels.CancelledKeyException
[junit4:junit4] 2> at
sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
[junit4:junit4] 2> at
sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:69)
[junit4:junit4] 2> at
org.apache.zookeeper.server.NIOServerCnxn$Factory.run(NIOServerCnxn.java:241)
[junit4:junit4] 2>
[junit4:junit4] 2> 693166 T17 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:49875 which had sessionid 0x13a9dfc31710003
[junit4:junit4] 2> 693166 T51 oaz.ClientCnxn$SendThread.run Unable to read
additional data from server sessionid 0x13a9dfc31710003, likely server has
closed socket, closing socket connection and attempting reconnect
[junit4:junit4] 2> 693166 T17 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:49887 which had sessionid 0x13a9dfc31710004
[junit4:junit4] 2> 693166 T63 oaz.ClientCnxn$SendThread.run Unable to read
additional data from server sessionid 0x13a9dfc31710004, likely server has
closed socket, closing socket connection and attempting reconnect
[junit4:junit4] 2> 693166 T17 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:49869 which had sessionid 0x13a9dfc31710002
[junit4:junit4] 2> 693166 T37 oaz.ClientCnxn$SendThread.run Unable to read
additional data from server sessionid 0x13a9dfc31710002, likely server has
closed socket, closing socket connection and attempting reconnect
[junit4:junit4] 2> 693166 T17 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:49905 which had sessionid 0x13a9dfc31710005
[junit4:junit4] 2> 693166 T75 oaz.ClientCnxn$SendThread.run Unable to read
additional data from server sessionid 0x13a9dfc31710005, likely server has
closed socket, closing socket connection and attempting reconnect
[junit4:junit4] 2> 693166 T17 oazs.NIOServerCnxn.closeSock Closed socket
connection for client /127.0.0.1:50590 which had sessionid 0x13a9dfc31710006
[junit4:junit4] 2> 693166 T19 oazs.NIOServerCnxn$Factory.run NIOServerCnxn
factory exited run method
[junit4:junit4] 2> 693166 T87 oaz.ClientCnxn$SendThread.run Unable to read
additional data from server sessionid 0x13a9dfc31710006, likely server has
closed socket, closing socket connection and attempting reconnect
[junit4:junit4] 2> 693166 T17 oazs.FinalRequestProcessor.shutdown shutdown of
request processor complete
[junit4:junit4] 2> 693166 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
49864
[junit4:junit4] 2> 693166 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=12689343
[junit4:junit4] 2> 693166 T17 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@343bb6
[junit4:junit4] 2> 693182 T17 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=0,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 693182 T17 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 693182 T17 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 693182 T17 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 693182 T17 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 693182 T39 oasc.Overseer$ClusterStateUpdater.amILeader
According to ZK I (id=88556849173954562-127.0.0.1:49864_solr-n_0000000000) am
no longer a leader.
[junit4:junit4] 2> 693276 T76 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1b9e7fc name:ZooKeeperConnection
Watcher:127.0.0.1:49846/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 693276 T64 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@18bbc98 name:ZooKeeperConnection
Watcher:127.0.0.1:49846/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 693276 T76 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 693276 T64 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 693276 T88 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@18b4ccb name:ZooKeeperConnection
Watcher:127.0.0.1:49846/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 693276 T52 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@1f93ace name:ZooKeeperConnection
Watcher:127.0.0.1:49846/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 693276 T17 oaz.ZooKeeper.close Session: 0x13a9dfc31710002
closed
[junit4:junit4] 2> 693276 T88 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 693276 T38 oascc.ConnectionManager.process Watcher
org.apache.solr.common.cloud.ConnectionManager@70c38c name:ZooKeeperConnection
Watcher:127.0.0.1:49846/solr got event WatchedEvent state:Disconnected
type:None path:null path:null type:None
[junit4:junit4] 2> 693276 T38 oascc.ConnectionManager.process
Client->ZooKeeper status change trigger but we are already closed
[junit4:junit4] 2> 693276 T52 oascc.ConnectionManager.process zkClient has
disconnected
[junit4:junit4] 2> 693276 T38 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 693277 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 693330 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
49870
[junit4:junit4] 2> 693330 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=7267332
[junit4:junit4] 2> 693330 T17 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@12088db
[junit4:junit4] 2> 693338 T17 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=0,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 693338 T17 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 693338 T17 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 693339 T17 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 693339 T17 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 694571 T75 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49846
[junit4:junit4] 2> 694681 T51 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49846
[junit4:junit4] 2> 695227 T87 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49846
[junit4:junit4] 2> 695242 T63 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49846
[junit4:junit4] 2> 695570 T75 oaz.ClientCnxn$SendThread.run WARNING Session
0x13a9dfc31710005 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 695735 T52 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 695735 T17 oaz.ZooKeeper.close Session: 0x13a9dfc31710003
closed
[junit4:junit4] 2> 695745 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 695796 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
49882
[junit4:junit4] 2> 695796 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=7080114
[junit4:junit4] 2> 695796 T17 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@19ab00e
[junit4:junit4] 2> 695801 T17 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=1,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 695802 T17 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 695802 T17 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 695802 T17 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 695804 T17 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 696221 T87 oaz.ClientCnxn$SendThread.run WARNING Session
0x13a9dfc31710006 for server null, unexpected error, closing socket connection
and attempting reconnect java.net.ConnectException: Connection refused: no
further information
[junit4:junit4] 2> at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
[junit4:junit4] 2> at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
[junit4:junit4] 2> at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1143)
[junit4:junit4] 2>
[junit4:junit4] 2> 696361 T64 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 696361 T17 oaz.ZooKeeper.close Session: 0x13a9dfc31710004
closed
[junit4:junit4] 2> 696377 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 696428 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
49900
[junit4:junit4] 2> 696428 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=263670
[junit4:junit4] 2> 696865 T75 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49846
[junit4:junit4] 2> 697770 T87 oaz.ClientCnxn$SendThread.startConnect Opening
socket connection to server 127.0.0.1/127.0.0.1:49846
[junit4:junit4] 2> 697972 T76 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 697972 T17 oaz.ZooKeeper.close Session: 0x13a9dfc31710005
closed
[junit4:junit4] 2> 697988 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 698039 T17 oasc.ChaosMonkey.monkeyLog monkey: stop shard!
50585
[junit4:junit4] 2> 698039 T17 oasc.CoreContainer.shutdown Shutting down
CoreContainer instance=21497915
[junit4:junit4] 2> 698039 T17 oasc.SolrCore.close [collection1] CLOSING
SolrCore org.apache.solr.core.SolrCore@8b3bb3
[junit4:junit4] 2> 698070 T17 oasu.DirectUpdateHandler2.close closing
DirectUpdateHandler2{commits=1,autocommits=0,soft
autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 698070 T17 oasc.SolrCore.decrefSolrCoreState Closing
SolrCoreState
[junit4:junit4] 2> 698070 T17 oasu.DefaultSolrCoreState.closeIndexWriter
SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 698070 T17 oasu.DefaultSolrCoreState.closeIndexWriter
closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 698070 T17 oasc.SolrCore.closeSearcher [collection1]
Closing main searcher on request.
[junit4:junit4] 2> 698881 T88 oaz.ClientCnxn$EventThread.run EventThread shut
down
[junit4:junit4] 2> 698881 T17 oaz.ZooKeeper.close Session: 0x13a9dfc31710006
closed
[junit4:junit4] 2> 698897 T17 oejsh.ContextHandler.doStop stopped
o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> 699151 T17 oas.SolrTestCaseJ4.tearDown ###Ending
testDistribSearch
[junit4:junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=BasicDistributedZkTest -Dtests.method=testDistribSearch
-Dtests.seed=303D619227FAC5C1 -Dtests.slow=true -Dtests.locale=en_AU
-Dtests.timezone=America/Argentina/Buenos_Aires -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 699s J0 | BasicDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: There are still
nodes recoverying
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([303D619227FAC5C1:B1DBEF8A50A5A5FD]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:115)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:110)
[junit4:junit4] > at
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:553)
[junit4:junit4] > at
org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:162)
[junit4:junit4] > at
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4:junit4] > at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4] > at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 699198 T17 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 821892 T17 oas.SolrTestCaseJ4.endTrackingSearchers SEVERE
ERROR: SolrIndexSearcher opens=7 closes=6
[junit4:junit4] 2> 821892 T16 ccr.ThreadLeakControl.checkThreadLeaks WARNING
Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 842047 T16 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1
thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] 2> 1) Thread[id=77, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] 2> at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 842047 T16 ccr.ThreadLeakControl.tryToInterruptAll
Starting to interrupt leaked threads:
[junit4:junit4] 2> 1) Thread[id=77, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> 845136 T16 ccr.ThreadLeakControl.tryToInterruptAll SEVERE
There are still zombie threads that couldn't be terminated:
[junit4:junit4] 2> 1) Thread[id=77, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] 2> at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] 2> at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40,
sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=en_AU,
timezone=America/Argentina/Buenos_Aires
[junit4:junit4] 2> NOTE: Windows Vista 6.0 x86/Sun Microsystems Inc. 1.6.0_23
(32-bit)/cpus=4,threads=2,free=5811016,total=17055744
[junit4:junit4] 2> NOTE: All tests run in this JVM:
[CoreContainerCoreInitFailuresTest, SimplePostToolTest, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=BasicDistributedZkTest -Dtests.seed=303D619227FAC5C1
-Dtests.slow=true -Dtests.locale=en_AU
-Dtests.timezone=America/Argentina/Buenos_Aires -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: ERROR:
SolrIndexSearcher opens=7 closes=6
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([303D619227FAC5C1]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at
org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:242)
[junit4:junit4] > at
org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:97)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[junit4:junit4] > at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4] > at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] > Throwable #2:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE
scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=77, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] > at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] > at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] > at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([303D619227FAC5C1]:0)
[junit4:junit4] > Throwable #3:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie
threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=77, name=searcherExecutor-44-thread-1,
state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] > at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
[junit4:junit4] > at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
[junit4:junit4] > at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
[junit4:junit4] > at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] > at
__randomizedtesting.SeedInfo.seed([303D619227FAC5C1]:0)
[junit4:junit4] Completed on J0 in 845.51s, 1 test, 2 failures, 2 errors <<<
FAILURES!
[junit4:junit4]
[junit4:junit4]
[junit4:junit4] Tests with failures:
[junit4:junit4] -
org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch
[junit4:junit4] - org.apache.solr.cloud.BasicDistributedZkTest (suite)
[junit4:junit4]
[junit4:junit4]
[junit4:junit4] JVM J0: 2.57 .. 867.91 = 865.34s
[junit4:junit4] JVM J1: 2.31 .. 225.68 = 223.37s
[junit4:junit4] JVM J2: 4.16 .. 225.38 = 221.21s
[junit4:junit4] JVM J3: 2.31 .. 218.48 = 216.17s
[junit4:junit4] JVM J4: 2.57 .. 219.21 = 216.64s
[junit4:junit4] JVM J5: 2.57 .. 261.88 = 259.30s
[junit4:junit4] JVM J6: 2.04 .. 218.93 = 216.89s
[junit4:junit4] JVM J7: 2.57 .. 266.15 = 263.57s
[junit4:junit4] Execution time total: 14 minutes 27 seconds
[junit4:junit4] Tests summary: 233 suites, 965 tests, 3 suite-level errors, 1
failure, 12 ignored (7 assumptions)
BUILD FAILED
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/build.xml>:39:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/solr/build.xml>:178:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/module-build.xml>:61:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:1171:
The following error occurred while executing this line:
<http://sierranevada.servebeer.com:8080/job/slow-io-beasting/ws/lucene/common-build.xml>:839:
There were test failures: 233 suites, 965 tests, 3 suite-level errors, 1
failure, 12 ignored (7 assumptions)
Total time: 21 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
