[
https://issues.apache.org/jira/browse/ACCUMULO-4469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15516854#comment-15516854
]
Sean Busbey commented on ACCUMULO-4469:
---------------------------------------
dug in.
When we stop the service
[here|https://github.com/apache/accumulo/blob/master/test/src/main/java/org/apache/accumulo/test/randomwalk/Module.java#L359]:
{code}
} finally {
if (null != service) {
service.shutdownNow();
}
}
{code}
[shutdownNow|https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html#shutdownNow()]
does not wait for currently executing tasks to finish. So on our last node
visit, we'll get [a TimeOutException and exit the
loop|https://github.com/apache/accumulo/blob/master/test/src/main/java/org/apache/accumulo/test/randomwalk/Module.java#L305],
but that last task may still be running by the time we get to our fixture
cleanup.
we should add a call to
[awaitTermination|https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html#awaitTermination(long,%20java.util.concurrent.TimeUnit)]
to wait for it to finish, I think.
The fixture clean up is the only thing that can run in parallel, since the
randomwalk executor only has one thread. We should make a copy of the list, but
that also might get a concurrent modification exception (e.g. if the
waitTermination timedout). It should be rare, so I think we should just retry
making a deep copy. (using CopyOnWriteArrayList would avoid the concurrent
modification exception but we might miss a table to delete if the final
operation was one of those that adds to the list.)
> ConcurrentModificationException while running MultiTable.xml node in Random
> Walk
> ---------------------------------------------------------------------------------
>
> Key: ACCUMULO-4469
> URL: https://issues.apache.org/jira/browse/ACCUMULO-4469
> Project: Accumulo
> Issue Type: Bug
> Components: test
> Affects Versions: 1.7.2
> Reporter: Dima Spivak
> Assignee: Dima Spivak
>
> After the resolution of ACCUMULO-4467, I got back to playing with Random Walk
> and had a failure caused by a {{ConcurrentModificationException}}:
> {code}
> 23 01:03:04,316 [randomwalk.Framework] ERROR: Error during random walk
> java.lang.Exception: Error running node MultiTable.xml
> at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:346)
> at
> org.apache.accumulo.test.randomwalk.Framework.run(Framework.java:59)
> at
> org.apache.accumulo.test.randomwalk.Framework.main(Framework.java:119)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.accumulo.start.Main$2.run(Main.java:157)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859)
> at java.util.ArrayList$Itr.next(ArrayList.java:831)
> at
> org.apache.accumulo.test.randomwalk.multitable.MultiTableFixture.tearDown(MultiTableFixture.java:64)
> at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:365)
> at org.apache.accumulo.test.randomwalk.Module$1.call(Module.java:283)
> at org.apache.accumulo.test.randomwalk.Module$1.call(Module.java:278)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at
> org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
> ... 1 more
> {code}
> [This section of
> code|https://github.com/apache/accumulo/blob/master/test/src/main/java/org/apache/accumulo/test/randomwalk/multitable/MultiTableFixture.java#L61-L71]
> seems to be at fault. In particular, it looks like we're getting the table
> list, but then instead of doing a deep copy to a new {{ArrayList<String>}}
> from which we choose tables to delete, we're looping through and deleting
> tables while referring to the changing list, which has the effect of
> modifying it and making Java unhappy. Am I missing something more complex or
> can I fix this one myself by just doing the aforementioned deep copy of the
> table list? Or is a better way to use the {{TableOperations.list()}} method
> and iterate through the {{SortedSet<String>}} it provides?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
