[
https://issues.apache.org/jira/browse/CASSANDRA-13797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382839#comment-16382839
]
Vincent White commented on CASSANDRA-13797:
-------------------------------------------
Now that we don't wait for the validations of each repair job to finish before
moving onto the next one, I don't see anything to stop the repair coordinator
from spinning through all the token ranges and effectively triggering all the
validations tasks at once, which could be a significant amount of validation
compactions on each node depending on your topology and common ranges for that
keyspace. I'm also not sure of the overhead of creating all the
futures/listeners on the coordinator at once in this case.
In 3 the validation executor thread pool has no size limit so a new validation
is started as soon as a validation request is received. I admit I haven't
caught up on the changes to repair in trunk, and while the validation executor
pool size is configurable in trunk, its default is still Integer.MAX_VALUE.
I understand this same affect (hundreds of concurrent validations) can still
happen if you trigger a repair across a keyspace with a large number of column
families but with this change there is no way of avoiding it without using
subrange repairs on a single column family (if you have a topology/replication
that cant be merged into a small number of common ranges) .
> RepairJob blocks on syncTasks
> -----------------------------
>
> Key: CASSANDRA-13797
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13797
> Project: Cassandra
> Issue Type: Bug
> Components: Repair
> Reporter: Blake Eggleston
> Assignee: Blake Eggleston
> Priority: Major
> Fix For: 3.0.15, 3.11.1, 4.0
>
>
> The thread running {{RepairJob}} blocks while it waits for the validations it
> starts to complete ([see
> here|https://github.com/bdeggleston/cassandra/blob/9fdec0a82851f5c35cd21d02e8c4da8fc685edb2/src/java/org/apache/cassandra/repair/RepairJob.java#L185]).
> However, the downstream callbacks (ie: the post-repair cleanup stuff) aren't
> waiting for {{RepairJob#run}} to return, they're waiting for a result to be
> set on RepairJob the future, which happens after the sync tasks have
> completed. This post repair cleanup stuff also immediately shuts down the
> executor {{RepairJob#run}} is running in. So in noop repair sessions, where
> there's nothing to stream, I'm seeing the callbacks sometimes fire before
> {{RepairJob#run}} wakes up, and causing an {{InterruptedException}} is thrown.
> I'm pretty sure this can just be removed, but I'd like a second opinion. This
> appears to just be a holdover from before repair coordination became async. I
> thought it might be doing some throttling by blocking, but each repair
> session gets it's own executor, and validation is throttled by the fixed
> size executors doing the actual work of validation, so I don't think we need
> to keep this around.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
