keith-turner commented on code in PR #5129:
URL: https://github.com/apache/accumulo/pull/5129#discussion_r1872043471
##########
core/src/main/java/org/apache/accumulo/core/fate/Fate.java:
##########
@@ -372,6 +375,42 @@ public Fate(T environment, FateStore<T> store, boolean
runDeadResCleaner,
break;
}
}
+ queueSizeHistory.clear();
+ } else {
+ // The property did not change, but should it based on the queue size?
Maintain
+ // the last X minutes of queue sizes. If the queue size is always
larger than the number
+ // of Fate threads multiplied by some factor, then suggest that the
+ // MANAGER_FATE_THREADPOOL_SIZE be increased.
+ final long interval = Math.min(60, TimeUnit.MILLISECONDS
+
.toMinutes(conf.getTimeInMillis(Property.MANAGER_FATE_QUEUE_CHECK_INTERVAL)));
+ if (interval == 0) {
+ queueSizeHistory.clear();
+ } else {
+ final int sizeFactor =
conf.getCount(Property.MANAGER_FATE_QUEUE_CHECK_FACTOR);
+ if (queueSizeHistory.size() >= interval * 2) { // this task runs
every 30s
+ final int warnThreshold = configured * sizeFactor;
+ boolean needMoreThreads = true;
+ for (Integer i : queueSizeHistory) {
+ if (i < warnThreshold) {
+ needMoreThreads = false;
+ break;
+ }
+ }
+ if (needMoreThreads) {
+ log.warn(
+ "Fate queue size is {} times the number of Fate threads for
the last {} minutes,"
+ + " consider increasing property: {}",
+ sizeFactor, interval,
Property.MANAGER_FATE_THREADPOOL_SIZE.getKey());
+ // Clear the history so that we don't log for another 5 minutes.
+ queueSizeHistory.clear();
+ } else {
+ while (queueSizeHistory.size() >= interval * 2) {
+ queueSizeHistory.remove();
+ }
+ }
+ }
+ queueSizeHistory.add(pool.getQueue().size());
Review Comment:
Individual fate task are not placed on this queue. The TransactionRunner()
task that are added to the thread pool run in an infinite loop and pull data
from a separate TransferQueue (I think you suggested using that on some older
PR to avoid complex bookeeping to avoid dupes).
Unrelated to this PR, looking at the way the existing code works it seems
like the # worke threads can grow based on config changes but will not shrink
based on config changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
