kfaraz commented on code in PR #19562:
URL: https://github.com/apache/druid/pull/19562#discussion_r3467675632
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/autoscaler/WeightedCostFunction.java:
##########
@@ -42,6 +42,12 @@ public class WeightedCostFunction
*/
static final double LAG_AMPLIFICATION_MULTIPLIER = 0.4;
+ /**
+ * Exponent (< 1) for sublinear busy redistribution in the idle
projection: busy grows as
+ * {@code (currentTaskCount / proposedTaskCount)^EXPONENT}, not linearly.
Calibrated as log2(1.25) ~= 0.32.
+ */
+ static final double IDLE_SUBLINEARITY_EXPONENT = 0.32;
Review Comment:
Why this specific value?
I feel that these constants somehow make the behaviour of the auto-scaler
more difficult to reason about as well as control via the weights.
We should probably not be trying to predict the idleness in this manner
anyway.
Instead, as an alternative, we should probably consider using the `avg.
processing rate * task count * idle ratio` as a measure of total work that
needs to be done. Assuming that the processing rate remains the same, we can
find the new idle ratio for the new task count.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
