GitHub user benstopford opened a pull request:
https://github.com/apache/kafka/pull/1893
KAFKA-4178: Windows in Rate Calculation
ReplicationQuotas included a new Rate class which changed the way the
window is calculated. @junrao asked that we look to consolidate this. On
balance I believe there is a good case for both, so this PR pulls out two
policies for calculating the window, and attempts to explain what they are and
why we might need them.
They are:
**Elapsed**: Set the window to the difference between the oldest and newest
measurement time. Handle NaN. _(Replication Quotas use this)_
**Fixed**: Fix the window to the full duration (10s by default). _(Client
Quotas use this)_
Replication Quotas uses **Elapsed** as **Fixed** provides underestimates
during the first window. Underestimates are best avoided as they create a load
spike when replication starts. So it's preferable to overestimate the rate, and
hence increase the throttled rate slowly. Elapsed is also significantly easier
to test.
I'm totally sure why Client Quotas were changed to use **Fixed**. I know
there was a NaN issue, but should really be tangential (it is handled in both
policies). However it seems sensible to slowly throttle on a client's
connection down to the desired rate when you initialise or change quotas.
So I think there is a requirement for both types of rate, and hence I've
included both in this PR. **Elapsed** suits throttled replication, where the
general use case is to initiate some rebalance and immediately apply a
conservative throttle. **Fixed** suits client quotas where we want to gently
throttle a client down to the desired rate over a period of seconds.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/benstopford/kafka KAFKA-4178
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1893.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1893
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
