[
https://issues.apache.org/jira/browse/CAMEL-9355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15029166#comment-15029166
]
Aaron Whiteside commented on CAMEL-9355:
----------------------------------------
The problem we are seeing with the current throttler, is that with a single
thread we are never quite able to reach the actual throttle limit, and with
multiple threads in a route we end up going over the throttle limit. For our
business use-case this is disastrous, we're not able to meeting any of our
SLAs.
With the semaphore based implementation requests are not actually spread evenly
over the time period, as you can consume say a throttle rate of 100/s in the
first 1ms then be blocked for the remaining 999ms.
Another benefit of the semaphore implementation is that it's not based on a
relative point of time reference. For example: the senders time slot starts at
6 seconds past the minute, vs the receivers time slot starts at 4 seconds past
the minute- assuming the clocks are synchronized. This can lead to the receiver
rejecting requests because they exceed his throttle, when from the senders
point of view he has not exceeded the throttle.
Of course this depends on how the receiver calculates received throughput. But
it is always better to use a rolling window of the last "time period" (Eg. 1s),
relative to the current time/current request. It may not be obvious at first
but the semaphore is a rolling window of time.
When using a semaphore it's not really possible to have the permit be released
without using another thread. There are other approaches but they suffer from
the problem of assuming the time period window starts at some fixed time
relative to them and have the issues mentioned above, or have very high
overheads.
Such an approach, that was not a semaphore, we were using a few years ago was a
throttling implementation based on a sentinel (head and tail are the same
object) linked list. Where we would add items to this queue with a timestamp
and then when checking if we exceeded the throttle, we would count back through
the items until we exceeded our throttle or reached a timestamp outside our
time period (again relative to the current time of the request). This achieved
the goal but was quite expensive, as we could end up looping over
o(throttle_rate) items, per requesting thread, to see if we were within our
throttle rate. With a high throttle rate and high concurrency this became a
bottle neck. Not to mention that you cannot just let the list grow unchecked,
it had to be pruned either with a dedicated thread or delegating the work to
each calling thread, slowing them down even more.
If this change is too radical, would you consider a throttler2 implementation,
living along side the original throttler?
> Current Throttler implementation is not accurate and does not work in a
> multi-threaded route
> --------------------------------------------------------------------------------------------
>
> Key: CAMEL-9355
> URL: https://issues.apache.org/jira/browse/CAMEL-9355
> Project: Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 2.16.0
> Reporter: Aaron Whiteside
> Attachments: CAMEL_9355.patch
>
>
> Current Throttler implementation is not accurate and is even more inaccurate
> in a multi-threaded route (think sjms....consumerCount=100).
> The delay to sleep cannot be calculated ahead of time in a multi-threaded
> environment, to this end the Throttler should not extend
> DelayProcessorSupport.
> Attached is a patch that changes throttler to use a Semaphore to do accurate
> and multi-thread safe throttling.
> The code I think is much cleaner, smaller and easier to understand. Than it
> used to be before.
> Unit tests still pass, I had to make some changes to ThrottlerTest as it made
> assumptions about the implementation and was doing bad things like adding a
> 750ms buffer to validating the minimum throttle delay.. ThrottlerTest is now
> very sane.
> I've also implemented support to allow the throttler construct to be used
> without any nested outputs. For example the follow code is now valid.
> {code}
> <throttle><constant>100</constant></throttle>
> {code}
> If you want to disable this feature it can be done in
> ThrottlerDefinition::createProcessor() line 82 changing false to true.
> I think this allows more flexible usage of the throttler, in my use case I
> want to delay the further execution of the route, and I don't want to have to
> split my routes up into separate sub-routes to be able to do that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
