[
https://issues.apache.org/jira/browse/IGNITE-24548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17939181#comment-17939181
]
Ivan Bessonov edited comment on IGNITE-24548 at 3/28/25 2:35 PM:
-----------------------------------------------------------------
h4. Investigation results:
* Target-ration based throttling with its exponential backoff strategy doesn't
really fit into real scenarios. It tanks the performance very fast, because it
never relies on real sizes of data regions and checkpoint buffers. If we want
to use this implementation, we should modify it heavily so that it dynamically
adjusts to external conditions. But we already have an attempt of such an
implementation.
* Speed-based throttling does save us in cases of high load. But the downside
is that it slows down the system when there's no high load, especially in tests
where there's barely any load. When enabled by default, it turns a 6 minute
{{ItSqlLogicTest}} into ~25 minute test. The overhead during the first
checkpoint (or maybe in general) is too high. {{onMarkDirty}} method is also
pretty slow by itself, I guess we never tuned its implementation. Calculations
that it performs are quite heavy on the CPU and {*}take about as much time as
thread parking itself{*}.
h4. What we should do:
* Enable speed-based throttling by default, but disable it in tests. Real
benchmarks don't have the same issues that we see in integration tests.
* We should work on optimizing the implementation itself, so that integration
tests pass equally fast with and without throttling.
* Ideally, we should smooth-out the performance, because it jumps from "mid"
to "high" constantly (correlated to checkpoints). I have no ideas of how to do
that, it might even be impossible. Or it might make average performance worse,
and those peaks in throughput are actually a good thing, I can't say for sure.
h4. Benchmark results:
||Case||Servers||Clients||Without throttling||With throttling||
|KV|2|2|199459|197719|
|KV TX|2|2|97816|100043|
|KV Batch|2|1|409556|390950|
|KV + Indexes|2|2|34361|30800|
Everything is more or less withing a margin of error.
was (Author: ibessonov):
Investigation results:
* Target-ration based throttling with its exponential backoff strategy doesn't
really fit into real scenarios. It tanks the performance very fast, because it
never relies on real sizes of data regions and checkpoint buffers. If we want
to use this implementation, we should modify it heavily so that it dynamically
adjusts to external conditions. But we already have an attempt of such an
implementation.
* Speed-based throttling does save us in cases of high load. But the downside
is that it slows down the system when there's no high load ({*}TODO provide
benchmark results{*}). When enabled by default, it turns a 6 minute
{{ItSqlLogicTest}} into ~25 minute test. The overhead during the first
checkpoint (or maybe in general) is too high. {{onMarkDirty}} method is also
pretty slow by itself, I guess we never tuned its implementation. Calculations
that it performs are quite heavy on the CPU and take about as much time as
thread parking itself.
What we should do:
* Enable speed-based throttling by default, but disable it in tests. Real
benchmarks don't have the same issues that we see in integration tests.
* We should work on optimizing the implementation itself, so that integration
tests pass equally fast with and without throttling.
* Ideally, we should smooth-out the performance, because it jumps from "mid"
to "high" constantly due to checkpoints. I have no ideas of how to do that, it
might even be impossible. Or it might make average performance worse, and those
peaks in throughput are actually a good thing, I can't say for sure.
> Compare throttling strategies in Ignite 3
> -----------------------------------------
>
> Key: IGNITE-24548
> URL: https://issues.apache.org/jira/browse/IGNITE-24548
> Project: Ignite
> Issue Type: Improvement
> Reporter: Ivan Bessonov
> Assignee: Ivan Bessonov
> Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We need to compare target ratio based and speed based throttling
> implementations in a number heavy tests and choose the default.
> We could provide an option to change the default through the unmanaged part
> of configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
