[
https://issues.apache.org/jira/browse/IGNITE-24011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladislav Pyatkov updated IGNITE-24011:
---------------------------------------
Description:
h3. Motivation
Despite the fact that we increased the amount of log slots previously
(IGNITE-23837), we still might get an issue with the exhausted slots. But the
data streamer has not to block itself in any amount of slots.
The streamer is implemented as a batch partition insert. It required as many
slots as the batch size (StreamerOptions#pageSize) multiplied by the amount of
parallel batch operation (StreamerOptions#perPartitionParallelOperations). We
must ensure the streamer works in an extremely low count of slots.
h3. Implementation notes
The main idea is reducing the amount of locks from the one streamer batch. It
is a count of entries that is handled in InternalTable#updateAll.
The amount of entries that would be sent to the replica side (in
{{ReadWriteMultiRowReplicaRequest}}) has to be less than the number of slots in
LockManager (for example, 10 times less). The lock manager slots capacity is
available in configuration IGNITE-20694).
In case we get a lock error ({{ACQUIRE_LOCK_ERR}}) we will divide the batch
(for example, divide in half) and send the batch (in
{{ReadWriteMultiRowReplicaRequest}}) even less.
h3. Definition of done
Streamer works in an environment with a few lock slots.
was:
h3. Motivation
Despite the fact that we increased the amount of log slots previously
(IGNITE-23837), we still might get an issue with the exhausted slots. But the
data streamer has not to block itself in any amount of slots.
The streamer is implemented as a batch partition insert. It required as many
slots as the batch size (StreamerOptions#pageSize) multiplied by the amount of
parallel batch operation (StreamerOptions#perPartitionParallelOperations). We
must ensure the streamer works in an extremely low count of slots.
h3. Definition of done
Streamer works in an environment with a few lock slots.
> Make streamer tolerance to exhausted lock slots
> -----------------------------------------------
>
> Key: IGNITE-24011
> URL: https://issues.apache.org/jira/browse/IGNITE-24011
> Project: Ignite
> Issue Type: Improvement
> Reporter: Vladislav Pyatkov
> Priority: Major
> Labels: ignite-3
>
> h3. Motivation
> Despite the fact that we increased the amount of log slots previously
> (IGNITE-23837), we still might get an issue with the exhausted slots. But the
> data streamer has not to block itself in any amount of slots.
> The streamer is implemented as a batch partition insert. It required as many
> slots as the batch size (StreamerOptions#pageSize) multiplied by the amount
> of parallel batch operation (StreamerOptions#perPartitionParallelOperations).
> We must ensure the streamer works in an extremely low count of slots.
> h3. Implementation notes
> The main idea is reducing the amount of locks from the one streamer batch. It
> is a count of entries that is handled in InternalTable#updateAll.
> The amount of entries that would be sent to the replica side (in
> {{ReadWriteMultiRowReplicaRequest}}) has to be less than the number of slots
> in LockManager (for example, 10 times less). The lock manager slots capacity
> is available in configuration IGNITE-20694).
> In case we get a lock error ({{ACQUIRE_LOCK_ERR}}) we will divide the batch
> (for example, divide in half) and send the batch (in
> {{ReadWriteMultiRowReplicaRequest}}) even less.
> h3. Definition of done
> Streamer works in an environment with a few lock slots.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
