[
https://issues.apache.org/jira/browse/IGNITE-27844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18059912#comment-18059912
]
Ivan Bessonov edited comment on IGNITE-27844 at 2/20/26 1:45 PM:
-----------------------------------------------------------------
The problem is in {{IndexBuildTask}}. To be more precise - it's the way it
reads the {{nextRowIdToBuild}} and constructs batches. For the sake of
simplicity I will use "batchSize=2" in the following demonstration: # Imagine
we have a partition with rows {{[1, 2, 3]}}. Primary replica is not collocated
with leader. Raft group has *2* peers.
# Build index process creates command {{A: \{index [1, 2]}}}
# Primary replica successfully replies after the replication of this command,
but it's only been applied on the leader. Locally command is still not applied.
# Build index process reads the {{nextRowIdToBuild}} and receives
{{minRowId}}. Because of that it create a command {{B: \{index [1, 2]}}},
which is identical to {{A}}.
# Primary replica successfully replies, but command {{B}} is not yet applied
locally. We're exactly in-between commands {{A}} and {{B}}.
# Build index process creates new command {{C: {index [3]}}} based on the
storage status after the application of {{A}}. The number of entries here is
less than {{batchSize}}, which means that the command will be replicated with
{{finish=true}}.
# Primary replica successfully replies. It so happened that we're in the
middle of applying the command {{B}}, and current value of {{nextRowIdToBuild}}
is {{2}}.
# Build index process uses this value to create command {{D: {index [2, 3]}}}.
Here we have 2 rows, which is equal to {{batchSize}}, so {{finish=false}} for
this command.
# Primary replica successfully replies. At this moment command {{D}} is not
applied, but command {{C}} is applied. Command {{C}} had {{finish=true}},
locally {{nextRowIdToBuild=null}}. Index build process stops.
# After that command {{D}} is applied. It had {{finish=false}}, which changes
{{nextRowIdToBuild}} from {{null}} to {{4}}.
# No new commands are created, index is stuck with {{nextRowIdToBuild!=null}}
until cluster restart.
was (Author: ibessonov):
The problem is in {{{}IndexBuildTask{}}}. To be more precise - it's the way it
reads the {{nextRowIdToBuild}} and constructs batches. For the sake of
simplicity I will use "batchSize=2" in the following demonstration: # Imagine
we have a partition with rows {{{}[1, 2, 3]{}}}. Primary replica is not
collocated with leader. Raft group has *2* peers.
# Build index process creates command {{A: \{index [1, 2]}}}
# Primary replica successfully replies after the replication of this command,
but it's only been applied on the leader. Locally command is still not applied.
# Build index process reads the {{nextRowIdToBuild}} and receives
{{{}minRowId{}}}. Because of that it create a command {{{}B: \{index [1,
2]}{}}}, which is identical to {{{}A{}}}.
# Primary replica successfully replies, but command {{B}} is not yet applied
locally. We're exactly in-between commands {{A}} and {{{}B{}}}.
# Build index process creates new command {{C: {index [3]}}} based on the
storage status after the application of {{{}A{}}}. The number of entries here
is less than {{{}batchSize{}}}, which means that the command will be replicated
with {{{}finish=true{}}}.
# Primary replica successfully replies. It so happened that we're in the
middle of applying the command {{{}B{}}}, and current value of
{{nextRowIdToBuild}} is {{{}2{}}}.
# Build index process uses this value to create command {{{}D: {index [2,
3]}{}}}. Here we have 2 rows, which is equal to {{{}batchSize{}}}, so
{{finish=false}} for this command.
# Primary replica successfully replies. At this moment command {{D}} is not
applied, but command {{C}} is applied. Command {{C}} had {{{}finish=true{}}},
locally {{{}nextRowIdToBuild=null{}}}. Index build process stops.
# After that command {{D}} is applied. It had {{{}finish=false{}}}, which
changes {{nextRowIdToBuild}} from {{null}} to {{{}4{}}}.
# No new commands are created, index is stuck with {{nextRowIdToBuild!=null}}
until cluster restart.
> Find a reason behind build index commands reordering
> ----------------------------------------------------
>
> Key: IGNITE-27844
> URL: https://issues.apache.org/jira/browse/IGNITE-27844
> Project: Ignite
> Issue Type: Task
> Reporter: Ivan Bessonov
> Assignee: Ivan Bessonov
> Priority: Major
> Labels: ignite-3
>
> It is possible to get this:
> {code:java}
> [2026-02-13T14:44:45,710][WARN
> ][%ibit_n_3346%JRaft-FSMCaller-Disruptor_stripe_1-0][BuildIndexCommandHandler]
> Build index command reordering detected [tableId=19, partitionId=5,
> indexId=21, finish=false] {code}
> in {{ItBuildIndexTest#testBuildIndexOnStableTopology}} if transfer leaders
> from primary replicas to other nodes and increase the number of rows. This
> should never happen, we should figure out why.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
