[
https://issues.apache.org/jira/browse/HDDS-16666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-16666:
----------------------------------
Labels: pull-request-available (was: )
> Flush SCM transaction in memory during apply transaction
> --------------------------------------------------------
>
> Key: HDDS-16666
> URL: https://issues.apache.org/jira/browse/HDDS-16666
> Project: Apache Ozone
> Issue Type: Improvement
> Components: SCM HA
> Reporter: Sarveksha Yeshavantha Raju
> Assignee: Sarveksha Yeshavantha Raju
> Priority: Major
> Labels: pull-request-available
>
> As observed,
> If a follower node starts after long duration, there are huge number of
> transactions accumulated at leader which needs to be synchronized to the
> follower node.
>
> The Sync of raft log to follower node is done at high speed and flushed to
> local directory. But speed of applyTransaction may not match and hence,
> * at some point of time, commitIndex is very high compared to appliedIndex
> (> 5million or more)
> * as ratis logic, it will try to apply all committed transaction to be
> applied in loop (before taking snapshot) and thus build up huge memory to be
> flushed to db.
>
> So as solution to avoid memory buildup (or memory pressure), applyTransaction
> can flush the db update to DB when pending count reaches like 10K.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
