Hi Rui I think that would be a very good optimisation. +1 It would reduce the number of updates made to delete table as well.
> The drawback of only updating retrycount at the limit is, if SCM restart at For this I would suggest we also update the retry count in intervals of 100(or some other value) along with retry limit. Thanks Lokesh > On 14-Jan-2021, at 1:10 AM, Rui Wang <amaliu...@apache.org> wrote: > > Hi Community, > > SCM maintains a DeleteBlockTransaction table [1]. For each transaction > record in this table, there is a retry count [2]. This retry count > increases every time when SCM retries the delete transaction and until it > exceeds the maximum limit, then SCM stops retrying and admin can analyze > why some blocks fail to delete. > > Because the count is written into DB every time upon retries, I want to > discuss whether it is worth an optimization that we can maintain the retry > count as an in-memory state and we only write to DB when the retry count > exceeds the limit (thus to leave for further analysis). > > The reason for this idea is in SCM HA we are replicating DB changes over > Ratis, and still persist retry count for every increase will have 3x cost > compared to now. > > The drawback of only updating retrycount at the limit is, if SCM restart at > a time, the retry count will be cleared and restart to count. > > > [1]: > https://github.com/apache/ozone/blob/master/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/metadata/SCMMetadataStore.java#L70 > [2]: > https://github.com/apache/ozone/blob/master/hadoop-hdds/interface-server/src/main/proto/ScmServerDatanodeHeartbeatProtocol.proto#L331 > > > Thanks, > Rui Wang --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org For additional commands, e-mail: dev-h...@ozone.apache.org
