[
https://issues.apache.org/jira/browse/HDDS-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng reassigned HDDS-17112:
---------------------------------
Assignee: Siyao Meng
> Avoid decoding BlockData on every single chunk PutBlock with incremental
> chunk list
> -----------------------------------------------------------------------------------
>
> Key: HDDS-17112
> URL: https://issues.apache.org/jira/browse/HDDS-17112
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
>
> With {{ozone.client.incremental.chunk.list=true}} (recommended for HBase),
> {{DatanodeStoreWithIncrementalChunkList#putBlockWithPartialChunks}} reads and
> decodes the block's {{BlockData}} from the block data table on every
> PutBlock. In the most common case, a PutBlock carrying a single partial chunk
> (for example each hsync of an HBase WAL), the value is only used to check
> whether the block already exists.
> In a CPU profile of the leader datanode under a WAL like workload (1 KB write
> followed by hsync, in a loop), this lookup took 3.9% of datanode CPU, and
> 2.5% of that was decoding the value. The decoding cost grows with the number
> of full chunks already recorded for the block.
> Proposed fix: use {{Table#isExist}} for the single chunk case, and only get
> the {{BlockData}} in the multiple chunk branch where it is actually used.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
