Siyao Meng created HDDS-17112:
---------------------------------
Summary: Avoid decoding BlockData on every single chunk PutBlock
with incremental chunk list
Key: HDDS-17112
URL: https://issues.apache.org/jira/browse/HDDS-17112
Project: Apache Ozone
Issue Type: Improvement
Reporter: Siyao Meng
With {{ozone.client.incremental.chunk.list=true}} (recommended for HBase),
{{DatanodeStoreWithIncrementalChunkList#putBlockWithPartialChunks}} reads and
decodes the block's {{BlockData}} from the block data table on every PutBlock.
In the most common case, a PutBlock carrying a single partial chunk (for
example each hsync of an HBase WAL), the value is only used to check whether
the block already exists.
In a CPU profile of the leader datanode under a WAL like workload (1 KB write
followed by hsync, in a loop), this lookup took 3.9% of datanode CPU, and 2.5%
of that was decoding the value. The decoding cost grows with the number of full
chunks already recorded for the block.
Proposed fix: use {{Table#isExist}} for the single chunk case, and only get the
{{BlockData}} in the multiple chunk branch where it is actually used.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
