Jiandan Yang created HDFS-17805:
------------------------------------
Summary: A more detailed flushOrSync Warn Log
Key: HDFS-17805
URL: https://issues.apache.org/jira/browse/HDFS-17805
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Reporter: Jiandan Yang
When DataNode receives a request to flush or sync a block being written by a
client , the method flushOrSync (boolean isSync) will be invoked.
In this method, the duration of the whole process will be recorded, also the
flush and sync time of streams will be summed and then marked as
‘flushTotalNanos’.
If the duration of this ‘flush or sync’ process exceeds a certain threshold, a
warn log would appear and tells the value of the duration and the
‘flushTotalNanos’, enabling us to pinpoint the slower part of the execution
within the flushOrSync(boolean isSync) methods.
The current log records the time for both flush and sync operations as a single
metric called flushTotalNanos. However, if flushTotalNanos takes too long,
users will not be able to further diagnose the issue in detail based on the log.
I propose that the log separately display the durations of flush and sync
operations, instead of combining them into flushTotalNanos. This would allow
users to pinpoint issues more accurately.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
