[
https://issues.apache.org/jira/browse/HDDS-13613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aswin Shakil updated HDDS-13613:
--------------------------------
Description:
_XceiverServerRatis#calculatePipelineBytesWritten_ is called every time we send
a pipeline report, This method goes through all the containers to filter out
those container belonging to a pipelineID. In a dense DN, this would cause a
bottleneck when we send pipeline report. It would also cause a rippling effect
as the pipeline creation will become slow and the pipeline scrubber in SCM will
close this pipeline if the DN doesn't create the pipeline within the time. The
DN will consisently be in a loop to create and close pipeline which would make
the DN unresponsive.
This _calculatePipelineBytesWritten_ is only used by a metric in SCM and thus
it should be removed. These aggregations need to be done by Prometheus.
was:
_XceiverServerRatis#calculatePipelineBytesWritten_ is called everytime when we
send a pipeline report, This method goes through all the containers to filter
out those container belonging to a pipelineID. In a dense DN, this would cause
a bottleneck when we send pipeline report. It would also cause a rippling
effect as the pipeline creation will become slow and the piple scrubber in SCM
will close this pipeline if the DN doesn't create the pipeline within time. The
DN will consisently be in a loop to create and close pipeline which would make
the DN unresponsive.
This _calculatePipelineBytesWritten_ is only used by a metric in SCM an thus it
should be removed, These aggregations needs to be done by Prometheus.
> Remove calculatePipelineBytesWritten from DN code path
> ------------------------------------------------------
>
> Key: HDDS-13613
> URL: https://issues.apache.org/jira/browse/HDDS-13613
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Aswin Shakil
> Assignee: Aswin Shakil
> Priority: Critical
>
> _XceiverServerRatis#calculatePipelineBytesWritten_ is called every time we
> send a pipeline report, This method goes through all the containers to filter
> out those container belonging to a pipelineID. In a dense DN, this would
> cause a bottleneck when we send pipeline report. It would also cause a
> rippling effect as the pipeline creation will become slow and the pipeline
> scrubber in SCM will close this pipeline if the DN doesn't create the
> pipeline within the time. The DN will consisently be in a loop to create and
> close pipeline which would make the DN unresponsive.
> This _calculatePipelineBytesWritten_ is only used by a metric in SCM and thus
> it should be removed. These aggregations need to be done by Prometheus.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
