Wei-Chiu Chuang created HDDS-17489:
--------------------------------------
Summary: Add Grafana dashboard coverage for Erasure Coding metrics
Key: HDDS-17489
URL: https://issues.apache.org/jira/browse/HDDS-17489
Project: Apache Ozone
Issue Type: Improvement
Components: Ozone Dashboards
Reporter: Wei-Chiu Chuang
HDDS-14807 documents EC metrics across datanodes, clients, and SCM. Add or
extend Grafana dashboards to visualize these metrics for troubleshooting EC
reconstruction, replication, failures, and safemode progress.
Reuse the existing Datanode Decommission and Maintenance, SCM overview, and SCM
Safemode dashboards where appropriate. HDDS-15352 already provides part of the
coverage requested by HDDS-11341.
h3. Metrics to cover
* Datanode / ECReconstructionMetrics: {{blockGroupReconstructionTotal}},
{{blockGroupReconstructionFailsTotal}}, {{reconstructionTotal}},
{{reconstructionFailsTotal}}.
* Client / XceiverClientMetrics: {{ecReconstructionTotal}},
{{ecReconstructionFailsTotal}}.
* SCM / ReplicationManagerMetrics: {{InflightEcReplication}},
{{InflightEcDeletion}}, {{ecReplicationCmdsSentTotal}},
{{ecReconstructionCmdsSentTotal}}, {{ecReplicasCreatedTotal}},
{{ecReplicasDeletedTotal}}, {{ecReplicaCreateTimeoutTotal}},
{{ecReplicaDeleteTimeoutTotal}}, {{ecPartialReconstructionSkippedTotal}},
{{ecPartialReconstructionCriticalTotal}},
{{ecPartialReconstructionNoneOverloadedTotal}},
{{ecPartialReplicationForOutOfServiceReplicasTotal}}.
* SCM / SafeModeMetrics: {{numContainerWithECDataReplicaReportedThreshold}},
{{currentContainersWithECDataReplicaReportedCount}}.
h3. Known coverage gaps
A review of the existing dashboard JSON files found the four datanode counters,
two client counters, and these four SCM counters missing:
{{ecReplicasDeletedTotal}}, {{ecReplicaCreateTimeoutTotal}},
{{ecReplicaDeleteTimeoutTotal}},
{{ecPartialReconstructionNoneOverloadedTotal}}. The other ten metrics already
have panels across the existing dashboards. Verify coverage against the
implementation baseline before making changes.
h3. Acceptance criteria
* Provide dashboard coverage for all metrics listed in HDDS-14807, adding the
missing panels and reusing existing coverage.
* Use the actual exported Prometheus metric names, suitable rates for counters,
gauges for inflight work, and target-versus-actual values for safemode.
* Provide clear panel descriptions, units, and instance selection where
applicable. For client metrics, document the required client metrics
export/scrape setup.
* Validate dashboard import and queries against scraped metrics, and document
where operators can find the EC panels.
This issue tracks dashboard coverage of the existing metrics in HDDS-14807.
Additional EC reconstruction bytes, in-progress tasks, and latency metrics are
tracked separately by HDDS-17167.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
