zmuxuny opened a new issue, #4519:
URL: https://github.com/apache/rocketmq-dashboard/issues/4519

   ### Before Creating the Bug Report
   
   - [x] I searched open/closed issues and PRs for missing Broker runtime 
metrics, false alert recovery, disk/heap/send-queue availability, and native 
alert reconciliation.
   - [x] I verified the control flow on current `master` 
(`987b748e8f4f421c5cd3c4c4e51a064cc7e59f18`).
   
   ### Problem
   
   `ApacheRocketMqClusterMetricsCollector` treats a successful 
`fetchBrokerRuntimeStats` call as Broker availability, but silently **omits** 
individual metric samples when `commitLogDiskRatio`, JVM heap fields, or 
send-thread-pool queue fields are missing/invalid.
   
   `NativeAlertProcessor.processSuccessfulCollection` later reconciles missing 
active states for the entire cluster-metric scope. Its whole-scope failure 
guard only stops reconciliation for an UNAVAILABLE sample with empty labels. A 
missing broker-scoped metric emits no sample at all, so an existing active rule 
fingerprint is absent from `presentKeys` and can be advanced to RESOLVED.
   
   ### Impact
   
   A transiently absent or malformed Broker runtime field can look like a real 
recovery. For example, a firing `broker.disk.usage_ratio` alert may be marked 
RESOLVED and enqueue a recovery notification merely because 
`commitLogDiskRatio` was missing for one collection, while 
`broker.availability` still reports the Broker as healthy.
   
   The same failure mode applies to:
   - `broker.disk.usage_ratio`;
   - `broker.jvm.heap.usage_ratio`;
   - `broker.send_queue.usage_ratio`.
   
   ### Expected behavior
   
   Once Broker runtime stats are reachable, each supported metric must produce 
either an AVAILABLE value or a **broker-scoped UNAVAILABLE** sample. 
Missing/invalid individual fields must never be represented by absence.
   
   A broker-scoped UNAVAILABLE sample keeps the rule fingerprint present, skips 
numeric evaluation, and prevents missing-state reconciliation from generating a 
false recovery while leaving other Broker metrics usable.
   
   ### Verification plan
   
   Add red-first collector coverage proving a missing/invalid runtime field 
currently produces no metric sample. Add an integration-style 
NativeAlertProcessor regression: start from an active broker metric state, 
process a successful collection containing the same broker-scoped metric as 
UNAVAILABLE, and verify no RESOLVED lifecycle event/recovery notification is 
emitted.
   
   Preserve genuine Broker unavailability and all valid numeric parsing 
behavior.
   
   AI-assisted source audit; the collector and reconciliation control flow were 
verified before filing.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to