zjncs opened a new pull request, #4133:
URL: https://github.com/apache/rocketmq-dashboard/pull/4133
## Motivation
`ApacheRocketMqProxyMetricsCollector.collect` returns a single unavailable
sample with the placeholder label `proxyAddr=unknown` when proxy discovery
fails for the whole instance (e.g. nameserver outage):
```java
} catch (RuntimeException error) {
return List.of(unavailable(instance, null, Map.of("proxyAddr",
"unknown"), collectedAt));
}
```
#2957 added `NativeAlertProcessor.containsWholeScopeFailure` to skip
reconciliation when a collection reports a whole-scope failure — detected as an
unavailable sample with **empty labels** (the marker
`ApacheRocketMqClusterMetricsCollector` already emits for nameserver failures).
The proxy discovery-failure sample predates that convention (#2533) and was not
aligned, so:
1. `containsWholeScopeFailure` returns false (labels are non-empty) and
`reconcileMissingActiveStates` runs;
2. the placeholder fingerprint (`proxyAddr=unknown`) never matches the
active states of the **real** proxies;
3. every firing `proxy.availability` alert for the instance is **falsely
resolved** — resolution events are emitted and notifications enqueued — while
in reality the collector could not discover a single proxy and knows nothing
about their status.
So a nameserver hiccup turns all "proxy down" alerts into "resolved"
notifications, masking real outages.
## Modification
Emit the whole-scope failure marker (empty labels) for the discovery
failure, matching `ApacheRocketMqClusterMetricsCollector`\'s nameserver failure
sample and the detection added in #2957:
```java
return List.of(unavailable(instance, null, Map.of(), collectedAt));
```
The per-proxy failure paths (probe unreachable / probe error / malformed
address) keep their real `proxyAddr` labels, so per-target unavailability still
reconciles correctly and availability rules keep firing per real proxy. After
discovery recovers, any state fired for the whole-scope marker carries the
empty fingerprint and is resolved by the next successful reconciliation.
## Verification
`mvn -f server/pom.xml test
-Dtest='ApacheRocketMqProxyMetricsCollectorTest,NativeAlertProcessorTest'`
fail-before (fix reverted, tests kept):
```
ApacheRocketMqProxyMetricsCollectorTest.recordsWholeScopeFailureSampleWhenProxyDiscoveryFailsTest
<<< FAILURE!
Expecting empty but was: {"proxyAddr"="unknown"}
Tests run: 22, Failures: 1, Errors: 0
```
pass-after:
```
Tests run: 3, Failures: 0 ... in ApacheRocketMqProxyMetricsCollectorTest
Tests run: 19, Failures: 0 ... in NativeAlertProcessorTest
Tests run: 22, Failures: 0, Errors: 0
BUILD SUCCESS
```
Also updated the existing discovery-failure test to pin the whole-scope
marker (it previously asserted the placeholder label) and added a
processor-level test showing the firing state of a real proxy survives a
discovery outage.
Follow-up to #2957; no associated issue — found by code inspection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
