[
https://issues.apache.org/jira/browse/SOLR-18407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18133226#comment-18133226
]
ASF subversion and git services commented on SOLR-18407:
--------------------------------------------------------
Commit a336bf871b2c2b67abe55808966051d1c553b579 in solr's branch
refs/heads/branch_10x from Matthew Biscocho
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=a336bf871b2 ]
SOLR-18544: Source CPU utilization metric from JFR instead JMX (#5185)
Partially a regression from SOLR-18407 but also partially a JDK bug that got
exposed from the OTEL upgrade.
This was because previously the metric was sourced from JFR. Then after the
upgrade it started to source it from JMX which failed to some JDK bug. So this
change just sources it from JFR again.
> Solr 10 emits duplicate JVM metrics from both runtime-telemetry-java8 and
> runtime-telemetry-java17 scopes on Java 17
> --------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-18407
> URL: https://issues.apache.org/jira/browse/SOLR-18407
> Project: Solr
> Issue Type: Bug
> Components: metrics
> Affects Versions: 10.0
> Reporter: Jerry Chung
> Assignee: Eric Pugh
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 10.1
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> When running Solr 10 on Java 17, the {{/solr/admin/metrics?wt=prometheus}}
> endpoint emits JVM metrics (e.g. {{{}jvm_gc_duration_seconds_count{}}},
> {{{}jvm_memory_used_bytes{}}}) from two OTEL instrumentation scopes
> simultaneously:
> * {{io.opentelemetry.runtime-telemetry-java17}} (JFR-based)
> * {{io.opentelemetry.runtime-telemetry-java8}} (JMX-based)
> {{OtelRuntimeJvmMetrics}} only calls {{.enableAllFeatures()}} without
> {{{}.enableExperimentalJmxTelemetry(){}}}, so the java8/JMX scope should not
> activate. However, the bundled version of
> {{opentelemetry-runtime-telemetry-java17}} ({{{}2.22.0-alpha{}}}) implicitly
> activates JMX metrics alongside JFR, producing duplicate series with
> conflicting values for the same pool (e.g. G1 Eden Space {{used}} differs
> between scopes by an order of magnitude).
> *Impact:* Metric consumers receive duplicate series with different values for
> the same metric, making it impossible to reliably use these metrics in
> dashboards or alerts without scope-specific filtering.
> *Expected behavior:* Only {{runtime-telemetry-java17}} (JFR) metrics should
> be emitted on Java 17, or there should be a configuration to select
> *Workaround:* None available without patching Solr or upgrading the bundled
> OTEL library to 2.13.x+ where the implicit JMX activation was fixed.
> *Related:* SOLR-17806 /
> [apache/solr#3494|https://github.com/apache/solr/issues/3494],
> opentelemetry-java-instrumentation#13332,
> opentelemetry-java-instrumentation#13337
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
