bert82503 opened a new issue, #15333: URL: https://github.com/apache/dubbo/issues/15333
### Pre-check - [x] I am sure that all the content I provide is in English. ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/dubbo/issues?q=is%3Aissue) and found no similar issues. ### Apache Dubbo Component Java SDK (apache/dubbo) ### Dubbo Version - Dubbo Java 3.2.10 - JDK jdk1.8.0_333 - Spring Boot 2.3.12.RELEASE ### Steps to reproduce this issue We application use Dubbo as RPC framework. Today, when dubbo.metrics publishEvent that a lot of threads have blocked on ConcurrentHashMap.computeIfAbsent(). The key thread call stack log. ``` "DubboServerHandler-192.168.106.148:8504-thread-499" Id=846 BLOCKED on java.util.concurrent.ConcurrentHashMap$Node@1001d0e owned by "DubboServerHandler-192.168.106.148:8504-thread-292" Id=624 at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1674) - blocked on java.util.concurrent.ConcurrentHashMap$Node@1001d0e at org.apache.dubbo.metrics.listener.AbstractMetricsListener.isSupport(AbstractMetricsListener.java:33) at org.apache.dubbo.metrics.event.SimpleMetricsEventMulticaster.publishEvent(SimpleMetricsEventMulticaster.java:44) at org.apache.dubbo.metrics.event.MetricsEventBus.lambda$before$7(MetricsEventBus.java:109) at org.apache.dubbo.metrics.event.MetricsEventBus$$Lambda$2092/1902345087.run(Unknown Source) at org.apache.dubbo.metrics.event.MetricsEventBus.tryInvoke(MetricsEventBus.java:96) at org.apache.dubbo.metrics.event.MetricsEventBus.before(MetricsEventBus.java:109) at org.apache.dubbo.metrics.filter.MetricsFilter.invoke(MetricsFilter.java:80) at org.apache.dubbo.metrics.filter.MetricsProviderFilter.invoke(MetricsProviderFilter.java:37) ``` The source code by the above key thread call stack log. java.util.concurrent.ConcurrentHashMap#computeIfAbsent [lefit-class.log](https://github.com/user-attachments/files/19844822/lefit-class.log)  org.apache.dubbo.metrics.listener.AbstractMetricsListener#isSupport ```java public abstract class AbstractMetricsListener<E extends MetricsEvent> implements MetricsListener<E> { private final Map<Class<?>, Boolean> eventMatchCache = new ConcurrentHashMap<>(); /** * Whether to support the general determination of event points depends on the event type */ public boolean isSupport(MetricsEvent event) { // 事件类型匹配缓存 Boolean eventMatch = eventMatchCache.computeIfAbsent( event.getClass(), clazz -> ReflectionUtils.match(getClass(), AbstractMetricsListener.class, event)); return event.isAvailable() && eventMatch; } @Override public abstract void onEvent(E event); } ```  org.apache.dubbo.metrics.event.SimpleMetricsEventMulticaster#publishEvent ```java public class SimpleMetricsEventMulticaster implements MetricsEventMulticaster { @Override public void publishEvent(MetricsEvent event) { if (validateIfApplicationConfigExist(event)) return; for (MetricsListener listener : listeners) { // 指标监视器，是否支持这类事件 if (listener.isSupport(event)) { listener.onEvent(event); } } } } ```  org.apache.dubbo.metrics.event.MetricsEventBus#before ```java public class MetricsEventBus { /** * Applicable to the scene where execution and return are separated, * eventSaveRunner saves the event, so that the calculation rt is introverted */ public static void before(MetricsEvent event) { MetricsDispatcher dispatcher = validate(event); if (dispatcher == null) return; // 调用发布事件 tryInvoke(() -> dispatcher.publishEvent(event)); } } ```  org.apache.dubbo.metrics.filter.MetricsFilter#invoke(Invoker<?>, Invocation, boolean) ```java public class MetricsFilter implements ScopeModelAware { public Result invoke(Invoker<?> invoker, Invocation invocation, boolean isProvider) throws RpcException { if (rpcMetricsEnable) { try { // 请求相关的事件 RequestEvent requestEvent = RequestEvent.toRequestEvent( applicationModel, appName, metricsDispatcher, defaultMetricsCollector, invocation, isProvider ? PROVIDER : CONSUMER, serviceLevel); // 指标事件总线，发布事件 MetricsEventBus.before(requestEvent); invocation.put(METRIC_FILTER_EVENT, requestEvent); } catch (Throwable t) { LOGGER.warn(INTERNAL_ERROR, "", "", "Error occurred when invoke.", t); } } return invoker.invoke(invocation); } } ```  org.apache.dubbo.metrics.filter.MetricsProviderFilter#invoke ```java public class MetricsProviderFilter extends MetricsFilter implements Filter, BaseFilter.Listener { @Override public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException { return super.invoke(invoker, invocation, true); } } ```  The following content is the full `jstack` log. ``` "DubboServerHandler-192.168.106.148:8504-thread-499" Id=846 BLOCKED on java.util.concurrent.ConcurrentHashMap$Node@1001d0e owned by "DubboServerHandler-192.168.106.148:8504-thread-292" Id=624 at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1674) - blocked on java.util.concurrent.ConcurrentHashMap$Node@1001d0e at org.apache.dubbo.metrics.listener.AbstractMetricsListener.isSupport(AbstractMetricsListener.java:33) at org.apache.dubbo.metrics.event.SimpleMetricsEventMulticaster.publishEvent(SimpleMetricsEventMulticaster.java:44) at org.apache.dubbo.metrics.event.MetricsEventBus.lambda$before$7(MetricsEventBus.java:109) at org.apache.dubbo.metrics.event.MetricsEventBus$$Lambda$2092/1902345087.run(Unknown Source) at org.apache.dubbo.metrics.event.MetricsEventBus.tryInvoke(MetricsEventBus.java:96) at org.apache.dubbo.metrics.event.MetricsEventBus.before(MetricsEventBus.java:109) at org.apache.dubbo.metrics.filter.MetricsFilter.invoke(MetricsFilter.java:80) at org.apache.dubbo.metrics.filter.MetricsProviderFilter.invoke(MetricsProviderFilter.java:37) at com.leoao.starter.dubbo.generic.GenericFilterWrapper.invoke(GenericFilterWrapper.java:67) at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:349) at org.apache.dubbo.rpc.filter.ProfilerServerFilter.invoke(ProfilerServerFilter.java:66) at com.leoao.starter.dubbo.generic.GenericFilterWrapper.invoke(GenericFilterWrapper.java:67) at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:349) at org.apache.dubbo.rpc.filter.ContextFilter.invoke(ContextFilter.java:145) at com.leoao.starter.dubbo.generic.GenericFilterWrapper.invoke(GenericFilterWrapper.java:67) at com.leoao.lpaas.compatable.ContextFilterWrapper.invoke(ContextFilterWrapper.java:35) at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CopyOfFilterChainNode.invoke(FilterChainBuilder.java:349) at org.apache.dubbo.rpc.cluster.filter.FilterChainBuilder$CallbackRegistrationInvoker.invoke(FilterChainBuilder.java:197) at org.apache.dubbo.rpc.protocol.dubbo.DubboProtocol$1.reply(DubboProtocol.java:167) at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.handleRequest(HeaderExchangeHandler.java:110) at org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler.received(HeaderExchangeHandler.java:205) at org.apache.dubbo.remoting.transport.DecodeHandler.received(DecodeHandler.java:52) at org.apache.dubbo.remoting.transport.dispatcher.ChannelEventRunnable.run(ChannelEventRunnable.java:64) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.dubbo.common.threadlocal.InternalRunnable.run(InternalRunnable.java:39) at java.lang.Thread.run(Thread.java:750) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@3ab6beb1 ``` ### What you expected to happen The blocked behavior don't happen. ### Anything else _No response_ ### Are you willing to submit a pull request to fix on your own? - [x] Yes I am willing to submit a pull request on my own! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
