2020-10-23 10:25:41 UTC - Lari Hotari: The CI test flakiness is pretty bad. For example, PR #8326 failed 5 times in a row and I had to resume each time with "/pulsarbot run-failure-checks". One test that fails every often is PrometheusMetricsTest. @Devin G. Bost has reported a few issues before in this area and done an amazing job in PR <https://github.com/apache/pulsar/pull/6202> to reduce test flakiness. @Devin G. Bost are you planning to resume work on that PR? How could others help you with the effort? ---- 2020-10-23 13:09:07 UTC - Lari Hotari: @Devin G. Bost since PrometheusMetricsTest seems to be the most severe issue at the moment, would you please mind sending a PR with only <https://github.com/apache/pulsar/blob/81e23eb94216a2b8460fe3559e43f1cbe1ce2968/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java> from PR #6202? I'd assume that it would be easier to process that separately and it would hopefully fix most of the current flakiness. ---- 2020-10-23 13:26:35 UTC - Lari Hotari: it seems that this change in master branch also fixed this IntelliJ problem: <https://github.com/apache/pulsar/commit/ef614ad61a0d7253380174a3994788979c16f397> . Thank you @jia zhai ---- 2020-10-23 13:31:31 UTC - jia zhai: @Lari Hotari <https://stackoverflow.com/questions/30407065/getting-intellij-to-import-shaded-dependencies-in-a-multi-module-maven-project>
A better workaround seems to be: Right-click on shade-bug-repackaged -> pom.xml in the project view in IntelliJ, choose “Maven” -> “Ignore Projects”. Then do a “Maven” -> “Reimport” on the top-level pom.xml. ---- 2020-10-23 13:31:55 UTC - jia zhai: Do you mean this issue? the shaded package not working well in intellij ---- 2020-10-23 13:39:09 UTC - Lari Hotari: yes, it works without any tweaking in IntelliJ after the removal of shading for bouncycastle ---- 2020-10-23 13:43:30 UTC - Lari Hotari: I wonder if master branch is broken in CI now. getting errors like this. "java.lang.NoSuchMethodError: io.grpc.internal.DnsNameResolverProvider.newNameResolver(Ljava/net/URI;Lio/grpc/Attributes;)Lio/grpc/internal/DnsNameResolver;" Perhaps it's <https://github.com/apache/pulsar/pull/8351> / <https://github.com/apache/pulsar/pull/8351/commits/669b9a8127eaf937e19dc4327d2c38377638e5ed> that is causing this? This is an example log in <https://github.com/apache/pulsar/runs/1297863577?check_suite_focus=true> ---- 2020-10-23 13:44:17 UTC - Lari Hotari: ```2020-10-23T13:16:25.8338208Z 13:11:31.285 [main] ERROR org.apache.bookkeeper.common.component.AbstractLifecycleComponent - Failed to start Component: storage-service 2020-10-23T13:16:25.8339553Z 2020-10-23T13:16:25.8340974Z java.lang.NoSuchMethodError: io.grpc.internal.DnsNameResolverProvider.newNameResolver(Ljava/net/URI;Lio/grpc/Attributes;)Lio/grpc/internal/DnsNameResolver; 2020-10-23T13:16:25.8342327Z 2020-10-23T13:16:25.8345477Z at org.apache.bookkeeper.common.resolver.ServiceNameResolverProvider.newNameResolver(ServiceNameResolverProvider.java:95) ~[org.apache.bookkeeper-stream-storage-java-client-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8348285Z 2020-10-23T13:16:25.8351438Z at org.apache.bookkeeper.common.resolver.NameResolverProviderFactory.newNameResolver(NameResolverProviderFactory.java:45) ~[org.apache.bookkeeper-stream-storage-java-client-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8354229Z 2020-10-23T13:16:25.8355225Z at io.grpc.NameResolver$Factory.newNameResolver(NameResolver.java:207) ~[io.grpc-grpc-api-1.31.0.jar:1.31.0] 2020-10-23T13:16:25.8355929Z 2020-10-23T13:16:25.8356912Z at io.grpc.NameResolver$Factory.newNameResolver(NameResolver.java:235) ~[io.grpc-grpc-api-1.31.0.jar:1.31.0] 2020-10-23T13:16:25.8357630Z 2020-10-23T13:16:25.8359069Z at io.grpc.internal.ManagedChannelImpl.getNameResolver(ManagedChannelImpl.java:701) ~[io.grpc-grpc-core-1.31.0.jar:1.31.0] 2020-10-23T13:16:25.8360221Z 2020-10-23T13:16:25.8361357Z at io.grpc.internal.ManagedChannelImpl.<init>(ManagedChannelImpl.java:606) ~[io.grpc-grpc-core-1.31.0.jar:1.31.0] 2020-10-23T13:16:25.8362237Z 2020-10-23T13:16:25.8364076Z at io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:518) ~[io.grpc-grpc-core-1.31.0.jar:1.31.0] 2020-10-23T13:16:25.8534773Z 2020-10-23T13:16:25.8537550Z at org.apache.bookkeeper.clients.impl.internal.LocationClientImpl.<init>(LocationClientImpl.java:66) ~[org.apache.bookkeeper-stream-storage-java-client-base-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8539767Z 2020-10-23T13:16:25.8542914Z at org.apache.bookkeeper.clients.impl.internal.StorageServerClientManagerImpl.<init>(StorageServerClientManagerImpl.java:76) ~[org.apache.bookkeeper-stream-storage-java-client-base-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8545676Z 2020-10-23T13:16:25.8547968Z at org.apache.bookkeeper.stream.server.StorageServer.lambda$buildStorageServer$5(StorageServer.java:304) ~[org.apache.bookkeeper-stream-storage-server-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8549961Z 2020-10-23T13:16:25.8552852Z at org.apache.bookkeeper.stream.storage.StorageContainerStoreBuilder.build(StorageContainerStoreBuilder.java:162) ~[org.apache.bookkeeper-stream-storage-service-impl-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8555356Z 2020-10-23T13:16:25.8557660Z at org.apache.bookkeeper.stream.server.service.StorageService.doStart(StorageService.java:46) ~[org.apache.bookkeeper-stream-storage-server-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8559628Z 2020-10-23T13:16:25.8562170Z at org.apache.bookkeeper.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:83) ~[org.apache.bookkeeper-bookkeeper-common-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8564386Z 2020-10-23T13:16:25.8632224Z at org.apache.bookkeeper.common.component.LifecycleComponentStack.lambda$start$2(LifecycleComponentStack.java:113) ~[org.apache.bookkeeper-bookkeeper-common-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8634379Z 2020-10-23T13:16:25.8635922Z at com.google.common.collect.ImmutableList.forEach(ImmutableList.java:407) [com.google.guava-guava-25.1-jre.jar:?] 2020-10-23T13:16:25.8637045Z 2020-10-23T13:16:25.8639467Z at org.apache.bookkeeper.common.component.LifecycleComponentStack.start(LifecycleComponentStack.java:113) [org.apache.bookkeeper-bookkeeper-common-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8641557Z 2020-10-23T13:16:25.8644588Z at org.apache.bookkeeper.stream.server.StreamStorageLifecycleComponent.doStart(StreamStorageLifecycleComponent.java:51) [org.apache.bookkeeper-stream-storage-server-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8647476Z 2020-10-23T13:16:25.8650127Z at org.apache.bookkeeper.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:83) [org.apache.bookkeeper-bookkeeper-common-4.10.0.jar:4.10.0] 2020-10-23T13:16:25.8652344Z 2020-10-23T13:16:25.8654992Z at org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.runStreamStorage(LocalBookkeeperEnsemble.java:349) [org.apache.pulsar-pulsar-zookeeper-utils-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT] 2020-10-23T13:16:25.8657301Z 2020-10-23T13:16:25.8659907Z at org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.startStandalone(LocalBookkeeperEnsemble.java:431) [org.apache.pulsar-pulsar-zookeeper-utils-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT] 2020-10-23T13:16:25.8662178Z 2020-10-23T13:16:25.8663788Z at org.apache.pulsar.PulsarStandalone.start(PulsarStandalone.java:261) [org.apache.pulsar-pulsar-broker-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT] 2020-10-23T13:16:25.8665077Z 2020-10-23T13:16:25.8666915Z at org.apache.pulsar.PulsarStandaloneStarter.main(PulsarStandaloneStarter.java:119) [org.apache.pulsar-pulsar-broker-2.7.0-SNAPSHOT.jar:2.7.0-SNAPSHOT] 2020-10-23T13:16:25.8668440Z ``` ---- 2020-10-23 14:01:12 UTC - Lari Hotari: attempt to fix the problem <https://github.com/apache/pulsar/pull/8361> ---- 2020-10-23 14:31:23 UTC - Lari Hotari: There's a bug in the Github Flow, it considers a pom.xml change a documentation change. I filed a bug: <https://github.com/apache/pulsar/issues/8362> ---- 2020-10-23 14:39:20 UTC - Lari Hotari: cc @eolivelli. I noticed that you were wondering about some build failures in CI. ---- 2020-10-23 17:51:10 UTC - Lari Hotari: There seems to be more issues, also with protobuf compatibility. I'm running an experiment in <https://github.com/apache/pulsar/pull/8358|#8358> to see if reverting <https://github.com/apache/pulsar/commit/647d3c22ee219e091fc29d809e8da982a25b6541|647d3c2> fixes the issues. ---- 2020-10-23 18:09:54 UTC - Lari Hotari: PR for reverting the grpc upgrade, <https://github.com/apache/pulsar/pull/8363> /cc @Matteo Merli ----
