lvs071103 opened a new issue #307: URL: https://github.com/apache/pulsar-manager/issues/307
env: ``` [root@pulsar-shanghai-node01 pulsar_sh]# bin/pulsar-admin clusters list "shanghai" "beijing" [root@pulsar-shanghai-node01 pulsar_sh]# bin/pulsar-admin clusters get shanghai { "serviceUrl" : "http://10.1.104.136:8080", "serviceUrlTls" : "https://10.1.104.136:8443", "brokerServiceUrl" : "pulsar://10.1.104.136:6650", "brokerServiceUrlTls" : "pulsar+ssl://10.1.104.136:6651" } [root@pulsar-shanghai-node01 pulsar_sh]# bin/pulsar-admin clusters get beijing { "serviceUrl" : "http://10.1.104.220:8080", "serviceUrlTls" : "https://10.1.104.220:8443", "brokerServiceUrl" : "pulsar://10.1.104.220:6650", "brokerServiceUrlTls" : "pulsar+ssl://10.1.104.220:6651" } ```   shanghai cluster proxy config: ``` zookeeperServers=10.1.104.130:2181,10.1.104.131:2181,10.1.104.132:2181 configurationStoreServers=10.1.104.130:2184,10.1.104.131:2184,10.1.104.132:2184 brokerServiceURL=pulsar://10.1.104.130:6650,10.1.104.131:6650,10.1.104.132:6650 brokerServiceURLTLS=pulsar+ssl://10.1.104.130:6651,10.1.104.131:6651,10.1.104.132:6651 brokerWebServiceURL=http://10.1.104.130:8080,10.1.104.131:8080,10.1.104.132:8080 brokerWebServiceURLTLS=https://10.1.104.130:8443,10.1.104.131:8443,10.1.104.132:8443 servicePort=6650 servicePortTls=6651 webServicePort=8080 webServicePortTls=8443 superUserRoles=admin authorizationEnabled=true authorizationProvider=org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider authenticationEnabled=true authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken brokerClientAuthenticationParameters=token:eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImV4cCI6MTY4NzA4NDI5MH0.K5sAvvUjd9J8RxHaBVlDCPnoPW8T6FDa4x9CovoH6v4 tokenSecretKey=file:///opt/Apps/pulsar_sh/my-auth/admin.key ``` beijing cluster proxy config: ``` zookeeperServers=10.1.104.133:2181,10.1.104.134:2181,10.1.104.135:2181 configurationStoreServers=10.1.104.133:2184,10.1.104.134:2184,10.1.104.135:2184 brokerServiceURL=pulsar://10.1.104.133:6650,10.1.104.134:6650,10.1.104.135:6650 brokerServiceURLTLS=pulsar+ssl://10.1.104.133:6651,10.1.104.134:6651,10.1.104.135:6651 brokerWebServiceURL=http://10.1.104.133:8080,10.1.104.134:8080,10.1.104.135:8080 brokerWebServiceURLTLS=https://10.1.104.133:8443,10.1.104.134:8443,10.1.104.135:8443 servicePort=6650 servicePortTls=6651 webServicePort=8080 webServicePortTls=8443 superUserRoles=admin authorizationEnabled=true authorizationProvider=org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider authenticationEnabled=true authenticationProviders=org.apache.pulsar.broker.authentication.AuthenticationProviderToken brokerClientAuthenticationPlugin=org.apache.pulsar.client.impl.auth.AuthenticationToken brokerClientAuthenticationParameters=token:eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImV4cCI6MTY4NzA4NDI5MH0.K5sAvvUjd9J8RxHaBVlDCPnoPW8T6FDa4x9CovoH6v4 tokenSecretKey=file:///opt/Apps/pulsar_bj/my-auth/admin.key ``` pulsar-manager.log: ``` 2020-06-22 11:05:51.242 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec : Connection can be kept alive indefinitely 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection [id: 486][route: {}->http://10.1.104.130:8080] can be kept alive indefinitely 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] h.i.c.DefaultManagedHttpClientConnection : http-outgoing-486: set socket timeout to 0 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 486][route: {}->http://10.1.104.130:8080][total kept alive: 2; route allocated: 1 of 10; total allocated: 2 of 100] 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] o.a.h.i.client.DefaultRedirectStrategy : Redirect requested to location 'http://10.1.104.220:8080/admin/v2/brokers/beijing' 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.RedirectExec : Resetting target auth state 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.RedirectExec : Redirecting to 'http://10.1.104.220:8080/admin/v2/brokers/beijing' via {}->http://10.1.104.220:8080 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] o.a.h.client.protocol.RequestAddCookies : CookieSpec selected: default 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] o.a.h.client.protocol.RequestAuthCache : Auth cache not set in the context 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection request: [route: {}->http://10.1.104.220:8080][total kept alive: 2; route allocated: 0 of 10; total allocated: 2 of 100] 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection leased: [id: 487][route: {}->http://10.1.104.220:8080][total kept alive: 2; route allocated: 1 of 10; total allocated: 3 of 100] 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec : Opening connection {}->http://10.1.104.220:8080 2020-06-22 11:05:51.243 DEBUG 42 --- [pool-2-thread-1] .i.c.DefaultHttpClientConnectionOperator : Connecting to /10.1.104.220:8080 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] .i.c.DefaultHttpClientConnectionOperator : Connection established 172.17.0.2:56518<->10.1.104.220:8080 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] h.i.c.DefaultManagedHttpClientConnection : http-outgoing-487: set socket timeout to 28000 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec : Executing request GET /admin/v2/brokers/beijing HTTP/1.1 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec : Proxy auth state: UNCHALLENGED 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 >> GET /admin/v2/brokers/beijing HTTP/1.1 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 >> Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImV4cCI6MTY4NzA4NDI5MH0.K5sAvvUjd9J8RxHaBVlDCPnoPW8T6FDa4x9CovoH6v4 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 >> Host: 10.1.104.220:8080 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 >> Connection: Keep-Alive 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 >> User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_212) 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 >> Accept-Encoding: gzip,deflate 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 >> "GET /admin/v2/brokers/beijing HTTP/1.1[\r][\n]" 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 >> "Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImV4cCI6MTY4NzA4NDI5MH0.K5sAvvUjd9J8RxHaBVlDCPnoPW8T6FDa4x9CovoH6v4[\r][\n]" 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 >> "Host: 10.1.104.220:8080[\r][\n]" 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 >> "Connection: Keep-Alive[\r][\n]" 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 >> "User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_212)[\r][\n]" 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 >> "Accept-Encoding: gzip,deflate[\r][\n]" 2020-06-22 11:05:51.244 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 >> "[\r][\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "HTTP/1.1 401 Authentication required[\r][\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "Date: Mon, 22 Jun 2020 11:05:51 GMT[\r][\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "Content-Type: text/html;charset=iso-8859-1[\r][\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "Content-Length: 369[\r][\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "Server: Jetty(9.4.20.v20190813)[\r][\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "[\r][\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "<html>[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "<head>[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "<title>Error 401 Authentication required</title>[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "</head>[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "<body><h2>HTTP ERROR 401</h2>[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "<p>Problem accessing /admin/v2/brokers/beijing. Reason:[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "<pre> Authentication required</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.20.v20190813</a><hr/>[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "</body>[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.wire : http-outgoing-487 << "</html>[\n]" 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 << HTTP/1.1 401 Authentication required 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 << Date: Mon, 22 Jun 2020 11:05:51 GMT 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 << Cache-Control: must-revalidate,no-cache,no-store 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 << Content-Type: text/html;charset=iso-8859-1 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 << Content-Length: 369 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] org.apache.http.headers : http-outgoing-487 << Server: Jetty(9.4.20.v20190813) 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec : Connection can be kept alive indefinitely 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.auth.HttpAuthenticator : Authentication required 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.auth.HttpAuthenticator : 10.1.104.220:8080 requested authentication 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.auth.HttpAuthenticator : Response contains no authentication challenges 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec : Cancelling request execution 2020-06-22 11:05:51.247 DEBUG 42 --- [pool-2-thread-1] h.i.c.DefaultManagedHttpClientConnection : http-outgoing-487: Shutdown connection 2020-06-22 11:05:51.248 DEBUG 42 --- [pool-2-thread-1] o.a.http.impl.execchain.MainClientExec : Connection discarded 2020-06-22 11:05:51.248 DEBUG 42 --- [pool-2-thread-1] h.i.c.PoolingHttpClientConnectionManager : Connection released: [id: 487][route: {}->http://10.1.104.220:8080][total kept alive: 2; route allocated: 0 of 10; total allocated: 2 of 100] 2020-06-22 11:05:51.249 ERROR 42 --- [pool-2-thread-1] o.s.s.s.TaskUtils$LoggingErrorHandler : Unexpected error occurred in scheduled task. java.lang.NullPointerException: null at org.apache.pulsar.manager.service.impl.BrokersServiceImpl.getBrokersList(BrokersServiceImpl.java:54) ~[pulsar-manager.jar:na] at org.apache.pulsar.manager.service.impl.ClustersServiceImpl.getClustersList(ClustersServiceImpl.java:70) ~[pulsar-manager.jar:na] at org.apache.pulsar.manager.service.impl.BrokerStatsServiceImpl.scheduleCollectStats(BrokerStatsServiceImpl.java:129) ~[pulsar-manager.jar:na] at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212] at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) ~[spring-context-5.0.6.RELEASE.jar:5.0.6.RELEASE] at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.0.6.RELEASE.jar:5.0.6.RELEASE] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_212] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_212] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_212] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_212] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212] ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org