lhotari commented on pull request #190: URL: https://github.com/apache/pulsar-helm-chart/pull/190#issuecomment-1017620708
based on the logs, it looks like the `pulsar-bookkeeper-verify-clusterid` container fails: https://github.com/apache/pulsar-helm-chart/blob/a919f309c6d73342196dbaf6bf146cfda8d9e8e8/charts/pulsar/templates/pulsar-cluster-initialize.yaml#L61-L73 . I wonder why `pulsar-bookkeeper-verify-clusterid`. ``` [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:22:08.341 [main-SendThread(pulsar-ci-zookeeper:2281)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server pulsar-ci-zookeeper/10.244.1.11:2281. [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:22:08.346 [main-SendThread(pulsar-ci-zookeeper:2281)] INFO org.apache.zookeeper.ClientCnxn - SASL config status: Will not attempt to authenticate using SASL (unknown error) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:22:38.987 [main-SendThread(pulsar-ci-zookeeper:2281)] WARN org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 30726ms for session id 0x0 [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:22:39.125 [main-SendThread(pulsar-ci-zookeeper:2281)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for sever pulsar-ci-zookeeper/10.244.1.11:2281, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] org.apache.zookeeper.ClientCnxn$SessionTimeoutException: Client session timed out, have not heard from server in 30726ms for session id 0x0 [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1258) [org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:22:43.496 [main] INFO org.apache.zookeeper.ClientCnxnSocketNetty - channel is told closing [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:22:43.686 [globalEventExecutor-1-1] WARN org.apache.zookeeper.ClientCnxnSocketNetty - future isn't success. [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] io.netty.util.concurrent.DefaultPromise$LeanCancellationException: null [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at io.netty.util.concurrent.DefaultPromise.cancel(...)(Unknown Source) ~[io.netty-netty-common-4.1.72.Final.jar:4.1.72.Final] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:22:43.706 [main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x0 closed [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:22:43.721 [main-EventThread] INFO org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x0 [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:22:43.707 [main] ERROR org.apache.bookkeeper.meta.zk.ZKMetadataDriverBase - Failed to create zookeeper client to pulsar-ci-zookeeper:2281 [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) ~[org.apache.zookeeper-zookeeper-3.6.3.jar:3.6.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase.waitForConnection(ZooKeeperWatcherBase.java:159) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.zookeeper.ZooKeeperClient$Builder.build(ZooKeeperClient.java:260) ~[org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.meta.zk.ZKMetadataDriverBase.initialize(ZKMetadataDriverBase.java:207) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.meta.zk.ZKMetadataBookieDriver.initialize(ZKMetadataBookieDriver.java:60) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithMetadataBookieDriver(MetadataDrivers.java:345) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithRegistrationManager(MetadataDrivers.java:372) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.tools.cli.commands.bookies.InstanceIdCommand.apply(InstanceIdCommand.java:49) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.bookie.BookieShell$WhatIsInstanceId.runCmd(BookieShell.java:1492) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.bookie.BookieShell$MyCommand.runCmd(BookieShell.java:238) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.bookie.BookieShell.run(BookieShell.java:2275) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.bookie.BookieShell.main(BookieShell.java:2366) [org.apache.bookkeeper-bookkeeper-server-4.14.3.jar:4.14.3] [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] Exception in thread "main" com.google.common.util.concurrent.UncheckedExecutionException: Failed to create zookeeper client to pulsar-ci-zookeeper:2281 [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.tools.cli.commands.bookies.InstanceIdCommand.apply(InstanceIdCommand.java:61) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.bookie.BookieShell$WhatIsInstanceId.runCmd(BookieShell.java:1492) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.bookie.BookieShell$MyCommand.runCmd(BookieShell.java:238) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.bookie.BookieShell.run(BookieShell.java:2275) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.bookie.BookieShell.main(BookieShell.java:2366) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] Caused by: org.apache.bookkeeper.meta.exceptions.MetadataException: Failed to create zookeeper client to pulsar-ci-zookeeper:2281 [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.meta.zk.ZKMetadataDriverBase.initialize(ZKMetadataDriverBase.java:227) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.meta.zk.ZKMetadataBookieDriver.initialize(ZKMetadataBookieDriver.java:60) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithMetadataBookieDriver(MetadataDrivers.java:345) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.meta.MetadataDrivers.runFunctionWithRegistrationManager(MetadataDrivers.java:372) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.tools.cli.commands.bookies.InstanceIdCommand.apply(InstanceIdCommand.java:49) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] ... 4 more [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase.waitForConnection(ZooKeeperWatcherBase.java:159) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.zookeeper.ZooKeeperClient$Builder.build(ZooKeeperClient.java:260) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] at org.apache.bookkeeper.meta.zk.ZKMetadataDriverBase.initialize(ZKMetadataDriverBase.java:207) [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] ... 8 more [pod/pulsar-ci-pulsar-init-pzc88/pulsar-bookkeeper-verify-clusterid] 14:23:19.303 [epollEventLoopGroup-2-1] INFO org.apache.zookeeper.ClientCnxnSocketNetty - SSL handler added for channel: [id: 0xaa063649] Error from server (BadRequest): container "pulsar-ci-pulsar-init" in pod "pulsar-ci-pulsar-init-pzc88" is waiting to start: PodInitializing ``` I guess that the `bookie-init` has to finish first. Checking the logs for that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
