Hi, You can refer to the documentation for logging levels.
https://zookeeper.apache.org/doc/r3.4.11/zookeeperAdmin.html#Debug+Log+Cleanup+%28log4j%29 By clearing current logs will help you to read and understand new logs. Old logs wont get mixed up with the new one and you can read and debug events from the beginning of startup. Regards, Sunil. On Sun, 22 Nov 2020 at 5:52 PM, prat 007 <pratkul...@gmail.com> wrote: > Hi Sunil, > > Thanks for your reply. > You mean to say to empty kafka.log file? (I don't see zookeeper.log file > in server), but not sure how clearing the log file itself will help. > > I will search for debug switch for zookeeper start command but I was > talking about in which zookeeper config file I can change the value of a > parameter from INFO to DEBUG or TRACE level to enable the whole log file to > show the logs in more depth. > > thanks, > > On 2020/11/22 12:00:26, sunil chaudhari <sunilmchaudhar...@gmail.com> > wrote: > > Hi I am talking about the Error : > > > > 18:16:08,027] ERROR Invalid configuration, only one server specified > > > > (ignoring) (org.apache.zookeeper.server.quorum.QuorumPeerConfig) > > > > > > This seems to be the problem of whole cluster. > > > > Clear all logs means, clear existing server logs. Not data. > > > > You can see help of the zookeeper start command for debug switch. > > > > Regards, > > Sunil. > > > > > > On Sun, 22 Nov 2020 at 5:00 PM, prat 007 <pratkul...@gmail.com> wrote: > > > > > Hi Sunil, > > > > > > Thanks for reply. Q. Are you pointing to below line from point 6 to be > > > similar with url you have given (in Troubleshooting, under section: > Server > > > not coming up because of file corruption)? > > > > > > ov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: > java.io.EOFException > > > > > > Q. `Clear all logs` - you mean to say delete all files from > > > `datadir/version-2 and datalogdir/version-2/`? > > > If yes, is it safe to delete them. Do we require them later ? > > > > > > the url says, > > > `make sure all the other servers in your ensemble are up and working. > Use > > > "stat" command on the command port to see if they are in good health`. > > > > > > Q. Is below command correct to verify good health ? > > > # echo stat | nc localhost 2181 > > > > > > Q. Can you please let me know where can I set zookeeper logs for > verbose > > > mode for debugging the issue? > > > > > > Thanks, > > > > > > On 2020/11/22 01:26:52, sunil chaudhari <sunilmchaudhar...@gmail.com> > > > wrote: > > > > Hi, > > > > Please check if it helps: > > > > > > > > http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html#sc_supportedPlatforms > > > > > > > > You have similar symptoms in point 6 in your original email. > > > > > > > > Try running it manually with below steps: > > > > Clear all logs > > > > Start zookeeper manually, watch logs for any error. Rectify it > referring > > > > above link. > > > > Start kafka manually. Watch info/error logs. > > > > Then try to list topics. > > > > > > > > Regards, > > > > Sunil. > > > > > > > > > > > > On Sat, 21 Nov 2020 at 10:16 PM, prat 007 <pratkul...@gmail.com> > wrote: > > > > > > > > > Hi Team, > > > > > > > > > > I am running ansible to install elk on one server. (on Dev env, > > > standalone > > > > > setup) but my playbook is getting failed at kafka role for task > > > listing of > > > > > Kafka topics. Can you please help. > > > > > below are the logs. > > > > > > > > > > 1. > > > > > playbook getting failed with error: - > > > > > ``` > > > > > TASK [kafka : Kafka Topics | List topics] > > > > > > > > > ****************************************************************************************************************************** > > > > > fatal: [kafka_1]: FAILED! => { > > > > > "changed": false, > > > > > "cmd": [ > > > > > "/opt/kafka/kafka_2.12-2.2.2/bin/kafka-topics.sh", > > > > > "--zookeeper", > > > > > "<FQDN>:5181", > > > > > "--list" > > > > > ], > > > > > "delta": "0:00:34.084478", > > > > > "end": "2020-11-19 22:33:42.624353", > > > > > "rc": 1, > > > > > "start": "2020-11-19 22:33:08.539875" > > > > > } > > > > > > > > > > STDERR: > > > > > > > > > > Exception in thread "main" > > > > > kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting > for > > > > > connection while in state: CONNECTING > > > > > at > > > > > > > > > kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:242) > > > > > at > > > > > > scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) > > > > > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:251) > > > > > at > > > > > > > > > kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:238) > > > > > at > > > kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:96) > > > > > at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1825) > > > > > ``` > > > > > > > > > > 2. Getting same error when run command manually which playbook is > > > trying > > > > > to run. > > > > > > > > > > ``` > > > > > /opt/kafka/kafka_2.12-2.2.2/bin/kafka-topics.sh --zookeeper <ip of > > > > > localhost>:5181 --list > > > > > > > > > > Exception in thread "main" > > > > > kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting > for > > > > > connection while in state: CONNECTING > > > > > at > > > > > > > > > kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:242) > > > > > at > > > > > > scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) > > > > > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:251) > > > > > at > > > > > > > > > kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:238) > > > > > at > > > kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:96) > > > > > at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1825) > > > > > at > > > > > > > > > kafka.admin.TopicCommand$ZookeeperTopicService$.apply(TopicCommand.scala:262) > > > > > at kafka.admin.TopicCommand$.main(TopicCommand.scala:53) > > > > > at kafka.admin.TopicCommand.main(TopicCommand.scala) > > > > > ``` > > > > > > > > > > 3. both Kafka and zookeeper are failed and not getting started. > > > > > > > > > > ``` > > > > > systemctl status kafka.service -l > > > > > ● kafka.service > > > > > Loaded: loaded (/etc/systemd/system/kafka.service; enabled; > vendor > > > > > preset: disabled) > > > > > Active: failed (Result: exit-code) since Thu 2020-11-19 22:33:17 > > > IST; > > > > > 5min ago > > > > > Process: 5448 > > > > > ExecStop=/opt/kafka/kafka_2.12-2.2.2/bin/kafka-server-stop.sh > > > (code=exited, > > > > > status=1/FAILURE) > > > > > Process: 4742 ExecStart=/bin/sh -c > > > > > /opt/kafka/kafka_2.12-2.2.2/bin/kafka-server-start.sh > > > > > /opt/kafka/kafka_2.12-2.2.2/config/server.properties > > > > > > /opt/kafka/kafka_2.12-2.2.2/kafka.log 2>&1 (code=exited, > > > status=1/FAILURE) > > > > > Main PID: 4742 (code=exited, status=1/FAILURE) > > > > > > > > > > Nov 19 22:33:08 <HOSTNAME> systemd[1]: Started kafka.service. > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service: main process > > > exited, > > > > > code=exited, status=1/FAILURE > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service: control > process > > > > > exited, code=exited status=1 > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: Unit kafka.service entered > > > failed > > > > > state. > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service failed. > > > > > > > > > > > > > > > systemctl status zookeeper.service -l > > > > > ● zookeeper.service > > > > > Loaded: loaded (/etc/systemd/system/zookeeper.service; enabled; > > > vendor > > > > > preset: disabled) > > > > > Active: failed (Result: exit-code) since Thu 2020-11-19 22:33:09 > > > IST; > > > > > 5min ago > > > > > Process: 5281 > > > > > ExecStop=/opt/kafka/kafka_2.12-2.2.2/bin/zookeeper-server-stop.sh > > > > > (code=exited, status=1/FAILURE) > > > > > Process: 4448 > > > > > ExecStart=/opt/kafka/kafka_2.12-2.2.2/bin/zookeeper-server-start.sh > > > > > /opt/kafka/kafka_2.12-2.2.2/config/zookeeper.properties > (code=exited, > > > > > status=1/FAILURE) > > > > > Main PID: 4448 (code=exited, status=1/FAILURE) > > > > > > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118) > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:122) > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:89) > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:55) > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:119) > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81) > > > > > Nov 19 22:33:09 systemd[1]: zookeeper.service: main process exited, > > > > > code=exited, status=1/FAILURE > > > > > Nov 19 22:33:09 systemd[1]: zookeeper.service: control process > exited, > > > > > code=exited status=1 > > > > > Nov 19 22:33:09 systemd[1]: Unit zookeeper.service entered failed > > > state. > > > > > Nov 19 22:33:09 systemd[1]: zookeeper.service failed. > > > > > ``` > > > > > > > > > > 4. therefore unable to do telnet or netstat > > > > > > > > > > telnet <ip og localhost> 5181 > > > > > Trying 10.. > > > > > telnet: connect to address <ip of localhost>: Connection refused > > > > > > > > > > netstat -tnlp |grep 5181 > > > > > No output > > > > > > > > > > No Iptables or SELinux is running. > > > > > > > > > > 5. output of /opt/kafka/kafka_2.12-2.2.2/kafka.log > > > > > > > > > > [2020-11-19 22:33:09,853] INFO Registered > > > kafka:type=kafka.Log4jController > > > > > MBean (kafka.utils.Log4jControllerRegistration$) > > > > > [2020-11-19 22:33:10,584] INFO starting (kafka.server.KafkaServer) > > > > > [2020-11-19 22:33:10,585] INFO Connecting to zookeeper on <ip of > > > > > localhost>:5181 (kafka.server.KafkaServer) > > > > > [2020-11-19 22:33:10,640] INFO [ZooKeeperClient] Initializing a new > > > > > session to <ip of localhost>:5181. > (kafka.zookeeper.ZooKeeperClient) > > > > > [2020-11-19 22:33:10,646] INFO Client > > > > > > > > > environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, > > > > > built on 06/29/2018 00:39 GMT (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,646] INFO Client environment:host.name=<FQDN> > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,646] INFO Client > > > environment:java.version=1.8.0_221 > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,646] INFO Client > environment:java.vendor=Oracle > > > > > Corporation (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,646] INFO Client > > > > > environment:java.home=/opt/jre1.8.0_221 > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,646] INFO Client > > > > > > > > > environment:java.class.path=/opt/kafka/kafka_2.12-2.2.2/bin/../libs/activation-1.1.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/aopalliance-repackaged-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-api-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-basic-auth-extension-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-file-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-json-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-runtime-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-transforms-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/guava-20.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/hk2-api-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/hk2-locator-2.5.0-b42.jar:/opt/kafka/ka > > > > > > > > > > > > > > fka_2.12-2.2.2/bin/../libs/hk2-utils-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-annotations-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-core-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-databind-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-datatype-jdk8-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-jaxrs-base-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-jaxrs-json-provider-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-module-jaxb-annotations-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javassist-3.22.0-CR2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.annotation-api-1.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.inject-1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.inject-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bi > > > n/.. > > > > > > > > > > > > > > /libs/javax.servlet-api-3.1.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/ > > > > > javax.ws > > > > > > > > > .rs-api-2.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-client-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-common-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-container-servlet-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-container-servlet-core-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-hk2-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-media-jaxb-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-server-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-client-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-continuation-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-http-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-io-9.4.14.v20181114.jar:/opt/kafk > > > > > > > > > > > > > > a/kafka_2.12-2.2.2/bin/../libs/jetty-security-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-server-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-servlet-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-servlets-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-util-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka_2.12-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka_2.12-2.2.2-sources.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-clients-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-log4j-appender-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-streams-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-streams-examples-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-streams-scala_2.12-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-streams-test-utils-2.2.2.jar:/opt/kafka/kafk > > > a_2. > > > > > > > > > > > > > > 12-2.2.2/bin/../libs/kafka-tools-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/log4j-1.2.17.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/lz4-java-1.5.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/maven-artifact-3.6.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/plexus-utils-3.1.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/reflections-0.9.11.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/rocksdbjni-5.15.10.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/scala-library-2.12.8.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/scala-logging_2.12-3.9.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/scala-reflect-2.12.8.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/slf4j-api-1.7.25.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/slf4j-log4j12-1.7.25.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/snappy-java-1.1.7.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/validation-api-1 > > > .1.0 > > > > > > > > > .Final.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zkclient-0.11.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zookeeper-3.4.13.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zstd-jni-1.3.8-1.jar > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,646] INFO Client > > > > > > > > > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,646] INFO Client > environment:java.io.tmpdir=/tmp > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,647] INFO Client > environment:java.compiler=<NA> > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,647] INFO Client environment:os.name=Linux > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,647] INFO Client environment:os.arch=amd64 > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,647] INFO Client > > > > > environment:os.version=3.10.0-1062.el7.x86_64 > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,647] INFO Client environment:user.name=kafka > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,647] INFO Client > environment:user.home=/home/kafka > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,647] INFO Client environment:user.dir=/ > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,648] INFO Initiating client connection, > > > > > connectString=<ip of localhost>:5181 sessionTimeout=6000 > > > > > > > > > watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@3c419631 > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:10,667] INFO Opening socket connection to server > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate > using > > > SASL > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:10,673] INFO Socket error occurred: <FQDN>/<ip of > > > > > localhost>:5181: Connection refused > (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:10,676] INFO [ZooKeeperClient] Waiting until > > > connected. > > > > > (kafka.zookeeper.ZooKeeperClient) > > > > > [2020-11-19 22:33:11,796] INFO Opening socket connection to server > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate > using > > > SASL > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:11,797] INFO Socket error occurred: <FQDN>/<ip of > > > > > localhost>:5181: Connection refused > (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:12,897] INFO Opening socket connection to server > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate > using > > > SASL > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:12,898] INFO Socket error occurred: <FQDN>/<ip of > > > > > localhost>:5181: Connection refused > (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:13,999] INFO Opening socket connection to server > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate > using > > > SASL > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:13,999] INFO Socket error occurred: <FQDN>/<ip of > > > > > localhost>:5181: Connection refused > (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:15,101] INFO Opening socket connection to server > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate > using > > > SASL > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:15,101] INFO Socket error occurred: <FQDN>/<ip of > > > > > localhost>:5181: Connection refused > (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:16,202] INFO Opening socket connection to server > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate > using > > > SASL > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:16,202] INFO Socket error occurred: <FQDN>/<ip of > > > > > localhost>:5181: Connection refused > (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:16,678] INFO [ZooKeeperClient] Closing. > > > > > (kafka.zookeeper.ZooKeeperClient) > > > > > [2020-11-19 22:33:17,303] INFO Opening socket connection to server > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate > using > > > SASL > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:17,406] INFO Session: 0x0 closed > > > > > (org.apache.zookeeper.ZooKeeper) > > > > > [2020-11-19 22:33:17,407] INFO EventThread shut down for session: > 0x0 > > > > > (org.apache.zookeeper.ClientCnxn) > > > > > [2020-11-19 22:33:17,409] INFO [ZooKeeperClient] Closed. > > > > > (kafka.zookeeper.ZooKeeperClient) > > > > > [2020-11-19 22:33:17,412] ERROR Fatal error during KafkaServer > startup. > > > > > Prepare to shutdown (kafka.server.KafkaServer) > > > > > kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting > for > > > > > connection while in state: CONNECTING > > > > > at > > > > > > > > > kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:242) > > > > > at > > > > > > scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) > > > > > at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:251) > > > > > at > > > > > > > > > kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:238) > > > > > at > > > kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:96) > > > > > at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1825) > > > > > at > > > kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:361) > > > > > at > kafka.server.KafkaServer.initZkClient(KafkaServer.scala:385) > > > > > at kafka.server.KafkaServer.startup(KafkaServer.scala:205) > > > > > at > > > > > > > > > kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38) > > > > > at kafka.Kafka$.main(Kafka.scala:75) > > > > > at kafka.Kafka.main(Kafka.scala) > > > > > [2020-11-19 22:33:17,415] INFO shutting down > (kafka.server.KafkaServer) > > > > > [2020-11-19 22:33:17,422] INFO shut down completed > > > > > (kafka.server.KafkaServer) > > > > > [2020-11-19 22:33:17,422] ERROR Exiting Kafka. > > > > > (kafka.server.KafkaServerStartable) > > > > > [2020-11-19 22:33:17,426] INFO shutting down > (kafka.server.KafkaServer) > > > > > > > > > > 6. /var/log/messages > > > > > > > > > > Nov 19 18:16:07 <HOSTNAME> ansible-command: Invoked with > creates=None > > > > > executable=None _uses_shell=False strip_empty_ends=True > > > > > _raw_params=/opt/kafka/kafka_ > > > > > 2.12-2.2.2/bin/kafka-topics.sh --zookeeper <FQDN>:5181 --list > > > removes=None > > > > > argv=None warn=True chdir=None stdin_add_newline=True stdin=None > > > > > Nov 19 18:16:07 <HOSTNAME> ansible-command: Invoked with > creates=None > > > > > executable=None _uses_shell=False strip_empty_ends=True > > > > > _raw_params=/opt/kafka/kafka_ > > > > > 2.12-2.2.2/bin/kafka-topics.sh --zookeeper <FQDN>:5181 --list > > > removes=None > > > > > argv=None warn=True chdir=None stdin_add_newline=True stdin=None > > > > > Nov 19 18:16:07 <HOSTNAME> ansible-command: Invoked with > creates=None > > > > > executable=None _uses_shell=False strip_empty_ends=True > > > > > _raw_params=/opt/kafka/kafka_ > > > > > 2.12-2.2.2/bin/kafka-topics.sh --zookeeper <FQDN>:5181 --list > > > removes=None > > > > > argv=None warn=True chdir=None stdin_add_newline=True stdin=None > > > > > Nov 19 18:16:07 <HOSTNAME> heartbeat: > > > > > > > > > 2020-11-19T18:16:07.941+0530#011INFO#011kafka/log.go:53#011client/metadata > > > > > fetching metadata for [heartbeat] from bro > > > > > ker <ip of localhost>:3997 > > > > > <ip of localhost>:3997: connect: connection refused > > > > > 19T18:16:07.942+0530#011INFO#011kafka/log.go:53#011client/metadata > got > > > > > error from broker -1 while fetching me > > > > > tadata: dial tcp <ip of localhost>:3997: connect: connection > refused > > > > > Nov 19 18:16:07 <HOSTNAME> heartbeat: > > > > > 2020-11-19T18:16:07.942+0530#011INFO#011kafka/log.go:53#011kafka > > > message: > > > > > client/metadata no available broker to send > > > > > metadata request to > > > > > Nov 19 18:16:07 <HOSTNAME> heartbeat: > > > > > > > > > 2020-11-19T18:16:07.942+0530#011INFO#011[publisher]#011pipeline/retry.go:155#011Drop > > > > > batch > > > > > Nov 19 18:16:07 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:07,962] INFO Reading configuration from: > > > > > /opt/kafka/kafka_2.12-2.2.2/config/zookeep > > > > > er.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,009] INFO Resolved hostname: <FQDN> to address: > <FQDN>/<ip of > > > > > localhost> (org.apache.zookeeper.server.quorum.QuorumPeer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,009] ERROR Invalid configuration, only one server > specified > > > > > (ignoring) (org.apac > > > > > he.zookeeper.server.quorum.QuorumPeerConfig) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,011] INFO autopurge.snapRetainCount set to 3 > > > > > (org.apache.zookeeper.server.Datadi > > > > > rCleanupManager) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,011] INFO autopurge.purgeInterval set to 0 > > > > > (org.apache.zookeeper.server.DatadirC > > > > > leanupManager) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,011] INFO Purge task is not scheduled. > > > > > (org.apache.zookeeper.server.DatadirCleanupManager) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,011] WARN Either no config or no quorum defined in config, > > > > > running in standalone mode > > > > > (org.apache.zookeeper.server.quorum.QuorumPeerMain > > > > > > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,027] INFO Reading configuration from: > > > > > /opt/kafka/kafka_2.12-2.2.2/config/zookeeper.properties > > > > > (org.apache.zookeeper.server.quorum.QuorumPeerConfig) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,027] INFO Resolved hostname: <FQDN> to address: > <FQDN>/<ip of > > > > > localhost> (org.apache.zookeeper.server.quorum.QuorumPeer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,027] ERROR Invalid configuration, only one server > specified > > > > > (ignoring) (org.apache.zookeeper.server.quorum.QuorumPeerConfig) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,027] INFO Starting server > > > > > (org.apache.zookeeper.server.ZooKeeperServerMain) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,033] INFO Server > > > > > > > > > environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, > > > > > built on 06/29/2018 00:39 GMT > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,033] INFO Server environment:host.name=<FQDN> > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,033] INFO Server environment:java.version=1.8.0_221 > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,033] INFO Server environment:java.vendor=Oracle > Corporation > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,033] INFO Server environment:java.home=/opt/jre1.8.0_221 > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,033] INFO Server > > > > > > > > > environment:java.class.path=/opt/kafka/kafka_2.12-2.2.2/bin/../libs/activation-1.1.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/aopalliance-repackaged-2.5.0-2.3.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javassist-3.22.0-CR2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.annotation-api-1.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.inject-1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.inject-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.ws.rs-api-2.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-client-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-common-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-container-servlet-2.27.jar:/opt/kafka/kafka_2.12 > > > > > > > > > > > > > > -2.2.2/bin/../libs/jersey-container-servlet-core-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-hk2-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-media-jaxb-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-server-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-client-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-continuation-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-util-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka_2.12-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka_2.12-2.2.2-sources.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-clients-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-log4j-appender-2.2.2.jar: > > > > > > > > > 2.2.2/bin/../libs/slf4j-log4j12-1.7.25.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/snappy-java-1.1.7.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/validation-api-1.1.0.Final.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zkclient-0.11.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zookeeper-3.4.13.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zstd-jni-1.3.8-1.jar > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,034] INFO Server > > > > > > > > > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,034] INFO Server environment:java.io.tmpdir=/tmp > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,034] INFO Server environment:java.compiler=<NA> > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,034] INFO Server environment:os.name=Linux > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,034] INFO Server environment:os.arch=amd64 > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,034] INFO Server > environment:os.version=3.10.0-1062.el7.x86_64 > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,034] INFO Server environment:user.name=kafka > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,034] INFO Server environment:user.home=/home/kafka > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,034] INFO Server environment:user.dir=/ > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,043] INFO tickTime set to 3000 > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,043] INFO minSessionTimeout set to -1 > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,044] INFO maxSessionTimeout set to -1 > > > > > (org.apache.zookeeper.server.ZooKeeperServer) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,049] INFO Using > > > org.apache.zookeeper.server.NIOServerCnxnFactory > > > > > as server connection factory > > > (org.apache.zookeeper.server.ServerCnxnFactory) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,054] INFO binding to port 0.0.0.0/0.0.0.0:5181 > > > > > (org.apache.zookeeper.server.NIOServerCnxnFactory) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19 > > > > > 18:16:08,131] ERROR Unexpected exception, exiting abnormally > > > > > (org.apache.zookeeper.server.ZooKeeperServerMain) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: > > > java.io.EOFException > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > java.io.DataInputStream.readInt(DataInputStream.java:392) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:66) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:650) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.persistence.FileTxnSnapLog.fastForwardFromEdits(FileTxnSnapLog.java:219) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:176) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:217) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:284) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:407) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:122) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:89) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:55) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:119) > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81) > > > > > Nov 19 18:16:08 <HOSTNAME> systemd: zookeeper.service: main process > > > > > exited, code=exited, status=1/FAILURE > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-stop.sh: No zookeeper > > > server > > > > > to stop > > > > > Nov 19 18:16:08 <HOSTNAME> systemd: zookeeper.service: control > process > > > > > exited, code=exited status=1 > > > > > Nov 19 18:16:08 <HOSTNAME> systemd: Unit zookeeper.service entered > > > failed > > > > > state. > > > > > Nov 19 18:16:08 <HOSTNAME> systemd: zookeeper.service failed. > > > > > > > > > > 7. config file - > > > > > > > > > > **/opt/kafka/kafka_2.12-2.2.2/config/server.properties** > > > > > > > > > > broker.id=1 > > > > > advertised.listeners=PLAINTEXT://hostname of localhost:3997 > > > > > listeners=PLAINTEXT://hostname of localhost:3997 > > > > > > > > > > delete.topic.enable=true > > > > > auto.create.topics.enable=True > > > > > offsets.topic.replication.factor=1 > > > > > > > > > > # The number of threads handling network requests > > > > > num.network.threads=3 > > > > > > > > > > # The number of threads doing disk I/O > > > > > num.io.threads=6 > > > > > > > > > > # The send buffer (SO_SNDBUF) used by the socket server > > > > > #socket.send.buffer.bytes=102400 > > > > > > > > > > # The receive buffer (SO_RCVBUF) used by the socket server > > > > > #socket.receive.buffer.bytes=102400 > > > > > > > > > > # The maximum size of a request that the socket server will accept > > > > > (protection against OOM) > > > > > #socket.request.max.bytes=104857600 > > > > > > > > > > # A comma seperated list of directories under which to store log > files > > > > > log.dirs=/var/log/kafka > > > > > > > > > > # The default number of log partitions per topic. > > > > > num.partitions=3 > > > > > > > > > > # Default Replication Factor > > > > > default.replication.factor=1 > > > > > > > > > > # The minimum age of a log file to be eligible for deletion > > > > > log.retention.hours=168 > > > > > > > > > > # A size-based retention policy for logs. Segments are pruned from > the > > > log > > > > > as long as the remaining > > > > > # segments don't drop below log.retention.bytes. > > > > > #log.retention.bytes=1073741824 > > > > > > > > > > # The maximum size of a log segment file. When this size is > reached a > > > new > > > > > log segment will be created. > > > > > log.segment.bytes=1073741824 > > > > > > > > > > # The interval at which log segments are checked to see if they > can be > > > > > deleted according > > > > > # to the retention policies > > > > > log.retention.check.interval.ms=300000 > > > > > > > > > > zookeeper.connect= <ip of localhost>:5181 > > > > > zookeeper.connection.timeout.ms=6000 > > > > > > > > > > > > > > > **/opt/kafka/kafka_2.12-2.2.2/config/zookeeper.properties** > > > > > > > > > > # clientPortBindAddress=localhost > > > > > clientPort=5181 > > > > > > > > > > dataDir=/var/data/zookeeper > > > > > dataLogDir=/var/data/logs/zookeeper > > > > > maxClientCnxns=0 > > > > > > > > > > # time for inital synchronization > > > > > initLimit=10 > > > > > # how many ticks can pass before timeout > > > > > syncLimit=5 > > > > > > > > > > server.1=<hostname of localhost>:2888:3888 > > > > > > > > > > 8. > > > > > > > > > > journalctl -fu zookeeper > > > > > -- Logs begin at Thu 2020-11-19 01:05:17 IST. -- > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118) > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:122) > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:89) > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:55) > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:119) > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81) > > > > > Nov 19 22:33:09 <HOSTNAME> systemd[1]: zookeeper.service: main > process > > > > > exited, code=exited, status=1/FAILURE > > > > > Nov 19 22:33:09 <HOSTNAME> systemd[1]: zookeeper.service: control > > > process > > > > > exited, code=exited status=1 > > > > > Nov 19 22:33:09 <HOSTNAME> systemd[1]: Unit zookeeper.service > entered > > > > > failed state. > > > > > Nov 19 22:33:09 <HOSTNAME> systemd[1]: zookeeper.service failed. > > > > > > > > > > journalctl -fu kafka.service > > > > > -- Logs begin at Thu 2020-11-19 01:05:17 IST. -- > > > > > Nov 19 22:23:30 <HOSTNAME> systemd[1]: Started kafka.service. > > > > > Nov 19 22:23:40 <HOSTNAME> systemd[1]: kafka.service: main process > > > exited, > > > > > code=exited, status=1/FAILURE > > > > > Nov 19 22:23:40 <HOSTNAME> systemd[1]: kafka.service: control > process > > > > > exited, code=exited status=1 > > > > > Nov 19 22:23:40 <HOSTNAME> systemd[1]: Unit kafka.service entered > > > failed > > > > > state. > > > > > Nov 19 22:23:40 <HOSTNAME> systemd[1]: kafka.service failed. > > > > > Nov 19 22:33:08 <HOSTNAME> systemd[1]: Started kafka.service. > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service: main process > > > exited, > > > > > code=exited, status=1/FAILURE > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service: control > process > > > > > exited, code=exited status=1 > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: Unit kafka.service entered > > > failed > > > > > state. > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service failed. > > > > > > > > > > this is the single server where everything is installed of elk > along > > > with > > > > > tomcat etc.. so I have stopped other services like tomcat for > better > > > > > resource management. hope resource crunch is not the issue. > > > > > > > > > > free -g > > > > > total used free shared buff/cache available > > > > > Mem: 15 7 6 0 1 7 > > > > > Swap: 3 0 3 > > > > > > > > > > Thanks, > > > > > > > > > > > > > > >
