Hi Sunil, 

Thanks for your reply. 

all four words commands where giving connection refused. 

I moved files (log.1,log.103,log.142,log.16a,log.17d - all where having oct 
timestamp) present inside `/var/data/logs/zookeeper/version-2/` to some other 
directory and started zookeeper and then kafka and both started and are up and 
running. I am able to list the topics now. 
now I am see log.1 file got created. (its binary so can't see inside it)

Can any one know the reason behind it? why it worked after moving these log* 
files which seems to be little old. 

not sure if same has to do after 1 month from now if same issue occurs again. 

@Sasikumar, Thanks. 

Thanks,


On 2020/11/22 12:43:12, sunil chaudhari <sunilmchaudhar...@gmail.com> wrote: 
> Hi,
> You can refer to the documentation for logging levels.
> 
> 
> https://zookeeper.apache.org/doc/r3.4.11/zookeeperAdmin.html#Debug+Log+Cleanup+%28log4j%29
> 
> By clearing current logs will help you to read and understand new logs.
> 
> Old logs wont get mixed up with the new one and you can read and debug
> events from the beginning of startup.
> 
> Regards,
> Sunil.
> 
> On Sun, 22 Nov 2020 at 5:52 PM, prat 007 <pratkul...@gmail.com> wrote:
> 
> > Hi Sunil,
> >
> > Thanks for your reply.
> > You mean to say to empty kafka.log file? (I don't see zookeeper.log file
> > in server), but not sure how clearing the log file itself will help.
> >
> > I will search for debug switch for zookeeper start command but I was
> > talking about in which zookeeper config file I can change the value of a
> > parameter from INFO to DEBUG or TRACE level to enable the whole log file to
> > show the logs in more depth.
> >
> > thanks,
> >
> > On 2020/11/22 12:00:26, sunil chaudhari <sunilmchaudhar...@gmail.com>
> > wrote:
> > > Hi I am talking about the Error :
> > >
> > > 18:16:08,027] ERROR Invalid configuration, only one server specified
> > > > > (ignoring) (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
> > >
> > >
> > > This seems to be the problem of whole cluster.
> > >
> > > Clear all logs means, clear existing server logs. Not data.
> > >
> > > You can see help of the zookeeper start command for debug switch.
> > >
> > > Regards,
> > > Sunil.
> > >
> > >
> > > On Sun, 22 Nov 2020 at 5:00 PM, prat 007 <pratkul...@gmail.com> wrote:
> > >
> > > > Hi Sunil,
> > > >
> > > > Thanks for reply. Q. Are you pointing to below line from point 6 to be
> > > > similar with url you have given (in Troubleshooting, under section:
> > Server
> > > > not coming up because of file corruption)?
> > > >
> > > > ov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh:
> > java.io.EOFException
> > > >
> > > > Q. `Clear all logs` - you mean to say delete all files from
> > > > `datadir/version-2 and datalogdir/version-2/`?
> > > > If yes, is it safe to delete them. Do we require them later ?
> > > >
> > > > the url says,
> > > > `make sure all the other servers in your ensemble are up and working.
> > Use
> > > > "stat" command on the command port to see if they are in good health`.
> > > >
> > > > Q. Is below command correct to verify good health ?
> > > > #  echo stat | nc localhost 2181
> > > >
> > > > Q. Can you please let me know where can I set zookeeper logs for
> > verbose
> > > > mode for debugging the issue?
> > > >
> > > > Thanks,
> > > >
> > > > On 2020/11/22 01:26:52, sunil chaudhari <sunilmchaudhar...@gmail.com>
> > > > wrote:
> > > > > Hi,
> > > > > Please check if it helps:
> > > > >
> > > >
> > http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html#sc_supportedPlatforms
> > > > >
> > > > > You have similar symptoms in point 6 in your original email.
> > > > >
> > > > > Try running it manually with below steps:
> > > > > Clear all logs
> > > > > Start zookeeper manually, watch logs for any error. Rectify it
> > referring
> > > > > above link.
> > > > > Start kafka manually. Watch info/error logs.
> > > > > Then try to list topics.
> > > > >
> > > > > Regards,
> > > > > Sunil.
> > > > >
> > > > >
> > > > > On Sat, 21 Nov 2020 at 10:16 PM, prat 007 <pratkul...@gmail.com>
> > wrote:
> > > > >
> > > > > > Hi Team,
> > > > > >
> > > > > > I am running ansible to install elk on one server. (on Dev env,
> > > > standalone
> > > > > > setup) but my playbook is getting failed at kafka role for task
> > > > listing of
> > > > > > Kafka topics. Can you please help.
> > > > > > below are the logs.
> > > > > >
> > > > > > 1.
> > > > > > playbook getting failed with error: -
> > > > > > ```
> > > > > > TASK [kafka : Kafka Topics | List topics]
> > > > > >
> > > >
> > ******************************************************************************************************************************
> > > > > > fatal: [kafka_1]: FAILED! => {
> > > > > >     "changed": false,
> > > > > >     "cmd": [
> > > > > >         "/opt/kafka/kafka_2.12-2.2.2/bin/kafka-topics.sh",
> > > > > >         "--zookeeper",
> > > > > >         "<FQDN>:5181",
> > > > > >         "--list"
> > > > > >     ],
> > > > > >     "delta": "0:00:34.084478",
> > > > > >     "end": "2020-11-19 22:33:42.624353",
> > > > > >     "rc": 1,
> > > > > >     "start": "2020-11-19 22:33:08.539875"
> > > > > > }
> > > > > >
> > > > > > STDERR:
> > > > > >
> > > > > > Exception in thread "main"
> > > > > > kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting
> > for
> > > > > > connection while in state: CONNECTING
> > > > > >         at
> > > > > >
> > > >
> > kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:242)
> > > > > >         at
> > > > > >
> > scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> > > > > >         at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:251)
> > > > > >         at
> > > > > >
> > > >
> > kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:238)
> > > > > >         at
> > > > kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:96)
> > > > > >         at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1825)
> > > > > > ```
> > > > > >
> > > > > > 2. Getting same error when run command manually which playbook is
> > > > trying
> > > > > > to run.
> > > > > >
> > > > > > ```
> > > > > > /opt/kafka/kafka_2.12-2.2.2/bin/kafka-topics.sh --zookeeper <ip of
> > > > > > localhost>:5181 --list
> > > > > >
> > > > > > Exception in thread "main"
> > > > > > kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting
> > for
> > > > > > connection while in state: CONNECTING
> > > > > >         at
> > > > > >
> > > >
> > kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:242)
> > > > > >         at
> > > > > >
> > scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> > > > > >         at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:251)
> > > > > >         at
> > > > > >
> > > >
> > kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:238)
> > > > > >         at
> > > > kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:96)
> > > > > >         at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1825)
> > > > > >         at
> > > > > >
> > > >
> > kafka.admin.TopicCommand$ZookeeperTopicService$.apply(TopicCommand.scala:262)
> > > > > >         at kafka.admin.TopicCommand$.main(TopicCommand.scala:53)
> > > > > >         at kafka.admin.TopicCommand.main(TopicCommand.scala)
> > > > > > ```
> > > > > >
> > > > > > 3. both Kafka and zookeeper are failed and not getting started.
> > > > > >
> > > > > > ```
> > > > > > systemctl status  kafka.service -l
> > > > > > ● kafka.service
> > > > > >    Loaded: loaded (/etc/systemd/system/kafka.service; enabled;
> > vendor
> > > > > > preset: disabled)
> > > > > >    Active: failed (Result: exit-code) since Thu 2020-11-19 22:33:17
> > > > IST;
> > > > > > 5min ago
> > > > > >   Process: 5448
> > > > > > ExecStop=/opt/kafka/kafka_2.12-2.2.2/bin/kafka-server-stop.sh
> > > > (code=exited,
> > > > > > status=1/FAILURE)
> > > > > >   Process: 4742 ExecStart=/bin/sh -c
> > > > > > /opt/kafka/kafka_2.12-2.2.2/bin/kafka-server-start.sh
> > > > > > /opt/kafka/kafka_2.12-2.2.2/config/server.properties >
> > > > > > /opt/kafka/kafka_2.12-2.2.2/kafka.log 2>&1 (code=exited,
> > > > status=1/FAILURE)
> > > > > >  Main PID: 4742 (code=exited, status=1/FAILURE)
> > > > > >
> > > > > > Nov 19 22:33:08 <HOSTNAME> systemd[1]: Started kafka.service.
> > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service: main process
> > > > exited,
> > > > > > code=exited, status=1/FAILURE
> > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service: control
> > process
> > > > > > exited, code=exited status=1
> > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: Unit kafka.service entered
> > > > failed
> > > > > > state.
> > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service failed.
> > > > > >
> > > > > >
> > > > > >  systemctl status  zookeeper.service  -l
> > > > > > ● zookeeper.service
> > > > > >    Loaded: loaded (/etc/systemd/system/zookeeper.service; enabled;
> > > > vendor
> > > > > > preset: disabled)
> > > > > >    Active: failed (Result: exit-code) since Thu 2020-11-19 22:33:09
> > > > IST;
> > > > > > 5min ago
> > > > > >   Process: 5281
> > > > > > ExecStop=/opt/kafka/kafka_2.12-2.2.2/bin/zookeeper-server-stop.sh
> > > > > > (code=exited, status=1/FAILURE)
> > > > > >   Process: 4448
> > > > > > ExecStart=/opt/kafka/kafka_2.12-2.2.2/bin/zookeeper-server-start.sh
> > > > > > /opt/kafka/kafka_2.12-2.2.2/config/zookeeper.properties
> > (code=exited,
> > > > > > status=1/FAILURE)
> > > > > >  Main PID: 4448 (code=exited, status=1/FAILURE)
> > > > > >
> > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118)
> > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:122)
> > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:89)
> > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:55)
> > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:119)
> > > > > > Nov 19 22:33:08 zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
> > > > > > Nov 19 22:33:09 systemd[1]: zookeeper.service: main process exited,
> > > > > > code=exited, status=1/FAILURE
> > > > > > Nov 19 22:33:09  systemd[1]: zookeeper.service: control process
> > exited,
> > > > > > code=exited status=1
> > > > > > Nov 19 22:33:09  systemd[1]: Unit zookeeper.service entered failed
> > > > state.
> > > > > > Nov 19 22:33:09  systemd[1]: zookeeper.service failed.
> > > > > > ```
> > > > > >
> > > > > > 4. therefore unable to do telnet or netstat
> > > > > >
> > > > > > telnet  <ip og localhost> 5181
> > > > > > Trying 10..
> > > > > > telnet: connect to address <ip of localhost>: Connection refused
> > > > > >
> > > > > > netstat  -tnlp |grep 5181
> > > > > > No output
> > > > > >
> > > > > > No Iptables or SELinux is running.
> > > > > >
> > > > > > 5. output of  /opt/kafka/kafka_2.12-2.2.2/kafka.log
> > > > > >
> > > > > > [2020-11-19 22:33:09,853] INFO Registered
> > > > kafka:type=kafka.Log4jController
> > > > > > MBean (kafka.utils.Log4jControllerRegistration$)
> > > > > > [2020-11-19 22:33:10,584] INFO starting (kafka.server.KafkaServer)
> > > > > > [2020-11-19 22:33:10,585] INFO Connecting to zookeeper on <ip of
> > > > > > localhost>:5181 (kafka.server.KafkaServer)
> > > > > > [2020-11-19 22:33:10,640] INFO [ZooKeeperClient] Initializing a new
> > > > > > session to <ip of localhost>:5181.
> > (kafka.zookeeper.ZooKeeperClient)
> > > > > > [2020-11-19 22:33:10,646] INFO Client
> > > > > >
> > > >
> > environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03,
> > > > > > built on 06/29/2018 00:39 GMT (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,646] INFO Client environment:host.name=<FQDN>
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,646] INFO Client
> > > > environment:java.version=1.8.0_221
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,646] INFO Client
> > environment:java.vendor=Oracle
> > > > > > Corporation (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,646] INFO Client
> > > > > > environment:java.home=/opt/jre1.8.0_221
> > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,646] INFO Client
> > > > > >
> > > >
> > environment:java.class.path=/opt/kafka/kafka_2.12-2.2.2/bin/../libs/activation-1.1.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/aopalliance-repackaged-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-api-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-basic-auth-extension-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-file-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-json-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-runtime-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/connect-transforms-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/guava-20.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/hk2-api-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/hk2-locator-2.5.0-b42.jar:/opt/kafka/ka
> > > > > >
> > > > > >
> > > >
> > fka_2.12-2.2.2/bin/../libs/hk2-utils-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-annotations-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-core-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-databind-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-datatype-jdk8-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-jaxrs-base-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-jaxrs-json-provider-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jackson-module-jaxb-annotations-2.10.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jakarta.activation-api-1.2.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javassist-3.22.0-CR2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.annotation-api-1.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.inject-1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.inject-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bi
> > > >  n/..
> > > > > >
> > > > > >
> > > >
> > /libs/javax.servlet-api-3.1.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/
> > > > > > javax.ws
> > > > > >
> > > >
> > .rs-api-2.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-client-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-common-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-container-servlet-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-container-servlet-core-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-hk2-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-media-jaxb-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-server-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-client-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-continuation-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-http-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-io-9.4.14.v20181114.jar:/opt/kafk
> > > > > >
> > > > > >
> > > >
> > a/kafka_2.12-2.2.2/bin/../libs/jetty-security-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-server-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-servlet-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-servlets-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-util-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka_2.12-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka_2.12-2.2.2-sources.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-clients-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-log4j-appender-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-streams-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-streams-examples-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-streams-scala_2.12-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-streams-test-utils-2.2.2.jar:/opt/kafka/kafk
> > > >  a_2.
> > > > > >
> > > > > >
> > > >
> > 12-2.2.2/bin/../libs/kafka-tools-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/log4j-1.2.17.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/lz4-java-1.5.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/maven-artifact-3.6.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/plexus-utils-3.1.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/reflections-0.9.11.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/rocksdbjni-5.15.10.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/scala-library-2.12.8.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/scala-logging_2.12-3.9.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/scala-reflect-2.12.8.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/slf4j-api-1.7.25.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/slf4j-log4j12-1.7.25.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/snappy-java-1.1.7.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/validation-api-1
> > > >  .1.0
> > > > > >
> > > >
> > .Final.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zkclient-0.11.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zookeeper-3.4.13.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zstd-jni-1.3.8-1.jar
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,646] INFO Client
> > > > > >
> > > >
> > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,646] INFO Client
> > environment:java.io.tmpdir=/tmp
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,647] INFO Client
> > environment:java.compiler=<NA>
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,647] INFO Client environment:os.name=Linux
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,647] INFO Client environment:os.arch=amd64
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,647] INFO Client
> > > > > > environment:os.version=3.10.0-1062.el7.x86_64
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,647] INFO Client environment:user.name=kafka
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,647] INFO Client
> > environment:user.home=/home/kafka
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,647] INFO Client environment:user.dir=/
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,648] INFO Initiating client connection,
> > > > > > connectString=<ip of localhost>:5181 sessionTimeout=6000
> > > > > >
> > > >
> > watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@3c419631
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:10,667] INFO Opening socket connection to server
> > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate
> > using
> > > > SASL
> > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:10,673] INFO Socket error occurred: <FQDN>/<ip of
> > > > > > localhost>:5181: Connection refused
> > (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:10,676] INFO [ZooKeeperClient] Waiting until
> > > > connected.
> > > > > > (kafka.zookeeper.ZooKeeperClient)
> > > > > > [2020-11-19 22:33:11,796] INFO Opening socket connection to server
> > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate
> > using
> > > > SASL
> > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:11,797] INFO Socket error occurred: <FQDN>/<ip of
> > > > > > localhost>:5181: Connection refused
> > (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:12,897] INFO Opening socket connection to server
> > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate
> > using
> > > > SASL
> > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:12,898] INFO Socket error occurred: <FQDN>/<ip of
> > > > > > localhost>:5181: Connection refused
> > (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:13,999] INFO Opening socket connection to server
> > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate
> > using
> > > > SASL
> > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:13,999] INFO Socket error occurred: <FQDN>/<ip of
> > > > > > localhost>:5181: Connection refused
> > (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:15,101] INFO Opening socket connection to server
> > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate
> > using
> > > > SASL
> > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:15,101] INFO Socket error occurred: <FQDN>/<ip of
> > > > > > localhost>:5181: Connection refused
> > (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:16,202] INFO Opening socket connection to server
> > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate
> > using
> > > > SASL
> > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:16,202] INFO Socket error occurred: <FQDN>/<ip of
> > > > > > localhost>:5181: Connection refused
> > (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:16,678] INFO [ZooKeeperClient] Closing.
> > > > > > (kafka.zookeeper.ZooKeeperClient)
> > > > > > [2020-11-19 22:33:17,303] INFO Opening socket connection to server
> > > > > > <FQDN>/<ip of localhost>:5181. Will not attempt to authenticate
> > using
> > > > SASL
> > > > > > (unknown error) (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:17,406] INFO Session: 0x0 closed
> > > > > > (org.apache.zookeeper.ZooKeeper)
> > > > > > [2020-11-19 22:33:17,407] INFO EventThread shut down for session:
> > 0x0
> > > > > > (org.apache.zookeeper.ClientCnxn)
> > > > > > [2020-11-19 22:33:17,409] INFO [ZooKeeperClient] Closed.
> > > > > > (kafka.zookeeper.ZooKeeperClient)
> > > > > > [2020-11-19 22:33:17,412] ERROR Fatal error during KafkaServer
> > startup.
> > > > > > Prepare to shutdown (kafka.server.KafkaServer)
> > > > > > kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting
> > for
> > > > > > connection while in state: CONNECTING
> > > > > >         at
> > > > > >
> > > >
> > kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:242)
> > > > > >         at
> > > > > >
> > scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
> > > > > >         at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:251)
> > > > > >         at
> > > > > >
> > > >
> > kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:238)
> > > > > >         at
> > > > kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:96)
> > > > > >         at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1825)
> > > > > >         at
> > > > kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:361)
> > > > > >         at
> > kafka.server.KafkaServer.initZkClient(KafkaServer.scala:385)
> > > > > >         at kafka.server.KafkaServer.startup(KafkaServer.scala:205)
> > > > > >         at
> > > > > >
> > > >
> > kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
> > > > > >         at kafka.Kafka$.main(Kafka.scala:75)
> > > > > >         at kafka.Kafka.main(Kafka.scala)
> > > > > > [2020-11-19 22:33:17,415] INFO shutting down
> > (kafka.server.KafkaServer)
> > > > > > [2020-11-19 22:33:17,422] INFO shut down completed
> > > > > > (kafka.server.KafkaServer)
> > > > > > [2020-11-19 22:33:17,422] ERROR Exiting Kafka.
> > > > > > (kafka.server.KafkaServerStartable)
> > > > > > [2020-11-19 22:33:17,426] INFO shutting down
> > (kafka.server.KafkaServer)
> > > > > >
> > > > > > 6. /var/log/messages
> > > > > >
> > > > > > Nov 19 18:16:07 <HOSTNAME> ansible-command: Invoked with
> > creates=None
> > > > > > executable=None _uses_shell=False strip_empty_ends=True
> > > > > > _raw_params=/opt/kafka/kafka_
> > > > > > 2.12-2.2.2/bin/kafka-topics.sh --zookeeper <FQDN>:5181 --list
> > > > removes=None
> > > > > > argv=None warn=True chdir=None stdin_add_newline=True stdin=None
> > > > > > Nov 19 18:16:07 <HOSTNAME> ansible-command: Invoked with
> > creates=None
> > > > > > executable=None _uses_shell=False strip_empty_ends=True
> > > > > > _raw_params=/opt/kafka/kafka_
> > > > > > 2.12-2.2.2/bin/kafka-topics.sh --zookeeper <FQDN>:5181 --list
> > > > removes=None
> > > > > > argv=None warn=True chdir=None stdin_add_newline=True stdin=None
> > > > > > Nov 19 18:16:07 <HOSTNAME> ansible-command: Invoked with
> > creates=None
> > > > > > executable=None _uses_shell=False strip_empty_ends=True
> > > > > > _raw_params=/opt/kafka/kafka_
> > > > > > 2.12-2.2.2/bin/kafka-topics.sh --zookeeper <FQDN>:5181 --list
> > > > removes=None
> > > > > > argv=None warn=True chdir=None stdin_add_newline=True stdin=None
> > > > > > Nov 19 18:16:07 <HOSTNAME> heartbeat:
> > > > > >
> > > >
> > 2020-11-19T18:16:07.941+0530#011INFO#011kafka/log.go:53#011client/metadata
> > > > > > fetching metadata for [heartbeat] from bro
> > > > > > ker <ip of localhost>:3997
> > > > > > <ip of localhost>:3997: connect: connection refused
> > > > > > 19T18:16:07.942+0530#011INFO#011kafka/log.go:53#011client/metadata
> > got
> > > > > > error from broker -1 while fetching me
> > > > > > tadata: dial tcp <ip of localhost>:3997: connect: connection
> > refused
> > > > > > Nov 19 18:16:07 <HOSTNAME> heartbeat:
> > > > > > 2020-11-19T18:16:07.942+0530#011INFO#011kafka/log.go:53#011kafka
> > > > message:
> > > > > > client/metadata no available broker to send
> > > > > >  metadata request to
> > > > > > Nov 19 18:16:07 <HOSTNAME> heartbeat:
> > > > > >
> > > >
> > 2020-11-19T18:16:07.942+0530#011INFO#011[publisher]#011pipeline/retry.go:155#011Drop
> > > > > > batch
> > > > > > Nov 19 18:16:07 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:07,962] INFO Reading configuration from:
> > > > > > /opt/kafka/kafka_2.12-2.2.2/config/zookeep
> > > > > > er.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,009] INFO Resolved hostname: <FQDN> to address:
> > <FQDN>/<ip of
> > > > > > localhost> (org.apache.zookeeper.server.quorum.QuorumPeer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,009] ERROR Invalid configuration, only one server
> > specified
> > > > > > (ignoring) (org.apac
> > > > > > he.zookeeper.server.quorum.QuorumPeerConfig)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,011] INFO autopurge.snapRetainCount set to 3
> > > > > > (org.apache.zookeeper.server.Datadi
> > > > > > rCleanupManager)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,011] INFO autopurge.purgeInterval set to 0
> > > > > > (org.apache.zookeeper.server.DatadirC
> > > > > > leanupManager)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,011] INFO Purge task is not scheduled.
> > > > > > (org.apache.zookeeper.server.DatadirCleanupManager)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,011] WARN Either no config or no quorum defined in config,
> > > > > > running  in standalone mode
> > > > > > (org.apache.zookeeper.server.quorum.QuorumPeerMain
> > > > > >
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,027] INFO Reading configuration from:
> > > > > > /opt/kafka/kafka_2.12-2.2.2/config/zookeeper.properties
> > > > > > (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,027] INFO Resolved hostname: <FQDN> to address:
> > <FQDN>/<ip of
> > > > > > localhost> (org.apache.zookeeper.server.quorum.QuorumPeer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,027] ERROR Invalid configuration, only one server
> > specified
> > > > > > (ignoring) (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,027] INFO Starting server
> > > > > > (org.apache.zookeeper.server.ZooKeeperServerMain)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,033] INFO Server
> > > > > >
> > > >
> > environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03,
> > > > > > built on 06/29/2018 00:39 GMT
> > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,033] INFO Server environment:host.name=<FQDN>
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,033] INFO Server environment:java.version=1.8.0_221
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,033] INFO Server environment:java.vendor=Oracle
> > Corporation
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,033] INFO Server environment:java.home=/opt/jre1.8.0_221
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,033] INFO Server
> > > > > >
> > > >
> > environment:java.class.path=/opt/kafka/kafka_2.12-2.2.2/bin/../libs/activation-1.1.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/aopalliance-repackaged-2.5.0-2.3.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javassist-3.22.0-CR2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.annotation-api-1.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.inject-1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.inject-2.5.0-b42.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/javax.ws.rs-api-2.1.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-client-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-common-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-container-servlet-2.27.jar:/opt/kafka/kafka_2.12
> > > > > >
> > > > > >
> > > >
> > -2.2.2/bin/../libs/jersey-container-servlet-core-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-hk2-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-media-jaxb-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jersey-server-2.27.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-client-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-continuation-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jetty-util-9.4.14.v20181114.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka_2.12-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka_2.12-2.2.2-sources.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-clients-2.2.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/kafka-log4j-appender-2.2.2.jar:
> > > > > >
> > > >
> > 2.2.2/bin/../libs/slf4j-log4j12-1.7.25.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/snappy-java-1.1.7.2.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/validation-api-1.1.0.Final.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zkclient-0.11.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zookeeper-3.4.13.jar:/opt/kafka/kafka_2.12-2.2.2/bin/../libs/zstd-jni-1.3.8-1.jar
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,034] INFO Server
> > > > > >
> > > >
> > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,034] INFO Server environment:java.io.tmpdir=/tmp
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,034] INFO Server environment:java.compiler=<NA>
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,034] INFO Server environment:os.name=Linux
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,034] INFO Server environment:os.arch=amd64
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,034] INFO Server
> > environment:os.version=3.10.0-1062.el7.x86_64
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,034] INFO Server environment:user.name=kafka
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,034] INFO Server environment:user.home=/home/kafka
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,034] INFO Server environment:user.dir=/
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,043] INFO tickTime set to 3000
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,043] INFO minSessionTimeout set to -1
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,044] INFO maxSessionTimeout set to -1
> > > > > > (org.apache.zookeeper.server.ZooKeeperServer)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,049] INFO Using
> > > > org.apache.zookeeper.server.NIOServerCnxnFactory
> > > > > > as server connection factory
> > > > (org.apache.zookeeper.server.ServerCnxnFactory)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,054] INFO binding to port 0.0.0.0/0.0.0.0:5181
> > > > > > (org.apache.zookeeper.server.NIOServerCnxnFactory)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: [2020-11-19
> > > > > > 18:16:08,131] ERROR Unexpected exception, exiting abnormally
> > > > > > (org.apache.zookeeper.server.ZooKeeperServerMain)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh:
> > > > java.io.EOFException
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > > java.io.DataInputStream.readInt(DataInputStream.java:392)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:66)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:650)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.persistence.FileTxnSnapLog.fastForwardFromEdits(FileTxnSnapLog.java:219)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:176)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:217)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:284)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:407)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:122)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:89)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:55)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:119)
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-start.sh: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
> > > > > > Nov 19 18:16:08 <HOSTNAME> systemd: zookeeper.service: main process
> > > > > > exited, code=exited, status=1/FAILURE
> > > > > > Nov 19 18:16:08 <HOSTNAME> zookeeper-server-stop.sh: No zookeeper
> > > > server
> > > > > > to stop
> > > > > > Nov 19 18:16:08 <HOSTNAME> systemd: zookeeper.service: control
> > process
> > > > > > exited, code=exited status=1
> > > > > > Nov 19 18:16:08 <HOSTNAME> systemd: Unit zookeeper.service entered
> > > > failed
> > > > > > state.
> > > > > > Nov 19 18:16:08 <HOSTNAME> systemd: zookeeper.service failed.
> > > > > >
> > > > > > 7. config file -
> > > > > >
> > > > > > **/opt/kafka/kafka_2.12-2.2.2/config/server.properties**
> > > > > >
> > > > > > broker.id=1
> > > > > > advertised.listeners=PLAINTEXT://hostname of localhost:3997
> > > > > > listeners=PLAINTEXT://hostname of localhost:3997
> > > > > >
> > > > > > delete.topic.enable=true
> > > > > > auto.create.topics.enable=True
> > > > > > offsets.topic.replication.factor=1
> > > > > >
> > > > > > # The number of threads handling network requests
> > > > > > num.network.threads=3
> > > > > >
> > > > > > # The number of threads doing disk I/O
> > > > > > num.io.threads=6
> > > > > >
> > > > > > # The send buffer (SO_SNDBUF) used by the socket server
> > > > > > #socket.send.buffer.bytes=102400
> > > > > >
> > > > > > # The receive buffer (SO_RCVBUF) used by the socket server
> > > > > > #socket.receive.buffer.bytes=102400
> > > > > >
> > > > > > # The maximum size of a request that the socket server will accept
> > > > > > (protection against OOM)
> > > > > > #socket.request.max.bytes=104857600
> > > > > >
> > > > > > # A comma seperated list of directories under which to store log
> > files
> > > > > > log.dirs=/var/log/kafka
> > > > > >
> > > > > > # The default number of log partitions per topic.
> > > > > > num.partitions=3
> > > > > >
> > > > > > # Default Replication Factor
> > > > > > default.replication.factor=1
> > > > > >
> > > > > > # The minimum age of a log file to be eligible for deletion
> > > > > > log.retention.hours=168
> > > > > >
> > > > > > # A size-based retention policy for logs. Segments are pruned from
> > the
> > > > log
> > > > > > as long as the remaining
> > > > > > # segments don't drop below log.retention.bytes.
> > > > > > #log.retention.bytes=1073741824
> > > > > >
> > > > > > # The maximum size of a log segment file. When this size is
> > reached a
> > > > new
> > > > > > log segment will be created.
> > > > > > log.segment.bytes=1073741824
> > > > > >
> > > > > > # The interval at which log segments are checked to see if they
> > can be
> > > > > > deleted according
> > > > > > # to the retention policies
> > > > > > log.retention.check.interval.ms=300000
> > > > > >
> > > > > > zookeeper.connect= <ip of localhost>:5181
> > > > > > zookeeper.connection.timeout.ms=6000
> > > > > >
> > > > > >
> > > > > >  **/opt/kafka/kafka_2.12-2.2.2/config/zookeeper.properties**
> > > > > >
> > > > > > # clientPortBindAddress=localhost
> > > > > > clientPort=5181
> > > > > >
> > > > > > dataDir=/var/data/zookeeper
> > > > > > dataLogDir=/var/data/logs/zookeeper
> > > > > > maxClientCnxns=0
> > > > > >
> > > > > > # time for inital synchronization
> > > > > > initLimit=10
> > > > > > # how many ticks can pass before timeout
> > > > > > syncLimit=5
> > > > > >
> > > > > > server.1=<hostname of localhost>:2888:3888
> > > > > >
> > > > > > 8.
> > > > > >
> > > > > > journalctl -fu zookeeper
> > > > > > -- Logs begin at Thu 2020-11-19 01:05:17 IST. --
> > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118)
> > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:122)
> > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:89)
> > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:55)
> > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:119)
> > > > > > Nov 19 22:33:08 <HOSTNAME> zookeeper-server-start.sh[4448]: at
> > > > > >
> > > >
> > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
> > > > > > Nov 19 22:33:09 <HOSTNAME> systemd[1]: zookeeper.service: main
> > process
> > > > > > exited, code=exited, status=1/FAILURE
> > > > > > Nov 19 22:33:09 <HOSTNAME> systemd[1]: zookeeper.service: control
> > > > process
> > > > > > exited, code=exited status=1
> > > > > > Nov 19 22:33:09 <HOSTNAME> systemd[1]: Unit zookeeper.service
> > entered
> > > > > > failed state.
> > > > > > Nov 19 22:33:09 <HOSTNAME> systemd[1]: zookeeper.service failed.
> > > > > >
> > > > > > journalctl -fu kafka.service
> > > > > > -- Logs begin at Thu 2020-11-19 01:05:17 IST. --
> > > > > > Nov 19 22:23:30 <HOSTNAME> systemd[1]: Started kafka.service.
> > > > > > Nov 19 22:23:40 <HOSTNAME> systemd[1]: kafka.service: main process
> > > > exited,
> > > > > > code=exited, status=1/FAILURE
> > > > > > Nov 19 22:23:40 <HOSTNAME> systemd[1]: kafka.service: control
> > process
> > > > > > exited, code=exited status=1
> > > > > > Nov 19 22:23:40 <HOSTNAME> systemd[1]: Unit kafka.service entered
> > > > failed
> > > > > > state.
> > > > > > Nov 19 22:23:40 <HOSTNAME> systemd[1]: kafka.service failed.
> > > > > > Nov 19 22:33:08 <HOSTNAME> systemd[1]: Started kafka.service.
> > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service: main process
> > > > exited,
> > > > > > code=exited, status=1/FAILURE
> > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service: control
> > process
> > > > > > exited, code=exited status=1
> > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: Unit kafka.service entered
> > > > failed
> > > > > > state.
> > > > > > Nov 19 22:33:17 <HOSTNAME> systemd[1]: kafka.service failed.
> > > > > >
> > > > > > this is the single server where everything is installed of elk
> > along
> > > > with
> > > > > > tomcat etc.. so I have stopped other services like tomcat for
> > better
> > > > > > resource management. hope resource crunch is not the issue.
> > > > > >
> > > > > > free -g
> > > > > > total used free shared buff/cache available
> > > > > > Mem: 15 7 6 0 1 7
> > > > > > Swap: 3 0 3
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > >
> > > >
> > >
> >
> 

Reply via email to