Hi Zak, Why don't you try using zookeeper four letter admin tools <https://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#The+Four+Letter+Words> ?
echo stat | nc zookeeper-ip zookeeper-port echo stat | nc localhost 2181 These commands will tell you the current status of zookeeper client. https://zookeeper.apache.org/doc/trunk/zookeeperAdmin.html#The+Four+Letter+Words Thanks Sudev On Thu, 1 Dec 2016 at 5:11 AM, Flavio Junqueira <f...@apache.org> wrote: Hey Martin, Do you want to create a jira for this and report the issue? -Flavio > On 30 Nov 2016, at 18:33, Martin Gainty <mgai...@hotmail.com> wrote: > > a shock when a zk script goes fubar > > > zookeeper devs can we get some help for sh zkServer.sh status? > > > Thanks! > > Martin > ______________________________________________ > > > > ________________________________ > From: Zac Harvey <zac.har...@welltok.com> > Sent: Wednesday, November 30, 2016 10:25 AM > To: users@kafka.apache.org > Subject: Re: Messages intermittently get lost > > Hi Martin, makes sense. > > > When I SSH into all 3 of my ZK nodes and run: > > > sh zkServer.sh status > > > All three of them give me the following output: > > > JMX enabled by default > > zkServer.sh: 81: /opt/zookeeper/bin/zkEnv.sh: Syntax error: "(" unexpected (expecting "fi") > > > Looks like a bug in the ZK shell script? > > > Best, > > Zac > > ________________________________ > From: Martin Gainty <mgai...@hotmail.com> > Sent: Tuesday, November 29, 2016 11:18:21 AM > To: users@kafka.apache.org > Subject: Re: Messages intermittently get lost > > Hi Zach > > > we dont know whats causing this intermittent problem..so lets Divide and Conquer each part of this problem individually starting at the source of the data feeds > > > Let us eliminate any potential problem with feeds from external sources > > > Once you verify the zookeeper feeds are 100% reliable lets move onto Kafka > > > Pingback when you have verifiable results from zookeeper feeds > > > Thanks > > Martin > ______________________________________________ > > > > ________________________________ > From: Zac Harvey <zac.har...@welltok.com> > Sent: Tuesday, November 29, 2016 10:46 AM > To: users@kafka.apache.org > Subject: Re: Messages intermittently get lost > > Does anybody have any idea why ZK might be to blame if messages sent by a Kafka producer fail to be received by a Kafka consumer? > > ________________________________ > From: Zac Harvey <zac.har...@welltok.com> > Sent: Monday, November 28, 2016 9:07:41 AM > To: users@kafka.apache.org > Subject: Re: Messages intermittently get lost > > Thanks Martin, I will look at those links. > > > But you seem to be 100% confident that the problem is with ZooKeeper...can I ask why? What is it about my problem description that makes you think this is an issue with ZooKeeper? > > ________________________________ > From: Martin Gainty <mgai...@hotmail.com> > Sent: Friday, November 25, 2016 1:46:28 PM > To: users@kafka.apache.org > Subject: Re: Messages intermittently get lost > > > > ________________________________ > From: Zac Harvey <zac.har...@welltok.com> > Sent: Friday, November 25, 2016 6:17 AM > To: users@kafka.apache.org > Subject: Re: Messages intermittently get lost > > Hi Martin, > > > My server.properties looks like this: > > > listeners=PLAINTEXT://0.0.0.0:9092 > > advertised.host.name=<SERVER_PUBLIC_DNS> > > broker.id=2 > > port=9092 > > num.partitions=4 > > zookeeper.connect=zkA:2181,zkB:2181,zkC:2181 > > MG>can you check status for each ZK Node in the quorum? > > sh>$ZOOKEEPER_HOME/bin/zkServer.sh status > > http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html > ZooKeeper problems and solutions - IBM< http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html > > www.ibm.com > Use these solutions to resolve the problems that you might encounter with Apache ZooKeeper. > > > > ZooKeeper problems and solutions - IBM< http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html > > ZooKeeper problems and solutions - IBM< http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html > > www.ibm.com > Use these solutions to resolve the problems that you might encounter with Apache ZooKeeper. > > > > www.ibm.com<http://www.ibm.com> > [ http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png ]<http://www.ibm.com/> > > IBM - United States<http://www.ibm.com/> > www.ibm.com > For more than a century IBM has been dedicated to every client's success and to creating innovations that matter for the world > > > > Use these solutions to resolve the problems that you might encounter with Apache ZooKeeper. > > > > > ZooKeeper problems and solutions - IBM< http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html > > ZooKeeper problems and solutions - IBM< http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html > > www.ibm.com > Use these solutions to resolve the problems that you might encounter with Apache ZooKeeper. > > > > ZooKeeper problems and solutions - IBM< http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html > > ZooKeeper problems and solutions - IBM< http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.pd.doc/doc/containerstreamszookeeper.html > > www.ibm.com > Use these solutions to resolve the problems that you might encounter with Apache ZooKeeper. > > > > www.ibm.com<http://www.ibm.com> > [ http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png ]<http://www.ibm.com/> > > IBM - United States<http://www.ibm.com/> > www.ibm.com > For more than a century IBM has been dedicated to every client's success and to creating innovations that matter for the world > > > > Use these solutions to resolve the problems that you might encounter with Apache ZooKeeper. > > > > www.ibm.com<http://www.ibm.com> > [ http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png ]<http://www.ibm.com/> > > IBM - United States<http://www.ibm.com/> > www.ibm.com > For more than a century IBM has been dedicated to every client's success and to creating innovations that matter for the world > > > > [ http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png ]<http://www.ibm.com/> > > IBM - United States<http://www.ibm.com/> > [ http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png ]<http://www.ibm.com/> > > IBM - United States<http://www.ibm.com/> > www.ibm.com > For more than a century IBM has been dedicated to every client's success and to creating innovations that matter for the world > > > > www.ibm.com<http://www.ibm.com> > [ http://upload.wikimedia.org/wikipedia/commons/thumb/5/51/IBM_logo.svg/200px-IBM_logo.svg.png ]<http://www.ibm.com/> > > IBM - United States<http://www.ibm.com/> > www.ibm.com > For more than a century IBM has been dedicated to every client's success and to creating innovations that matter for the world > > > > For more than a century IBM has been dedicated to every client's success and to creating innovations that matter for the world > > > > Use these solutions to resolve the problems that you might encounter with Apache ZooKeeper. > > > > MG>*greetings from Plimoth Mass* > > MG>M > > > num.network.threads=3 > > num.io.threads=8 > > socket.send.buffer.bytes=102400 > > socket.receive.buffer.bytes=102400 > > log.dirs=/tmp/kafka-logs > > num.recovery.threads.per.data.dir=1 > > log.retention.hours=168 > > log.segment.bytes=1073741824 > > log.retention.check.interval.ms=300000 > > zookeeper.connection.timeout.ms=6000 > > delete.topic.enable=true > > auto.leader.rebalance.enable=true > > > Above, 'zkA', 'zkB' and 'zkC' are defined in /etc/hosts and are valid ZK servers, and <SERVER_PUBLIC_DNS> is the public DNS of the EC2 (AWS) node that this Kafka is running on. > > > Anything look incorrect to you? > > > And yes, yesterday was a holiday, but there is only work! I'll celebrate one big, long holiday when I'm dead! > > > Thanks for any-and-all input here! > > > Best, > > Zac > > > ________________________________ > From: Martin Gainty <mgai...@hotmail.com> > Sent: Thursday, November 24, 2016 9:03:33 AM > To: users@kafka.apache.org > Subject: Re: Messages intermittently get lost > > Hi Zach > > > there is a rumour that today thursday is a holiday? > > in server.properties how are you configuring your server? > > > specifically what are these attributes? > > > num.network.threads= > > > num.io.threads= > > > socket.send.buffer.bytes= > > > socket.receive.buffer.bytes= > > > socket.request.max.bytes= > > > num.partitions= > > > num.recovery.threads.per.data.dir= > > > ? > > Martin > ______________________________________________ > > > > ________________________________ > From: Zac Harvey <zac.har...@welltok.com> > Sent: Thursday, November 24, 2016 7:05 AM > To: users@kafka.apache.org > Subject: Re: Messages intermittently get lost > > Anybody?!? This is very disconcerting! > > ________________________________ > From: Zac Harvey <zac.har...@welltok.com> > Sent: Wednesday, November 23, 2016 5:07:45 AM > To: users@kafka.apache.org > Subject: Messages intermittently get lost > > I am playing around with Kafka and have a simple setup: > > > * 1-node Kafka (Ubuntu) server > > * 3-node ZK cluster (each on their own Ubuntu server) > > > I have a consumer written in Scala and am using the kafka-console-producer (v0.10) that ships with the distribution. > > > I'd say about 20% of the messages I send via the producer never get consumed by the Scala process (which is running continuously). No errors on either side (producer or consumer): the producer sends, and, nothing... > > > Any ideas as to what might be going on here, or how I could start troubleshooting? > > > Thanks!
