Thanks for the reply, I tried changing the data directory as follows:-
dataDir=/data/zookeeper
I also create a /data but still I am getting the following error.
ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.
ZooKeeperServerMain)
java.io.IOException: Unable to create data directory
/data/zookeeper/version-2
Gourab
On Sat, Oct 22, 2016 at 3:44 AM, Hans Jespersen <h...@confluent.io> wrote:
> You are going to lose everything you store in /tmp. In a production system
> you never configure Kafka or zookeeper to store critical data in /tmp.
> This has nothing to do with AWS or EBS it is just standard Linux than
> everything under /tmp is deleted when Linux reboots.
>
> -hans
>
> /**
> * Hans Jespersen, Principal Systems Engineer, Confluent Inc.
> * h...@confluent.io (650)924-2670
> */
>
> On Fri, Oct 21, 2016 at 10:38 AM, Gourab Chowdhury <gourab....@gmail.com>
> wrote:
>
> > I was testing a scenario of a server crash while the Kafka is still
> running
> > and passing messages. (just did a sudo reboot). By default, I am storing
> > the logs at /tmp/kafka-log.
> >
> > So in case an EC2 server restart, am I going to loose the data if I don't
> > store kafka logs in an EBS drive?
> >
> > Also, I have another doubt, Does zookeeper stores Kafka info (viz.
> topics,
> > brokers list, consumer groups) at /tmp/zookeeper/version-2/log.x ?
> >
> > I am using python zookeeper client: *kazoo* and here we can find various
> > Kafka info like '/brokers/topics', '/brokers/topics/topic/partitions' ,
> > '/consumers' , '/consumers/consumer/offsets'. Where can I get this info
> > directly?
> >
> > ~Gourab
> >
> > On Fri, Oct 21, 2016 at 8:13 PM, Ben Davison <ben.davi...@7digital.com>
> > wrote:
> >
> > > Are you restarting or terminating the instance? If your terminating,
> and
> > > don't have an EBS drive attached you will lose all data on the drive.
> > >
> > > On Fri, Oct 21, 2016 at 2:09 PM, Gourab Chowdhury <
> gourab....@gmail.com>
> > > wrote:
> > >
> > > > I am running kafka in a EC2 server. While running if there is a
> sudden
> > > > restart of server, and then we restart the kafka broker, we can't
> find
> > > the
> > > > details of all the topics.
> > > >
> > > > bin/kafka-topics.sh --describe --zookeeper <hostname:port>
> > > > gives no result
> > > >
> > > > bin/kafka-console-producer.sh --broker-list <hostname:port> --topic
> > test
> > > > yields this particular error as follows:
> > > >
> > > > WARN Error while fetching metadata with correlation id n :
> > > > {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.
> > > NetworkClient)
> > > >
> > > > Is the details of topics from the zookeeper getting deleted while
> > > restart?
> > > > or this is something else?
> > > >
> > > > Also how can I cope with this in in real life production?
> > > >
> > > > Regards,
> > > > Gourab Chowdhury,
> > > >
> > >
> > > --
> > >
> > >
> > > This email, including attachments, is private and confidential. If you
> > have
> > > received this email in error please notify the sender and delete it
> from
> > > your system. Emails are not secure and may contain viruses. No
> liability
> > > can be accepted for viruses that might be transferred by this email or
> > any
> > > attachment. Any unauthorised copying of this message or unauthorised
> > > distribution and publication of the information contained herein are
> > > prohibited.
> > >
> > > 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
> > > Registered in England and Wales. Registered No. 04843573.
> > >
> >
>
