You probably didn't setup permissions properly on the /data directory you
created.
You might want to do some free online training to learn some more Linux if
you are planning to become a Kafka administrator.
https://en.wikipedia.org/wiki/Chmod
https://www.edx.org/course/introduction-linux-linuxfoundationx-lfs101x-0
-hans
/**
* Hans Jespersen, Principal Systems Engineer, Confluent Inc.
* h...@confluent.io (650)924-2670
*/
On Mon, Oct 24, 2016 at 1:50 AM, Gourab Chowdhury <gourab....@gmail.com>
wrote:
> Thanks for the reply, I tried changing the data directory as follows:-
> dataDir=/data/zookeeper
>
> I also create a /data but still I am getting the following error.
>
> ERROR Unexpected exception, exiting abnormally
> (org.apache.zookeeper.server.
> ZooKeeperServerMain)
> java.io.IOException: Unable to create data directory
> /data/zookeeper/version-2
>
> Gourab
>
> On Sat, Oct 22, 2016 at 3:44 AM, Hans Jespersen <h...@confluent.io> wrote:
>
> > You are going to lose everything you store in /tmp. In a production
> system
> > you never configure Kafka or zookeeper to store critical data in /tmp.
> > This has nothing to do with AWS or EBS it is just standard Linux than
> > everything under /tmp is deleted when Linux reboots.
> >
> > -hans
> >
> > /**
> > * Hans Jespersen, Principal Systems Engineer, Confluent Inc.
> > * h...@confluent.io (650)924-2670
> > */
> >
> > On Fri, Oct 21, 2016 at 10:38 AM, Gourab Chowdhury <gourab....@gmail.com
> >
> > wrote:
> >
> > > I was testing a scenario of a server crash while the Kafka is still
> > running
> > > and passing messages. (just did a sudo reboot). By default, I am
> storing
> > > the logs at /tmp/kafka-log.
> > >
> > > So in case an EC2 server restart, am I going to loose the data if I
> don't
> > > store kafka logs in an EBS drive?
> > >
> > > Also, I have another doubt, Does zookeeper stores Kafka info (viz.
> > topics,
> > > brokers list, consumer groups) at /tmp/zookeeper/version-2/log.x ?
> > >
> > > I am using python zookeeper client: *kazoo* and here we can find
> various
> > > Kafka info like '/brokers/topics', '/brokers/topics/topic/partitions'
> ,
> > > '/consumers' , '/consumers/consumer/offsets'. Where can I get this info
> > > directly?
> > >
> > > ~Gourab
> > >
> > > On Fri, Oct 21, 2016 at 8:13 PM, Ben Davison <ben.davi...@7digital.com
> >
> > > wrote:
> > >
> > > > Are you restarting or terminating the instance? If your terminating,
> > and
> > > > don't have an EBS drive attached you will lose all data on the drive.
> > > >
> > > > On Fri, Oct 21, 2016 at 2:09 PM, Gourab Chowdhury <
> > gourab....@gmail.com>
> > > > wrote:
> > > >
> > > > > I am running kafka in a EC2 server. While running if there is a
> > sudden
> > > > > restart of server, and then we restart the kafka broker, we can't
> > find
> > > > the
> > > > > details of all the topics.
> > > > >
> > > > > bin/kafka-topics.sh --describe --zookeeper <hostname:port>
> > > > > gives no result
> > > > >
> > > > > bin/kafka-console-producer.sh --broker-list <hostname:port> --topic
> > > test
> > > > > yields this particular error as follows:
> > > > >
> > > > > WARN Error while fetching metadata with correlation id n :
> > > > > {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.
> > > > NetworkClient)
> > > > >
> > > > > Is the details of topics from the zookeeper getting deleted while
> > > > restart?
> > > > > or this is something else?
> > > > >
> > > > > Also how can I cope with this in in real life production?
> > > > >
> > > > > Regards,
> > > > > Gourab Chowdhury,
> > > > >
> > > >
> > > > --
> > > >
> > > >
> > > > This email, including attachments, is private and confidential. If
> you
> > > have
> > > > received this email in error please notify the sender and delete it
> > from
> > > > your system. Emails are not secure and may contain viruses. No
> > liability
> > > > can be accepted for viruses that might be transferred by this email
> or
> > > any
> > > > attachment. Any unauthorised copying of this message or unauthorised
> > > > distribution and publication of the information contained herein are
> > > > prohibited.
> > > >
> > > > 7digital Limited. Registered office: 69 Wilson Street, London EC2A
> 2BB.
> > > > Registered in England and Wales. Registered No. 04843573.
> > > >
> > >
> >
>
