No issues, I have solved the problem by changing the permission of /data
Thanks,
Gourab
On Mon, Oct 24, 2016 at 2:20 PM, Gourab Chowdhury <gourab....@gmail.com>
wrote:
> Thanks for the reply, I tried changing the data directory as follows:-
> dataDir=/data/zookeeper
>
> I also create a /data but still I am getting the following error.
>
> ERROR Unexpected exception, exiting abnormally
> (org.apache.zookeeper.server.ZooKeeperServerMain)
> java.io.IOException: Unable to create data directory
> /data/zookeeper/version-2
>
> Gourab
>
> On Sat, Oct 22, 2016 at 3:44 AM, Hans Jespersen <h...@confluent.io> wrote:
>
>> You are going to lose everything you store in /tmp. In a production system
>> you never configure Kafka or zookeeper to store critical data in /tmp.
>> This has nothing to do with AWS or EBS it is just standard Linux than
>> everything under /tmp is deleted when Linux reboots.
>>
>> -hans
>>
>> /**
>> * Hans Jespersen, Principal Systems Engineer, Confluent Inc.
>> * h...@confluent.io (650)924-2670
>> */
>>
>> On Fri, Oct 21, 2016 at 10:38 AM, Gourab Chowdhury <gourab....@gmail.com>
>> wrote:
>>
>> > I was testing a scenario of a server crash while the Kafka is still
>> running
>> > and passing messages. (just did a sudo reboot). By default, I am storing
>> > the logs at /tmp/kafka-log.
>> >
>> > So in case an EC2 server restart, am I going to loose the data if I
>> don't
>> > store kafka logs in an EBS drive?
>> >
>> > Also, I have another doubt, Does zookeeper stores Kafka info (viz.
>> topics,
>> > brokers list, consumer groups) at /tmp/zookeeper/version-2/log.x ?
>> >
>> > I am using python zookeeper client: *kazoo* and here we can find various
>> > Kafka info like '/brokers/topics', '/brokers/topics/topic/partitions' ,
>> > '/consumers' , '/consumers/consumer/offsets'. Where can I get this info
>> > directly?
>> >
>> > ~Gourab
>> >
>> > On Fri, Oct 21, 2016 at 8:13 PM, Ben Davison <ben.davi...@7digital.com>
>> > wrote:
>> >
>> > > Are you restarting or terminating the instance? If your terminating,
>> and
>> > > don't have an EBS drive attached you will lose all data on the drive.
>> > >
>> > > On Fri, Oct 21, 2016 at 2:09 PM, Gourab Chowdhury <
>> gourab....@gmail.com>
>> > > wrote:
>> > >
>> > > > I am running kafka in a EC2 server. While running if there is a
>> sudden
>> > > > restart of server, and then we restart the kafka broker, we can't
>> find
>> > > the
>> > > > details of all the topics.
>> > > >
>> > > > bin/kafka-topics.sh --describe --zookeeper <hostname:port>
>> > > > gives no result
>> > > >
>> > > > bin/kafka-console-producer.sh --broker-list <hostname:port> --topic
>> > test
>> > > > yields this particular error as follows:
>> > > >
>> > > > WARN Error while fetching metadata with correlation id n :
>> > > > {test=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.
>> > > NetworkClient)
>> > > >
>> > > > Is the details of topics from the zookeeper getting deleted while
>> > > restart?
>> > > > or this is something else?
>> > > >
>> > > > Also how can I cope with this in in real life production?
>> > > >
>> > > > Regards,
>> > > > Gourab Chowdhury,
>> > > >
>> > >
>> > > --
>> > >
>> > >
>> > > This email, including attachments, is private and confidential. If you
>> > have
>> > > received this email in error please notify the sender and delete it
>> from
>> > > your system. Emails are not secure and may contain viruses. No
>> liability
>> > > can be accepted for viruses that might be transferred by this email or
>> > any
>> > > attachment. Any unauthorised copying of this message or unauthorised
>> > > distribution and publication of the information contained herein are
>> > > prohibited.
>> > >
>> > > 7digital Limited. Registered office: 69 Wilson Street, London EC2A
>> 2BB.
>> > > Registered in England and Wales. Registered No. 04843573.
>> > >
>> >
>>
>
>
