Re: Kafka unable to find topic details after sudden EC2 server restart

2016-10-24 Thread Hans Jespersen
You probably didn't setup permissions properly on the /data directory you created. You might want to do some free online training to learn some more Linux if you are planning to become a Kafka administrator. https://en.wikipedia.org/wiki/Chmod https://www.edx.org/course/introduction-linux-linuxfoun

Re: Kafka unable to find topic details after sudden EC2 server restart

2016-10-24 Thread Gourab Chowdhury
No issues, I have solved the problem by changing the permission of /data Thanks, Gourab On Mon, Oct 24, 2016 at 2:20 PM, Gourab Chowdhury wrote: > Thanks for the reply, I tried changing the data directory as follows:- > dataDir=/data/zookeeper > > I also create a /data but still I am getting th

Re: Kafka unable to find topic details after sudden EC2 server restart

2016-10-24 Thread Gourab Chowdhury
Thanks for the reply, I tried changing the data directory as follows:- dataDir=/data/zookeeper I also create a /data but still I am getting the following error. ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server. ZooKeeperServerMain) java.io.IOException: Unable to create

Re: Kafka unable to find topic details after sudden EC2 server restart

2016-10-21 Thread Hans Jespersen
You are going to lose everything you store in /tmp. In a production system you never configure Kafka or zookeeper to store critical data in /tmp. This has nothing to do with AWS or EBS it is just standard Linux than everything under /tmp is deleted when Linux reboots. -hans /** * Hans Jespersen,

Re: Kafka unable to find topic details after sudden EC2 server restart

2016-10-21 Thread Gourab Chowdhury
I was testing a scenario of a server crash while the Kafka is still running and passing messages. (just did a sudo reboot). By default, I am storing the logs at /tmp/kafka-log. So in case an EC2 server restart, am I going to loose the data if I don't store kafka logs in an EBS drive? Also, I have

Re: Kafka unable to find topic details after sudden EC2 server restart

2016-10-21 Thread Ben Davison
Are you restarting or terminating the instance? If your terminating, and don't have an EBS drive attached you will lose all data on the drive. On Fri, Oct 21, 2016 at 2:09 PM, Gourab Chowdhury wrote: > I am running kafka in a EC2 server. While running if there is a sudden > restart of server, a

Kafka unable to find topic details after sudden EC2 server restart

2016-10-21 Thread Gourab Chowdhury
I am running kafka in a EC2 server. While running if there is a sudden restart of server, and then we restart the kafka broker, we can't find the details of all the topics. bin/kafka-topics.sh --describe --zookeeper gives no result bin/kafka-console-producer.sh --broker-list --topic test yield