Re: kafka logs gone after reboot the server

2015-01-02 Thread Joe Stein
Either should be fine but is all dependent on your environment and how you want to operate your cluster in the long run i.e. if you have multiple volumes then you need to have them each in the log.dirs separated by comma, etc. If there is no good reason to have the broker.id in the directory they y

Re: kafka logs gone after reboot the server

2015-01-02 Thread Sa Li
One more question, when I set the log.dirs in different nodes in the cluster, should I set them different name, say kafka-logs-1 which associated with broker id, or I can set the same directory name, like /var/log/kafka for every node (assume one broker in each server). thanks On Fri, Jan 2, 201

Re: kafka logs gone after reboot the server

2015-01-02 Thread Sa Li
Thanks a lot! On Fri, Jan 2, 2015 at 12:15 PM, Jay Kreps wrote: > Nice catch Joe--several people have complained about this as a problem and > we were a bit mystified as to what kind of bug could lead to all their logs > getting deleted and re-replicated when they bounced the server. We assumed

Re: kafka logs gone after reboot the server

2015-01-02 Thread Jay Kreps
Nice catch Joe--several people have complained about this as a problem and we were a bit mystified as to what kind of bug could lead to all their logs getting deleted and re-replicated when they bounced the server. We assumed "bounced" meant restarted the app, but I think likely what is happening i

Re: kafka logs gone after reboot the server

2015-01-02 Thread Joe Stein
That is because your logs are in /tmp which you can change by setting log.dirs to something else. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop