Either should be fine but is all dependent on your environment and how you want to operate your cluster in the long run i.e. if you have multiple volumes then you need to have them each in the log.dirs separated by comma, etc. If there is no good reason to have the broker.id in the directory they you should probably have it be consistent on each node.
/******************************************* Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> ********************************************/ On Fri, Jan 2, 2015 at 5:26 PM, Sa Li <sal...@gmail.com> wrote: > One more question, when I set the log.dirs in different nodes in the > cluster, should I set them different name, say kafka-logs-1 which > associated with broker id, or I can set the same directory name, like > /var/log/kafka for every node (assume one broker in each server). > > thanks > > > On Fri, Jan 2, 2015 at 2:20 PM, Sa Li <sal...@gmail.com> wrote: > > > Thanks a lot! > > > > > > On Fri, Jan 2, 2015 at 12:15 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > >> Nice catch Joe--several people have complained about this as a problem > and > >> we were a bit mystified as to what kind of bug could lead to all their > >> logs > >> getting deleted and re-replicated when they bounced the server. We > assumed > >> "bounced" meant restarted the app, but I think likely what is happening > is > >> what you describe--the logs were in /tmp and bouncing the server meant > >> restarting. > >> > >> -Jay > >> > >> On Fri, Jan 2, 2015 at 11:02 AM, Joe Stein <joe.st...@stealth.ly> > wrote: > >> > >> > That is because your logs are in /tmp which you can change by > >> > setting log.dirs to something else. > >> > > >> > /******************************************* > >> > Joe Stein > >> > Founder, Principal Consultant > >> > Big Data Open Source Security LLC > >> > http://www.stealth.ly > >> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > >> > ********************************************/ > >> > > >> > On Fri, Jan 2, 2015 at 1:58 PM, Sa Li <sal...@gmail.com> wrote: > >> > > >> > > Hi, All > >> > > > >> > > I've just notice one thing, when I am experiencing some errors in > >> Kafka > >> > > servers, I reboot the dev servers (not a good way), after reboot, I > >> get > >> > > into zkCli, I can see all the topics still exist. But when I get > into > >> > kafka > >> > > log directory, I found all data gone, see > >> > > > >> > > root@DO-mq-dev:/tmp/kafka-logs-1/ui_test_topic_4-0# ll > >> > > total 8 > >> > > drwxr-xr-x 2 root root 4096 Jan 2 09:39 ./ > >> > > drwxr-xr-x 46 root root 4096 Jan 2 10:46 ../ > >> > > -rw-r--r-- 1 root root 10485760 Jan 2 09:39 > >> 00000000000000000000.index > >> > > -rw-r--r-- 1 root root 0 Jan 2 09:39 > 00000000000000000000.log > >> > > > >> > > I wonder, if for some reasons, the server down, and restart it, all > >> the > >> > > data in hard drive will be gone? > >> > > > >> > > thanks > >> > > > >> > > -- > >> > > > >> > > Alec Li > >> > > > >> > > >> > > > > > > > > -- > > > > Alec Li > > > > > > -- > > Alec Li >
