Yup definitely i would like to try that If controlled.shutdown.enable property works in case of kill -9.
I hope that this option will be perfect. Thanks for quick response, really appreciate it. On Wed, Dec 18, 2013 at 10:52 PM, Joe Stein <joe.st...@stealth.ly> wrote: > Wouldn't you want to set the controlled.shutdown.enable to true so the > broker would do this for you before ending itself? > > /******************************************* > Joe Stein > Founder, Principal Consultant > Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ********************************************/ > > > On Wed, Dec 18, 2013 at 11:36 AM, pushkar priyadarshi < > priyadarshi.push...@gmail.com> wrote: > > > my doubt was they are dropping off at producer level only.so suggested > > playing with paramaters like retries and backoff.ms and also with > > refreshinterval on producer side. > > > > Regards, > > Pushkar > > > > > > On Wed, Dec 18, 2013 at 10:01 PM, Guozhang Wang <wangg...@gmail.com> > > wrote: > > > > > Hanish, > > > > > > Did you "kill -9" one of the brokers only or bouncing them iteratively? > > > > > > Guozhang > > > > > > > > > On Wed, Dec 18, 2013 at 8:02 AM, Joe Stein <joe.st...@stealth.ly> > wrote: > > > > > > > How many replicas do you have? > > > > > > > > > > > > On Wed, Dec 18, 2013 at 8:57 AM, Hanish Bansal < > > > > hanish.bansal.agar...@gmail.com> wrote: > > > > > > > > > Hi pushkar, > > > > > > > > > > I tried with configuring "message.send.max.retries" to 10. Default > > > value > > > > > for this is 3. > > > > > > > > > > But still facing data loss. > > > > > > > > > > > > > > > On Wed, Dec 18, 2013 at 12:44 PM, pushkar priyadarshi < > > > > > priyadarshi.push...@gmail.com> wrote: > > > > > > > > > > > You can try setting a higher value for "message.send.max.retries" > > in > > > > > > producer config. > > > > > > > > > > > > Regards, > > > > > > Pushkar > > > > > > > > > > > > > > > > > > On Wed, Dec 18, 2013 at 5:34 PM, Hanish Bansal < > > > > > > hanish.bansal.agar...@gmail.com> wrote: > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > We are having kafka cluster of 2 nodes. (using 0.8.0 final > > release) > > > > > > > Replication Factor: 2 > > > > > > > Number of partitions: 2 > > > > > > > > > > > > > > > > > > > > > I have configured request.required.acks in producer > configuration > > > to > > > > > -1. > > > > > > > > > > > > > > As mentioned in documentation > > > > > > > http://kafka.apache.org/documentation.html#producerconfigs, > > > setting > > > > > this > > > > > > > value to -1 provides guarantee that no messages will be lost. > > > > > > > > > > > > > > I am getting below behaviour: > > > > > > > > > > > > > > If kafka is running as foreground process and i am shutting > down > > > the > > > > > > kafka > > > > > > > leader node using "ctrl+C" then no data is lost. > > > > > > > > > > > > > > But if i abnormally terminate the kafka using "kill -9 <pid>" > > then > > > > > still > > > > > > > facing data loss even after configuring request.required.acks > to > > > -1. > > > > > > > > > > > > > > Any suggestions? > > > > > > > -- > > > > > > > *Thanks & Regards* > > > > > > > *Hanish Bansal* > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > *Thanks & Regards* > > > > > *Hanish Bansal* > > > > > > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > -- *Thanks & Regards* *Hanish Bansal*
