Re: Killing broker leader

Hello Hanish, Did you see "failed to send messages after 10 tries" in your producer log? Guozhang On Wed, Jan 15, 2014 at 8:38 AM, Hanish Bansal < hanish.bansal.agar...@gmail.com> wrote: > Hi Francois, > > Probably Kafka-1193 is not due to any misconfiguration, there may be > something else is

Re: Killing broker leader

Hi Francois, Probably Kafka-1193 is not due to any misconfiguration, there may be something else is missing. I also tried both (max retries 10 and producer acks -1) together, that was also causing data loss. On Wed, Jan 15, 2014 at 9:40 PM, François Langelier wrote: > Yeah, that's what i found.

Re: Killing broker leader

Yeah, that's what i found... sorry about that... Well, i tested it out! At first, my command was : bin/kafka-console-producer.sh --broker-list localhost:9092 --topic my-replicated-topic --message-send-max-retries 10 --retry-backoff-ms 1000 Then i noticed that only the second message i sent got

Re: Killing broker leader

Those are actually producer side configs. Thanks, Jun On Wed, Jan 15, 2014 at 6:51 AM, François Langelier wrote: > Nope, it's the "3 tries" messages... maybe I did something wrong... > > i put > > message.send.max.retries=10 > retry.backoff.ms=1000 > > in my server.properties of each broker >

Re: Killing broker leader

Nope, it's the "3 tries" messages... maybe I did something wrong... i put message.send.max.retries=10 retry.backoff.ms=1000 in my server.properties of each broker I'm checking it right now! François Langelier Étudiant en génie Logiciel - École de Technologie Supérieure

Re: Killing broker leader

Hi Francois, just a quick question, when you set the number of retries to 10, does its log still have "ailed to send messages after 10 tries." entry? On Tue, Jan 14, 2014 at 11:41 AM, Francois Langelier < francois.langel...@mate1inc.com> wrote: > Of course! As soon as it will close, i'll try! >

Re: Killing broker leader

Of course! As soon as it will close, i'll try! On Tue, Jan 14, 2014 at 2:29 PM, Guozhang Wang wrote: > Hanish and Francois, > > The current patch of 1193 still have something missing and I am currently > working on it to be closed soon. Could you retry the scenario after it is > checked in? > >

Re: Killing broker leader

Hanish and Francois, The current patch of 1193 still have something missing and I am currently working on it to be closed soon. Could you retry the scenario after it is checked in? Guozhang On Tue, Jan 14, 2014 at 8:03 AM, Francois Langelier < francois.langel...@mate1inc.com> wrote: > @Guozhan

Re: Killing broker leader

@Guozhang Wang: I set the max retries to 10 and the backoof at 1000 ms but the bug still there and some messages don't reach my consumers... @Hanish : Yes, it looks like we have the same issue! On Mon, Jan 13, 2014 at 9:43 PM, Hanish Bansal < hanish.bansal.agar...@gmail.com> wrote: > I am not

Re: Killing broker leader

I am not sure but this may be same scenario as described in https://issues.apache.org/jira/browse/KAFKA-1193 On Tue, Jan 14, 2014 at 2:36 AM, Guozhang Wang wrote: > When the producer have exhausted all retries on sending, the data will be > dropped on the floor. One possible reason for this to

Re: Killing broker leader

When the producer have exhausted all retries on sending, the data will be dropped on the floor. One possible reason for this to happen is the leader failover taking long to let producer fails all 3 retries. You may want to tune the following two configs on producers ( https://kafka.apache.org/docu

Re: Killing broker leader

Yes, i have this message : ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread) kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries. On Sun, Jan 12, 2014 at 10:17 PM, Guozhang Wang wrote: > On the producer client log, did you see so

Re: Killing broker leader

On the producer client log, did you see something like "failed to send ... after .. retries"? Guozhang On Wed, Jan 8, 2014 at 11:44 AM, Francois Langelier < francois.langel...@mate1inc.com> wrote: > Thank you for your answers > > @Guozhang: I can't find the "ack value" in my console... > > @Mar

Re: Killing broker leader

Thank you for your answers @Guozhang: I can't find the "ack value" in my console... @Marc: I'm testing some stuff on 0.8 before migrating 0.7 to 0.8, that's why I'm killing it instead of controlled shutdown. @Jun: I create it using this command : *bin/kafka-create-topic.sh --zookeeper localhost:

Re: Killing broker leader

What's the replication factor of the topic? Is it larger than 1? You can find out using the list topic command. Thanks, Jun On Tue, Dec 17, 2013 at 2:39 PM, Francois Langelier < francois.langel...@mate1inc.com> wrote: > Hi, > > I installed zookeeper and kafka 8.0 following the quick start ( >

Re: Killing broker leader

It is worth mentioning you can reduce the likelyhood of loosing message by running the controlled shutdown before killing the broker. https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-Whatiscontrolledshutdown? The connection refused is a bit surprising though. T

Re: Killing broker leader

Hello Francois, What is the producer ack value in your console producer? If it is equal to 1 then when a leader is down it is possible to lose data, and hence not consumed by the consumer. Guozhang On Tue, Dec 17, 2013 at 2:39 PM, Francois Langelier < francois.langel...@mate1inc.com> wrote: >

Killing broker leader

Hi, I installed zookeeper and kafka 8.0 following the quick start ( https://kafka.apache.org/documentation.html#quickstart) and when i try to kill my leader, i got a lot of exception in my producer and consumer consoles. Then, after the exceptions stop printing, some of the messages I produce in