When the producer have exhausted all retries on sending, the data will be dropped on the floor. One possible reason for this to happen is the leader failover taking long to let producer fails all 3 retries.
You may want to tune the following two configs on producers ( https://kafka.apache.org/documentation.html#producerconfigs) to see if this scenario can be solved: message.send.max.retries (default 3) retry.backoff.ms (default 100) Guozhang On Mon, Jan 13, 2014 at 6:34 AM, Francois Langelier < francois.langel...@mate1inc.com> wrote: > Yes, i have this message : ERROR Error in handling batch of 1 events > (kafka.producer.async.ProducerSendThread) > kafka.common.FailedToSendMessageException: Failed to send messages after 3 > tries. > > > > > On Sun, Jan 12, 2014 at 10:17 PM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > On the producer client log, did you see something like "failed to send > ... > > after .. retries"? > > > > Guozhang > > > > > > On Wed, Jan 8, 2014 at 11:44 AM, Francois Langelier < > > francois.langel...@mate1inc.com> wrote: > > > > > Thank you for your answers > > > > > > @Guozhang: I can't find the "ack value" in my console... > > > > > > @Marc: I'm testing some stuff on 0.8 before migrating 0.7 to 0.8, > that's > > > why I'm killing it instead of controlled shutdown. > > > > > > @Jun: I create it using this command : *bin/kafka-create-topic.sh > > > --zookeeper localhost:2181 --replica 3 --partition 1 --topic > > > my-replicated-topic* > > > And here is the output of the list topic command: > > > > > > *topic: my-replicated-topic partition: 0 leader: 0 replicas: 0,2,1 isr: > > > 0,1,2* > > > > > > > > > I continued investigating on my own and here some others informations : > > > > > > - I use the *bin/kafka-server-start.sh *script to start the servers > > and > > > I use the consumers and producers scripts in the bin/ > > > - When I kill the leader, for about 5 seconds I receive java > exception > > > error in my consumers consoles and if i try to send message through > > the > > > producers console, I also have java exception. Furthermore, all the > > > messages I send during that time through the producers never reach > the > > > consumers, even after the "5 seconds" > > > - When the "5 seconds" is over, the link is "repaired" and all the > > news > > > messages reach their destinations (not those within the "5 seconds") > > > > > > > > > > > > On Wed, Dec 18, 2013 at 12:16 AM, Jun Rao <jun...@gmail.com> wrote: > > > > > > > What's the replication factor of the topic? Is it larger than 1? You > > can > > > > find out using the list topic command. > > > > > > > > Thanks, > > > > > > > > Jun > > > > > > > > > > > > On Tue, Dec 17, 2013 at 2:39 PM, Francois Langelier < > > > > francois.langel...@mate1inc.com> wrote: > > > > > > > > > Hi, > > > > > > > > > > I installed zookeeper and kafka 8.0 following the quick start ( > > > > > https://kafka.apache.org/documentation.html#quickstart) and when i > > try > > > > to > > > > > kill my leader, i got a lot of exception in my producer and > consumer > > > > > consoles. > > > > > > > > > > Then, after the exceptions stop printing, some of the messages I > > > produce > > > > in > > > > > my console don't print in my consumer console... > > > > > > > > > > The exception I get is "java.net.ConnectException : Connection > > > refused". > > > > > > > > > > Did someone already had this problem? > > > > > > > > > > Thank you! > > > > > > > > > > PS: I have 3 brokers running on my system. > > > > > > > > > > > > > > > > > > > > -- > > -- Guozhang > > > -- -- Guozhang
