Re: Failed on publish to Kafka

2018-05-23 Thread Omer Litov
following? 1) reduce buffer size to exactly 2x of your batch size ? 2) back up and save you old properties file. 3) create a dummy topic 4) produce and consume messages using the new config. Let us know. On Wed, 23 May 2018 at 07:02, Omer Li

Re: Failed on publish to Kafka

2018-05-22 Thread Omer Litov
is working :) ? From the provided exception stack, it seems as if you are waiting to batch a lot of request but they are getting timed out for some reasons. How have you upgraded the cluster to 1.0.1, and have you changed any config? Regards, On 22

Failed on publish to Kafka

2018-05-22 Thread Omer Litov
Hi, I have 4 producers, writing to a kafka cluster with 12 partitions. Periodically (about once per hour or two), we get exceptions while trying to send messages to kafka. The exceptions are “Expiring 12 record(s) for -: 49193 ms has passed since batch creation plus linger time”. We get that for

No retries when a broker is unavailable

2018-04-28 Thread Omer Litov
Hi, I noticed that when a broker is unavailable, the producer does not attempt any retries. When diving into the source code, I noticed that since the node is considered “not ready”, the request is never sent in the first place, and the batch expires without any retry. Steps to reproduce: -