Hi,
Thanks for the quick response.
What I meant is, that had those exceptions on both versions (0.10.1 and 1.0.1),
so upgrading didn't solve the problem. And the same for increasing the cluster
size.
Here are the properties we set:
"key.serializer" = "org.apache.kafka.common.serialization.StringSerializer"
"value.serializer" = "org.apache.kafka.common.serialization.ByteArraySerializer"
"acks" = "all"
"retries" = 5
"buffer.memory" = 33554432
"batch.size" = 16384
"request.timeout.ms" = 50000
thanks,
Omer
On 22/05/2018, 8:14 PM, "M. Manna" <manme...@gmail.com> wrote:
Hi,
You said "On all of those setting there was no change in the described
behavior" - this is slightly confusing. Could you please clarify this? If
there is no change, that means everything is working :) ?
From the provided exception stack, it seems as if you are waiting to batch
a lot of request but they are getting timed out for some reasons.
How have you upgraded the cluster to 1.0.1, and have you changed any config?
Regards,
On 22 May 2018 at 16:32, Omer Litov <omer.li...@imperva.com> wrote:
> Hi,
> I have 4 producers, writing to a kafka cluster with 12 partitions.
> Periodically (about once per hour or two), we get exceptions while trying
> to send messages to kafka.
> The exceptions are “Expiring 12 record(s) for <topic_name>-<partition>:
> 49193 ms has passed since batch creation plus linger time”.
> We get that for all partitions (some more than others).
> When those exceptions arrive, they usually come a few hundred together in
> under a second, and then stops.
> We don’t see any indications for retries (even though they are
configured).
>
> We first used a cluster of 3 brokers with version 0.10.1.
> Then upgraded to version 1.0.1 (with 3 brokers), and eventually added
> additional 2 brokers.
> On all of those setting there was no change in the described behavior.
>
>
> Would appreciate any lead on investigation the issue.
>
>
> Thanks,
> Omer
> -------------------------------------------
> NOTICE:
> This email and all attachments are confidential, may be proprietary, and
> may be privileged or otherwise protected from disclosure. They are
intended
> solely for the individual or entity to whom the email is addressed.
> However, mistakes sometimes happen in addressing emails. If you believe
> that you are not an intended recipient, please stop reading immediately.
Do
> not copy, forward, or rely on the contents in any way. Notify the sender
> and/or Imperva, Inc. by telephone at +1 (650) 832-6006 and then delete or
> destroy any copy of this email and its attachments. The sender reserves
and
> asserts all rights to confidentiality, as well as any privileges that may
> apply. Any disclosure, copying, distribution or action taken or omitted to
> be taken by an unintended recipient in reliance on this message is
> prohibited and may be unlawful.
> Please consider the environment before printing this email.
>
