We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part of the upgrade our clients upgraded the client libraries as well. After the upgrade we are facing following issue org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for OM-LOCAL-DC3-REQUISITION-SUBMISSION-PRD-E2-14: 30027 ms has passed since last append
My understanding is that Producer batches the send requests going to topic/partition per leader and sends it once based on following parameters ülinger.ms ß is time based batchingübatch.size ß is size based batching We had batch.size as 16K and linger.ms 0 and we expected producer to send requests right away even if the batch size is not met because linger.ms is zero. Our request.timeout.ms 30 seconds, looking at the error, it looks like requests are kept in the sender queue longer and when the Sender polling thread sees the requests are older, it is expiring them out. I have tried following multiple combinations and none seem to work linger.ms 0 and batch.size - < 16 klinger.ms 0 and batch.size - 0 k linger.ms 1000 and batch.size - < 0 Any clue on why linger.ms 0 or btach.size 0 is not taken into consideration ??? Thanks,Naidu Saladi
