[
https://issues.apache.org/jira/browse/KAFKA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16899811#comment-16899811
]
Luke Stephenson commented on KAFKA-8325:
----------------------------------------
I suspect the trigger for this is having a large `batch.size` on the producer.
I had configured:
"batch.size" = 1000000
I was initially under the impression that this was just used to influence
throughput by causing the producer to wait for more bytes. However, when the
batch is "split and retried", it also uses the `batch.size` value for splitting.
Curious if the other people who raised this bug also had a high value for this
setting. I've reduced the value by half and I've stopped seeing the Exception
with similar load.
> Remove from the incomplete set failed. This should be impossible
> ----------------------------------------------------------------
>
> Key: KAFKA-8325
> URL: https://issues.apache.org/jira/browse/KAFKA-8325
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 2.1.0, 2.3.0
> Reporter: Mattia Barbon
> Assignee: Bob Barrett
> Priority: Major
>
> I got this error when using the Kafka producer. So far it happened twice,
> with an interval of about 1 week.
> {{ERROR [2019-05-05 08:43:07,505]
> org.apache.kafka.clients.producer.internals.Sender: [Producer
> clientId=<redacted>, transactionalId=<redacted>] Uncaught error in kafka
> producer I/O thread:}}
> {{ ! java.lang.IllegalStateException: Remove from the incomplete set failed.
> This should be impossible.}}
> {{ ! at
> org.apache.kafka.clients.producer.internals.IncompleteBatches.remove(IncompleteBatches.java:44)}}
> {{ ! at
> org.apache.kafka.clients.producer.internals.RecordAccumulator.deallocate(RecordAccumulator.java:645)}}
> {{ ! at
> org.apache.kafka.clients.producer.internals.Sender.failBatch(Sender.java:717)}}
> {{ ! at
> org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:365)}}
> {{ ! at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:308)}}
> {{ ! at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:233)}}
> {{ ! at java.lang.Thread.run(Thread.java:748)}}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
