[
https://issues.apache.org/jira/browse/KAFKA-9674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17053696#comment-17053696
]
ASF GitHub Bot commented on KAFKA-9674:
---------------------------------------
abbccdda commented on pull request #8242: KAFKA-9674: corruption should also
cleanup producer and recreate
URL: https://github.com/apache/kafka/pull/8242
The task producer cleanup doesn't involve handling of task corruption.
Adding recreation of task producer to avoid reusing a fatal state producer in
next cycle.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Task corruption should also close the producer if necessary
> -----------------------------------------------------------
>
> Key: KAFKA-9674
> URL: https://issues.apache.org/jira/browse/KAFKA-9674
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Boyang Chen
> Assignee: Boyang Chen
> Priority: Major
>
> The task revive call only transits the task to CREATED mode. It should handle
> the recreation of task producer as well.
> Sequence is like:
> # Task hits out of range exception and throws CorruptedException
> # Task producer closed along with the task
> # Task revived and rebalance triggered
> # Task was assigned back to the same thread
> # Trying to use task producer will throw as it has already been closed.
> The full log:
>
> [2020-03-03T21:56:29-08:00]
> (streams-soak-trunk-eos_soak_i-0eaa3f3a6a197f876_streamslog) [2020-03-04
> 05:56:29,070] WARN
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3]
> stream-thread
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3]
> Encountered org.apache.kafka.clients.consumer.OffsetOutOfRangeException
> fetching records from restore consumer for partitions
> [stream-soak-test-KSTREAM-AGGREGATE-STATE-STORE-0000000019-changelog-0], it
> is likely that the consumer's position has fallen out of the topic partition
> offset range because the topic was truncated or compacted on the broker,
> marking the corresponding tasks as corrupted and re-initializing it later.
> (org.apache.kafka.streams.processor.internals.StoreChangelogReader)
> [2020-03-03T21:56:29-08:00]
> (streams-soak-trunk-eos_soak_i-0eaa3f3a6a197f876_streamslog) [2020-03-04
> 05:56:29,071] WARN
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3]
> stream-thread
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3]
> Detected the states of tasks
> \{1_0=[stream-soak-test-KSTREAM-AGGREGATE-STATE-STORE-0000000019-changelog-0]}
> are corrupted. Will close the task as dirty and re-create and bootstrap from
> scratch. (org.apache.kafka.streams.processor.internals.StreamThread)
>
> [2020-03-03T21:56:30-08:00]
> (streams-soak-trunk-eos_soak_i-0eaa3f3a6a197f876_streamslog) [2020-03-04
> 05:56:30,010] INFO
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3]
> [Producer
> clientId=stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3-1_0-producer,
> transactionalId=stream-soak-test-1_0] Closing the Kafka producer with
> timeoutMillis = 9223372036854775807 ms.
> (org.apache.kafka.clients.producer.KafkaProducer)
>
>
> [2020-03-03T21:56:30-08:00]
> (streams-soak-trunk-eos_soak_i-0eaa3f3a6a197f876_streamslog) [2020-03-04
> 05:56:30,017] INFO
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3]
> stream-thread
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3] task
> [1_0] Closed clean (org.apache.kafka.streams.processor.internals.StreamTask)
>
>
> [2020-03-03T21:56:22-08:00]
> (streams-soak-trunk-eos_soak_i-0eaa3f3a6a197f876_streamslog) [2020-03-04
> 05:56:22,827] INFO
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3]
> [Producer
> clientId=stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3-1_0-producer,
> transactionalId=stream-soak-test-1_0] Closing the Kafka producer with
> timeoutMillis = 9223372036854775807 ms.
> (org.apache.kafka.clients.producer.KafkaProducer)
> [2020-03-03T21:56:22-08:00]
> (streams-soak-trunk-eos_soak_i-0eaa3f3a6a197f876_streamslog) [2020-03-04
> 05:56:22,829] INFO
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3]
> stream-thread
> [stream-soak-test-93df69e6-1d85-4b6a-81a1-c6d554693e3f-StreamThread-3] task
> [1_0] Closed dirty (org.apache.kafka.streams.processor.internals.StreamTask)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
