Eduwer Camacaro created KAFKA-18032:
---------------------------------------
Summary: ProducerFencedExceptions should wipe out the state store
only for the tasks that are related to the failed transaction
Key: KAFKA-18032
URL: https://issues.apache.org/jira/browse/KAFKA-18032
Project: Kafka
Issue Type: Improvement
Components: streams
Affects Versions: 3.9.0
Reporter: Eduwer Camacaro
Assignee: Eduwer Camacaro
Attachments: app1.log.tgz
A ProducerFencedException is thrown when the StreamThread reaches the
`transaction.timeout.ms` limit and EOS is enabled, causing dirty closes on the
running StreamTask in order to protect Exactly Once Semantics. Even if the
failed transaction contained records associated with a single stream task, it
wipes out all associated StreamTask states. In the inflight transaction, the
StreamsProducer should keep track of the associated task and only close 'dirty'
these tasks when the producer is fenced.
Steps to reproduce:
# Deploys a Kafka Streams instance that uses multiple tasks in a single
StreamThread
# Set the `transaction.timeout.ms` small enough to cause a timeout during
processing.
# Send load to your input topics.
Check how many of your StreamTasks are currently closed dirty.
{code:java}
stream-thread [app1-StreamThread-1] task [0_0] Closed dirty{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
