[
https://issues.apache.org/jira/browse/KAFKA-19128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alieh Saeedi updated KAFKA-19128:
---------------------------------
Description:
Kafka Streams calls `prepareCommit()` in `Taskmanager#closeTaskDirty()`. The
initial purpose of this call has only been to flush the case if necessary
before suspending and closing the topology. But, calling `prepareCommit()`
results into calling other methods as well, which compute the next offsets for
the partition. However, the dirty task must not get committed and therefore,
getting offsets should not be needed as well.
Once, we encountered the following exception with AK 3.9:
{code:java}
Streams Soak encountered IllegalStateException: You can only check the position
for partitions assigned to this consumer.{code}
However, due to changes and improvements in AK 4.0, this exception can not
happen any more with AK 4.0+. BTW, the issue must get fixed. Proposed fixes are
as follows:
1. Introduction of a flag {{prepareCommit(boolean clean)}} to determine clean
or unclean shutdown, avoiding unnecessary offset checking during dirty
shutdowns.
2. Separating `flush` and `prepareCommit`. When closing dirty or in other
required cases, only call `flush()`.
{{}}
was:
Kafka Streams calls `prepareCommit()` in `Taskmanager#closeTaskDirty()`. The
initial purpose of this call has been only to flush the case if necessary
before suspending and closing the topology. But, calling `prepareCommit()`
results into calling other methods as well, which compute the next offsets for
the partition. However, the dirty task must not get committed and therefore,
getting offsets should not be needed as well.
Once, we encountered the following exception in AK 3.9:
{code:java}
Streams Soak encountered IllegalStateException: You can only check the position
for partitions assigned to this consumer.{code}
However, due to changes and improvements in AK 4.0, this exception can not
happen any more. BTW, the issue must get fixed. Proposed fixes are as follows:
1. Introduction of a flag {{prepareCommit(boolean clean)}} to determine clean
or unclean shutdown, avoiding unnecessary offset checking during dirty
shutdowns.
2. Separating `flush` and `prepareCommit`. When closing dirty or in other
required cases, only call `flush()`.
{{}}
> Kafka Streams should not get offsets when close dirty
> -----------------------------------------------------
>
> Key: KAFKA-19128
> URL: https://issues.apache.org/jira/browse/KAFKA-19128
> Project: Kafka
> Issue Type: Bug
> Reporter: Alieh Saeedi
> Priority: Minor
>
> Kafka Streams calls `prepareCommit()` in `Taskmanager#closeTaskDirty()`. The
> initial purpose of this call has only been to flush the case if necessary
> before suspending and closing the topology. But, calling `prepareCommit()`
> results into calling other methods as well, which compute the next offsets
> for the partition. However, the dirty task must not get committed and
> therefore, getting offsets should not be needed as well.
> Once, we encountered the following exception with AK 3.9:
> {code:java}
> Streams Soak encountered IllegalStateException: You can only check the
> position for partitions assigned to this consumer.{code}
> However, due to changes and improvements in AK 4.0, this exception can not
> happen any more with AK 4.0+. BTW, the issue must get fixed. Proposed fixes
> are as follows:
> 1. Introduction of a flag {{prepareCommit(boolean clean)}} to determine clean
> or unclean shutdown, avoiding unnecessary offset checking during dirty
> shutdowns.
> 2. Separating `flush` and `prepareCommit`. When closing dirty or in other
> required cases, only call `flush()`.
> {{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
