[
https://issues.apache.org/jira/browse/KAFKA-8204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16816561#comment-16816561
]
John Roesler commented on KAFKA-8204:
-------------------------------------
Thanks [~mjsax]. Actually, in the code review, [~guozhang] asked some awkward
questions that made me re-evaluate the nature of the bug. Upon review, I think
it was introduced in
[https://github.com/apache/kafka/pull/4215|https://github.com/apache/kafka/pull/4215/files]
and therefore was introduced in 1.1 .
The details are here:
[https://github.com/apache/kafka/pull/6555#discussion_r274997288]
> Streams may flush state stores in the incorrect order
> -----------------------------------------------------
>
> Key: KAFKA-8204
> URL: https://issues.apache.org/jira/browse/KAFKA-8204
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 1.1.0, 1.1.1, 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.1.1
> Reporter: John Roesler
> Assignee: John Roesler
> Priority: Blocker
> Fix For: 2.1.2, 2.2.1
>
>
> Cached state stores may forward records during a flush call, so Streams
> should flush the stores in topological order. Otherwise, Streams may flush a
> downstream store before an upstream one, resulting in sink results being
> committed without the corresponding state changelog updates being committed.
> This behavior is partly responsible for the bug reported in KAFKA-7895 .
> The fix is simply to flush the stores in topological order, then when the
> upstream store forwards records to a downstream stateful processor, the
> corresponding state changes will be correctly flushed as well.
> An alternative would be to repeatedly call flush on all state stores until
> they report there is nothing left to flush, but this requires a public API
> change to enable state stores to report whether they need a flush or not.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
