Matthias J. Sax created KAFKA-19669:
---------------------------------------
Summary: Improve repartition optimization
Key: KAFKA-19669
URL: https://issues.apache.org/jira/browse/KAFKA-19669
Project: Kafka
Issue Type: Bug
Components: streams
Reporter: Matthias J. Sax
Kafka Streams supports "merge.repartition.topic" optimization. This
optimization tries to push repartition topics further upstream, to be able to
merge multiple repartition topics from different downstream branches into a
single repartition topic.
In the current implementation, the optimization stops to push repartition
topics upstream, if the parent node is a value-changing operation. We cannot
push-up further, because the data type might change, and thus we won't have the
correct serde at hand.
However, we could actually also inherit the correct serde from further
upstream. Thus, we could actually push a repartition-step upstream (and at the
same time, switch the serdes), if we can find the right serde before the
value-changing operations further upstream.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
