[
https://issues.apache.org/jira/browse/KAFKA-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ulrik updated KAFKA-13638:
--------------------------
Description:
I have a topology where I stream messages from an input topic, transform the
message to multiple messages (via context.forward), and then store those
messages in a KTable.
Since upgrading from kafka-streams 2.8.1 to 3.1.0 I have noticed that my tests
take significantly longer time to run.
I have attached a test class to demonstrate my scenario. When running this test
with kafka-streams versions 2.8.1 and 3.1.0 I came up with the following
numbers:
*Version 2.8.1*
* one input message and one output message: 541 ms
* 8 input message and 30 output message per input message (240 output messages
in total): 919 ms
*Version 3.1.0*
* one input message and one output message: 908 ms
* 8 input message and 30 output message per input message (240 output messages
in total): 6 sec 94 ms
Even when the transformer just transforms and forwards one input message to one
output message, the test takes approx. 400 ms longer to run.
When transforming 8 input messages to 240 output messages it takes approx 5
seconds longer.
was:
I have a topology where I stream messages from an input topic, transform the
message to multiple messages (via context.forward), and then store those
messages in a KTable.
Since upgrading from kafka-streams 2.8.1 to 3.1.0 I have noticed that my tests
takes significantly longer time to run.
I have attached a test class to demonstrate my scenario. When running this test
with kafka-streams versions 2.8.1 and 3.1.0 I came up with the following
numbers:
*Version 2.8.1*
* one input message and one output message: 541 ms
* 8 input message and 30 output message per input message (240 output messages
in total): 919 ms
*Version 3.1.0*
* one input message and one output message: 908 ms
* 8 input message and 30 output message per input message (240 output messages
in total): 6 sec 94 ms
Even when the transformer just transforms and forwards one input message to one
output message, the test takes approx. 400 ms longer to run.
When transforming 8 input messages to 240 output messages it takes approx 5
seconds longer.
> Slow KTable update when forwarding multiple values from transformer
> -------------------------------------------------------------------
>
> Key: KAFKA-13638
> URL: https://issues.apache.org/jira/browse/KAFKA-13638
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 3.1.0, 3.0.0
> Reporter: Ulrik
> Priority: Major
> Attachments: KafkaTest.java
>
>
> I have a topology where I stream messages from an input topic, transform the
> message to multiple messages (via context.forward), and then store those
> messages in a KTable.
> Since upgrading from kafka-streams 2.8.1 to 3.1.0 I have noticed that my
> tests take significantly longer time to run.
>
> I have attached a test class to demonstrate my scenario. When running this
> test with kafka-streams versions 2.8.1 and 3.1.0 I came up with the following
> numbers:
>
> *Version 2.8.1*
> * one input message and one output message: 541 ms
> * 8 input message and 30 output message per input message (240 output
> messages in total): 919 ms
>
> *Version 3.1.0*
> * one input message and one output message: 908 ms
> * 8 input message and 30 output message per input message (240 output
> messages in total): 6 sec 94 ms
>
> Even when the transformer just transforms and forwards one input message to
> one output message, the test takes approx. 400 ms longer to run.
> When transforming 8 input messages to 240 output messages it takes approx 5
> seconds longer.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
