If you use `merge()` merge preserve the (relative) order for each input, but the result will contain records for both inputs interleaved.
For example: topicA-p0: A B C topicB-p0: X Y Z In the output KStream, A will be before B, and B will be before C. Similar for X, Y, Z. How A,B,C and X,Y,Z interleave in the output stream, depends on the timestamps of the records. Kafka Streams processed records in timestamp order and thus alternates between both input. Example with timestamps: topicA-p0: A(3) B(7) C(10) topicB-p0: X(2) Y(8) Z(9) The output KStream would be exactly: X(2) A(3) B(7) Y(8) Z(9) C(10) As mentioned in a previous answer to another email, the only thing that could "mess up" the order of the output stream is if one input is empty and the other is processed and later the first empty input get data that could be out-of-order now. Using `max.task.idle.ms` you can pause processing for a certain amount of time though if one input becomes empty to work against this potential issue. -Matthias On 7/13/20 1:18 PM, Dumitru-Nicolae Marasoui wrote: > Hi, > I would like to understand the ordering guarantees if any at the merge > operator level. I think unless I am writing into a topic, there can be no > ordering guarantees, is that so? > > I would normally do a key transformation, merge, write to an output topic, > and i know the partition (key) ordering guarantees, and have a second > kafka-streams from that topic to the aggregation and to the final topic. > > Is there a way to make kafka-streams do this behind the scenes? I do not > see a way to guarantee the ordering outside of an intermediate topic, > implicit or explicit? > Thank you, >
signature.asc
Description: OpenPGP digital signature
