Matthias J. Sax created KAFKA-13602:
---------------------------------------
Summary: Allow to broadcast a result record
Key: KAFKA-13602
URL: https://issues.apache.org/jira/browse/KAFKA-13602
Project: Kafka
Issue Type: New Feature
Components: streams
Reporter: Matthias J. Sax
>From time to time, users ask how they can send a record to more than one
>partition in a sink topic. Currently, this is only possible by replicate the
>message N times before the sink and use a custom partitioner to write the N
>messages into the N different partitions.
It might be worth to make this easier and add a new feature for it. There are
multiple options:
* extend `to()` / `addSink()` with a "broadcast" option/config
* add `toAllPartitions()` / `addBroadcastSink()` methods
* allow StreamPartitioner to return `-1` for "all partitions"
* extend `StreamPartitioner` to allow returning more than one partition (ie a
list/collection of integers instead of a single int)
The first three options imply that a "full broadcast" is supported only, so
it's less flexible. On the other hand, it's easier to use (especially the first
two options are easy as they do not require to implement a custom partitioner).
The last option would be most flexible and also allow for a "partial broadcast"
(aka multi-cast) pattern. It might also be possible to combine two options, or
maye even a totally different idea.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
