Hi, kumar Flink support consume/produce from/to multiple kafka topics[1], in your case you can implement KeyedSerializationSchema(legacy interface) or KafkaSerializationSchema[2] to make one producer instance support send data to multiple topics. There is an ITCase you can reference[3].
Best, Leonard Xu [1] https://ci.apache.org/projects/flink/flink-docs-master/dev/connectors/kafka.html#kafka-producer <https://ci.apache.org/projects/flink/flink-docs-master/dev/connectors/kafka.html#kafka-producer> [2]https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/KafkaSerializationSchema.java <https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/KafkaSerializationSchema.java> [3]https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-0.11/src/test/java/org/apache/flink/streaming/connectors/kafka/Kafka011ITCase.java#L126 <https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-0.11/src/test/java/org/apache/flink/streaming/connectors/kafka/Kafka011ITCase.java#L126> > 在 2020年6月1日，15:35，Prasanna kumar <prasannakumarram...@gmail.com> 写道： > > Hi, > > I have Use Case where i read events from a Single kafka Stream comprising of > JSON messages. > > Requirement is to split the stream into multiple output streams based on some > criteria say based on Type of Event or Based on Type and Customer associated > with the event. > > We could achieve the splitting of stream using Side outputs as i have seen in > the documentation. > > Our business environment is such that there could be new event types flowing > in and would the Flink Kafka producer create the topics dynamically based on > the inflowing events. I did not see any documentation saying that it could > create. > > Or should it be always pre created by running a script separately. (Not a > good scalable practice in our case) > > Thanks, > Prasanna.
