Re: Streams - multiple clusters support

2020-02-13 Thread Matthias J. Sax
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 There are two reasons (some would be easier to address than others): 1) a client can only connect to one cluster; to allow data repartitioning the producer must write the repartition data into the input cluster, otherwise the consumer cannot read th

Re: Streams - multiple clusters support

2020-02-13 Thread Ryanne Dolan
Cyrille, I don't see why using MM1/2 would break your isolation requirement. But if you can't mirror topics for some reason consider Flink instead of Kafka Streams. Ryanne On Thu, Feb 13, 2020 at 10:52 AM Cyrille Karmann wrote: > Hello, > > We are trying to create a streaming pipeline of data b

Streams - multiple clusters support

2020-02-13 Thread Cyrille Karmann
Hello, We are trying to create a streaming pipeline of data between different Kafka clusters. Our users send data to the input Kafka cluster, and we want to process this data and send the result to topics on another Kafka cluster. We have different reasons for this setup, but mainly it's for isol