We have similar usecase and we use flink for transformation. Flink reads from kafka , does the transformation and writes back to kafka.
Thanks, Prince > On Nov 14, 2024, at 8:22 PM, Neeraj Vaidya > <neeraj.vai...@yahoo.co.in.INVALID> wrote: > > I don’t think KStreams is a good option just by itself for inter site > replication. > How about using a replication technology like MM2 to first replicate to a > topic in the destination cluster and then run KStreams client there in the > destination cluster to consume, transform and then produce to your final > topic. > > Regards, > Neeraj > > >> On 15 Nov 2024, at 1:04 PM, Surbhi Mungre <mungre.sur...@gmail.com> wrote: >> >> Can Kafka Connect be used to read messages from one Kafka Cluster, apply >> some basic transformation and write messages to another Kafka Cluster? I >> did not find a Kafka Connect Connector in the list of connectors provided >> by Confluence[1]. I only found a Replicator[2] but for my use-case I want >> to apply some transformation on the messages. >> >> Instead of using Kafka Connect, does it make more sense to use Kafka >> Streams or Spark Streaming. I want to perform very simple transformations. >> >> [1] https://www.confluent.io/product/connectors/ >> [2] >> https://docs.confluent.io/platform/current/multi-dc-deployments/replicator/ >> >> Thanks, >> -Surbhi >
