We have similar usecase and we use flink for transformation. Flink reads from 
kafka , does the transformation and writes back to kafka.

Thanks,
Prince

> On Nov 14, 2024, at 8:22 PM, Neeraj Vaidya 
> <neeraj.vai...@yahoo.co.in.INVALID> wrote:
> 
> I don’t think KStreams is a good option just by itself for inter site 
> replication.
> How about using a replication technology like MM2 to first replicate to a 
> topic in the destination cluster and then run  KStreams client there in the 
> destination cluster to consume, transform and then produce to your final 
> topic.
> 
> Regards,
> Neeraj
> 
> 
>> On 15 Nov 2024, at 1:04 PM, Surbhi Mungre <mungre.sur...@gmail.com> wrote:
>> 
>> Can Kafka Connect be used to read messages from one Kafka Cluster, apply
>> some basic transformation and write messages to another Kafka Cluster? I
>> did not find a Kafka Connect Connector in the list of connectors provided
>> by Confluence[1]. I only found a Replicator[2] but for my use-case I want
>> to apply some transformation on the messages.
>> 
>> Instead of using Kafka Connect, does it make more sense to use Kafka
>> Streams or Spark Streaming. I want to perform very simple transformations.
>> 
>> [1] https://www.confluent.io/product/connectors/
>> [2]
>> https://docs.confluent.io/platform/current/multi-dc-deployments/replicator/
>> 
>> Thanks,
>> -Surbhi
> 

Reply via email to