Yes, I agree with you. Basically kafka connect works in two different ways i.e standalone and distributed mode.
In standard mode ignite sink connectors run very smoothly. So if I want to create multiple topic cache data transmission, I need to setup same number of workers and connectors which is not recommended in production because its standalone, if connect node goes down connector will stop. But in distributed mode single worker is running across nodes and rebalancing happens for connectors tasks if any node goes down. In this case I am able to create just 1 ignite connector at a time. No parallelism. Hope this gives better clarity. Thanks and regards, Shubham On Sun, Sep 12, 2021, 11:50 PM Saikat Maitra <saikat.mai...@gmail.com> wrote: > Can you please elaborate on distributed mode? > > You can always connect multiple client to different kafka topic and write > data to cluster of Ignite nodes. > > Regards, > Saikat > > On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur <shirurshub...@gmail.com> > wrote: > >> Hey, >> >> Thanks for replying. I have gone through the documentation and could >> setup and run connectors in standalone mode as described there in docs. But >> I want to run it in distributed mode, can you help me with that? >> >> Thanks & Regards, >> Shubham >> >> On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <saikat.mai...@gmail.com> >> wrote: >> >>> Hi Shubham, >>> >>> Here are the documents for Apache Ignite Sink Connector using Kafka >>> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext >>> >>> Please let us know if you have any questions. >>> >>> Regards, >>> Saikat >>> >>> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur <shirurshub...@gmail.com> >>> wrote: >>> >>>> Hi, >>>> >>>> I did not find any specific documentation on Apache Ignite Sink >>>> Connector. I am using a kafka ignite sink connector and want to push some >>>> kafka topic data in ignite where kafka and ignite are on remote nodes. My >>>> connector should ideally run on a remote node from the ignite node. >>>> >>>> How can I achieve this? >>>> What configurations I need to pass in the spring xml file which I pass >>>> as a parameter in connector properties? >>>> What configurations I need to pass in the spring xml file of the ignite >>>> server node? >>>> >>>> Thanks, >>>> Shubham >>>> >>>
