Are you looking for a parallel Ignite sink connector that can connect to kafka topic?
My thoughts are you should be able to do it as long as your group_id is same like mentioned here https://medium.com/@jhansireddy007/how-to-parallelise-kafka-consumers-59c8b0bbc37a I may have misunderstood the question as well. Regards, Saikat On Sun, Sep 12, 2021 at 1:32 PM Shubham Shirur <shirurshub...@gmail.com> wrote: > Yes, I agree with you. Basically kafka connect works in two different ways > i.e standalone and distributed mode. > > In standard mode ignite sink connectors run very smoothly. So if I want to > create multiple topic cache data transmission, I need to setup same number > of workers and connectors which is not recommended in production because > its standalone, if connect node goes down connector will stop. > > But in distributed mode single worker is running across nodes and > rebalancing happens for connectors tasks if any node goes down. In this > case I am able to create just 1 ignite connector at a time. No parallelism. > > Hope this gives better clarity. > > Thanks and regards, > Shubham > > On Sun, Sep 12, 2021, 11:50 PM Saikat Maitra <saikat.mai...@gmail.com> > wrote: > >> Can you please elaborate on distributed mode? >> >> You can always connect multiple client to different kafka topic and write >> data to cluster of Ignite nodes. >> >> Regards, >> Saikat >> >> On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur <shirurshub...@gmail.com> >> wrote: >> >>> Hey, >>> >>> Thanks for replying. I have gone through the documentation and could >>> setup and run connectors in standalone mode as described there in docs. But >>> I want to run it in distributed mode, can you help me with that? >>> >>> Thanks & Regards, >>> Shubham >>> >>> On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra <saikat.mai...@gmail.com> >>> wrote: >>> >>>> Hi Shubham, >>>> >>>> Here are the documents for Apache Ignite Sink Connector using Kafka >>>> https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext >>>> >>>> Please let us know if you have any questions. >>>> >>>> Regards, >>>> Saikat >>>> >>>> On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur < >>>> shirurshub...@gmail.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> I did not find any specific documentation on Apache Ignite Sink >>>>> Connector. I am using a kafka ignite sink connector and want to push some >>>>> kafka topic data in ignite where kafka and ignite are on remote nodes. My >>>>> connector should ideally run on a remote node from the ignite node. >>>>> >>>>> How can I achieve this? >>>>> What configurations I need to pass in the spring xml file which I pass >>>>> as a parameter in connector properties? >>>>> What configurations I need to pass in the spring xml file of the >>>>> ignite server node? >>>>> >>>>> Thanks, >>>>> Shubham >>>>> >>>>
