Re: Apache Ignite Sink Connector

2021-10-04 Thread Saikat Maitra
Hi, Can you please check these following performance and troubleshooting tips and share if these are helpful in your case? persistence tuning https://ignite.apache.org/docs/latest/perf-and-troubleshooting/persistence-tuning Mem and JVM tuning https://ignite.apache.org/docs/latest/perf-and-trouble

Re: Apache Ignite Sink Connector

2021-09-30 Thread Shubham Shirur
Hi, My ignite cluster contains 3 nodes. And I am using java thick client for processing the data present in cache. But when RAM is at its peak and completely filled. Cluster server starts getting down. Can you help me with this? How can I tune it ? Also, I am using Ignite's native persistence p

Re: Apache Ignite Sink Connector

2021-09-21 Thread Saikat Maitra
Hi, Yes, it makes sense to connect the kafka connector to an existing cluster of Ignite nodes. Are you facing any issues with it? Here is the docs for clustering and discovery process in Ignite https://ignite.apache.org/docs/latest/clustering/clustering Let us know if you have any questions. R

Re: Apache Ignite Sink Connector

2021-09-20 Thread Shubham Shirur
Hi, With parallelism I mean number for connectors in parallel. Even if group id is same, it will work like any standalone application. If node get down due to some reason, connector running on node will be down as well. In distributed mode, JVMs are shared across connect nodes which makes it more

Re: Apache Ignite Sink Connector

2021-09-12 Thread Saikat Maitra
Are you looking for a parallel Ignite sink connector that can connect to kafka topic? My thoughts are you should be able to do it as long as your group_id is same like mentioned here https://medium.com/@jhansireddy007/how-to-parallelise-kafka-consumers-59c8b0bbc37a I may have misunderstood the qu

Re: Apache Ignite Sink Connector

2021-09-12 Thread Shubham Shirur
Yes, I agree with you. Basically kafka connect works in two different ways i.e standalone and distributed mode. In standard mode ignite sink connectors run very smoothly. So if I want to create multiple topic cache data transmission, I need to setup same number of workers and connectors which is n

Re: Apache Ignite Sink Connector

2021-09-12 Thread Saikat Maitra
Can you please elaborate on distributed mode? You can always connect multiple client to different kafka topic and write data to cluster of Ignite nodes. Regards, Saikat On Sun, Sep 12, 2021 at 10:47 AM Shubham Shirur wrote: > Hey, > > Thanks for replying. I have gone through the documentation

Re: Apache Ignite Sink Connector

2021-09-12 Thread Shubham Shirur
Hey, Thanks for replying. I have gone through the documentation and could setup and run connectors in standalone mode as described there in docs. But I want to run it in distributed mode, can you help me with that? Thanks & Regards, Shubham On Sun, Sep 12, 2021, 7:58 PM Saikat Maitra wrote: >

Re: Apache Ignite Sink Connector

2021-09-12 Thread Saikat Maitra
Hi Shubham, Here are the documents for Apache Ignite Sink Connector using Kafka https://github.com/apache/ignite-extensions/tree/master/modules/kafka-ext Please let us know if you have any questions. Regards, Saikat On Wed, Aug 18, 2021 at 11:40 PM Shubham Shirur wrote: > Hi, > > I did not fi

Re: Apache Ignite Sink Connector

2021-09-10 Thread Shubham Shirur
Hi, Thanks its working now. But I am unable to run more than one connector Do you know why? Basically I have 1 kafka topic to 1 raw cache mapping. So I want to run them in parallel. Thanks Shubham On Wed, Sep 1, 2021, 3:09 AM Alexandr Shapkin wrote: > Hello! > > Please share the exception and

Re: Apache Ignite Sink Connector

2021-08-31 Thread Alexandr Shapkin
Hello! Please share the exception and attach Ignite logs. > On 27 Aug 2021, at 15:12, Shubham Shirur wrote: > > Hi, > > Connector is giving me error while running in distributed mode. But running > in standalone. > > Can you please highlight? > > Thanks > Shubham