Hi Ashish, You are right. Flink does not use Kafka based group management. So if you have two clusters consuming the same topic, they will not divide the partitions. The cross cluster HA is not quite possible at this point. It would be good to know the reason you want to have such HA and see if Flink meets you requirement in another way.
Thanks, Jiangjie (Becket) Qin On Thu, Aug 29, 2019 at 9:19 PM ashish pok <ashish...@yahoo.com> wrote: > Looks like Flink is using “assign” partitions instead of “subscribe” which > will not allow participating in a group if I read the code correctly. > > Has anyone solved this type of problem in past of active-active HA across > 2 clusters using Kafka? > > > - Ashish > > On Wednesday, August 28, 2019, 6:52 PM, ashish pok <ashish...@yahoo.com> > wrote: > > All, > > I was wondering what the expected default behavior is when same app is > deployed in 2 separate clusters but with same group Id. In theory idea was > to create active-active across separate clusters but it seems like both > apps are getting all the data from Kafka. > > Anyone else has tried something similar or have an insight on expected > behavior? I was expecting to see partial data on both apps and to get all > data in one app if other was turned off. > > Thanks in advance, > > - Ashish > >
