Hi, Any help here would be highly appreciated :)
Sagar. On Wed, 27 Dec 2017 at 07:53, Sagar <sagarmeansoc...@gmail.com> wrote: > We have a use case where in we want to assign partitions manually for a > set of topics to allow fine grained control of the records we are fetching. > > Basically what we are trying to achieve is that a group of messages which > logically belong to a particular entity should be sent to the same > partition . So, Topic A and topic B both have a certain field which are > unique and common( say some id = 101) so we want to send both of those > records to partition 1 ( 101 % 10 is our simple partition assignment > strategy written in both producer and Kafka Connect interceptor. > > Fro what I understood, if I want to have a consumer which listens to > partition 1 for both Topic A and Topic B, then we need to use the assign > method. > > I have been reading up a bit and what has been mentioned everywhere is > that we won't have any rebalancing triggered. Also, I tried a simple use > case where in I didn't poll for more than the value configured in > group.max.session.timeout.ms but it didn't die. Is it because its not > part of a consumer group per se? > > So just wanted to understand what points should we declare a consumer is > dead so that we can spin up a new consumer for the same topic partition. We > are using AWS ECS for running our consumers so target group would spin up a > new consumer based upon health checks. > > Any examples + guidelines around this would be highly appreciated. > > Thanks! > Sagar. >
