Re: Setting up the CooperativeStickyAssignor in Java

2022-03-25 Thread Richard Ney
cations_aoa_v3:4", On Sat, Mar 19, 2022 at 6:00 PM Liam Clarke-Hutchinson wrote: > Hi Richard, > > Yeah, the old 2.8.1 version of Kafka clients used by trunk fs2-kafka is > what I think might be the issue, not the wrapper itself, sorry if I was > unclear on that. > > Ple

Re: Setting up the CooperativeStickyAssignor in Java

2022-03-19 Thread Richard Ney
ests to force rebalancing of partitions to see if I can replicate the issue using the latest version. On Sat, Mar 19, 2022 at 2:06 AM Liam Clarke-Hutchinson wrote: > So to clarify, you're using kafka-clients directly? Or via fx2-kafka? If > it's kafka-clients directly, what version

Re: Setting up the CooperativeStickyAssignor in Java

2022-03-18 Thread Richard Ney
/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L422 And it's only including 8 of the 10 partitions that were assigned to that consumer instance. -Richard On Fri, Mar 18, 2022 at 11:15 PM Richard Ney wrote: > Hi Ngā mihi, > > I believe the log entry

Re: Setting up the CooperativeStickyAssignor in Java

2022-03-18 Thread Richard Ney
version is the first version that has correct support for the messaging from the ConsumerCoordinator. I am planning to do a test with the 3.0.0-M5 version which is built on the updated 3.1.0 kafka-clients library and will let the list know. -Richard Ney Sent from my iPhone > On Mar 18, 2022, at

Re: Setting up the CooperativeStickyAssignor in Java

2022-03-18 Thread Richard Ney
ould you please open a JIRA ticket with > steps to reproduce the bug. > > Best, > Bruno > > On 16.03.22 10:01, Luke Chen wrote: > > Hi Richard, > > > > Right, you are not missing any settings beyond the partition assignment > > strategy and the group instance

Re: Setting up the CooperativeStickyAssignor in Java

2022-03-16 Thread Richard Ney
during group rebalance? > > Thank you. > Luke > > On Wed, Mar 16, 2022 at 1:00 PM Richard Ney .invalid> > wrote: > > > Trying to find a good sample of what consumer settings besides setting > > > > ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG to >

Setting up the CooperativeStickyAssignor in Java

2022-03-15 Thread Richard Ney
Trying to find a good sample of what consumer settings besides setting ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG to org.apache.kafka.clients.consumer.CooperativeStickyAssignor is needed to make the rebalance happen cleanly. Unable to find and decent documentation or code samples. I have

Priority queues using kafka topics and a consumer group?

2020-07-21 Thread Richard Ney
Trying to see if there is a way to implement priority topics where a single consumer group is reading from multiple topics each denoting a level of priority and having the Kafka client automatically prioritize reads against the message in the higher priority topics.

Question on Kafka Connect

2020-05-17 Thread Richard Ney
I'm looking at deploying Kafka Connect for a simple export cluster. I'm trying to understand how the connectors are loaded. Are the connectors getting deployed through a process "fork" and they are monitored via the Kafka Connect workers. Or is the connector just instantiated inside a Kafka worker