[
https://issues.apache.org/jira/browse/KAFKA-8207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16816210#comment-16816210
]
neeraj commented on KAFKA-8207:
-------------------------------
StreamPartitionAssignor sticky policy is for the stickiness of Tasks to
partition. (StickyTaskAssignor)
I am looking for Stickiness of consumer to partition like in normal Kafka
Consumer
We have three consumers c1,c2, and c3 in the same consumer group using KStream
(reading from a Kafka topic with multiple). Each consumer is in a separate
machine. We are aggregating and saving it in the local state store at each
node.
When node 1 goes down (for rolling deployment or for some other reason)
rebalance kicks in and replica which is given by num.standby.replicas take over
so it is fault tolerant, but when node 1 comes back again rebalance again kicks
in and node 1 processes different partitions and updates the state store for
different partitions.
So Ideally, we want that node 1 consumer c1 should work on the same partitions
as it was working earlier so that we can save replay time from change log and
disk space as because of frequent rebalancing local store has data for all the
partitions
problem similar to
[https://tech.transferwise.com/achieving-high-availability-with-kafka-streams/]
> StickyPartitionAssignor for KStream
> -----------------------------------
>
> Key: KAFKA-8207
> URL: https://issues.apache.org/jira/browse/KAFKA-8207
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 2.0.0
> Reporter: neeraj
> Priority: Major
>
> In KStreams I am not able to give a sticky partition assignor or my custom
> partition assignor.
> Overriding the property while building stream does not work
> streams props.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
> CustomAssignor.class.getName());
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
