[
https://issues.apache.org/jira/browse/KAFKA-8207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16817133#comment-16817133
]
Matthias J. Sax commented on KAFKA-8207:
----------------------------------------
Thanks for following up!
{quote}StreamPartitionAssignor sticky policy is for the stickiness of Tasks to
partition. (StickyTaskAssignor)
{quote}
Well, StreamPartitionAssignor's stickiness is tailored to not move tasks (and
thus not move partitions), but assign tasks to the same node if possible. (What
you describe is also true, but this property is called "co-partitioning", not
"stickiness".)
{quote}When node 1 goes down (for rolling deployment or for some other reason)
rebalance kicks in and replica which is given by num.standby.replicas take over
so it is fault tolerant, but when node 1 comes back again rebalance again kicks
in and node 1 processes different partitions and updates the state store for
different partitions.
{quote}
This case should actually be covered by StreamPartitionAssignor and it should
migrate the original partitions/task back to node1.
Hence, we still don't want to allow users to overwrite
`StreamPartitionAssignor`. However, we know that `StreamPartitionAssignor` has
some issue for which the stickiness is not achieved. It seems you are hitting
this case. It's not easy to fix unfortunately.
What should help, is current work on KP-345:
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances]
It has a high change to be included in 2.3 release – however, it will also
require a broker upgrade.
> StickyPartitionAssignor for KStream
> -----------------------------------
>
> Key: KAFKA-8207
> URL: https://issues.apache.org/jira/browse/KAFKA-8207
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 2.0.0
> Reporter: neeraj
> Priority: Major
>
> In KStreams I am not able to give a sticky partition assignor or my custom
> partition assignor.
> Overriding the property while building stream does not work
> streams props.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
> CustomAssignor.class.getName());
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
