[
https://issues.apache.org/jira/browse/KAFKA-13555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17950476#comment-17950476
]
Bruno Cadonna commented on KAFKA-13555:
---------------------------------------
[~lorcanj] Thank you for your work on this ticket!
I agree that opening a draft PR is the easiest to discuss the solution.
You do not need to solve this ticket in both assignors at once. Solve it first
in one and then when that is stable, you can also solve it in the other. In the
second step you might also find some ways to generalize the solution and avoid
code duplication.
> Consider number if input topic partitions for task assignment
> -------------------------------------------------------------
>
> Key: KAFKA-13555
> URL: https://issues.apache.org/jira/browse/KAFKA-13555
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Assignee: Lorcan
> Priority: Major
>
> StreamsAssignor tries to distribute tasks evenly across all instances/threads
> of a Kafka Streams application. It knows about instances/thread (to give more
> capacity to instances with more thread), and it distinguishes between
> stateless and stateful tasks. We also try to not move state around but to use
> a sticky assignment if possible. However, the assignment does not take the
> number of input topic partitions into account.
> For example, an upstream tasks could compute two joins, and thus has 3 input
> partitions, while a downstream task compute a follow up aggregation with a
> single input partitions (from the repartition topic). It could happen that
> one thread gets the 3 input partition tasks assigned, while the other thread
> get the single input partition tasks assigned resulting to an uneven
> partition assignment across both threads.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
