[
https://issues.apache.org/jira/browse/KAFKA-2652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14969687#comment-14969687
]
ASF GitHub Bot commented on KAFKA-2652:
---------------------------------------
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/353
KAFKA-2652: integrate new group protocol into partition grouping
@guozhangwang
* added ```PartitionGrouper``` (abstract class)
* This class is responsible for grouping partitions to form tasks.
* Users may implement this class for custom grouping.
* added ```DefaultPartitionGrouper```
* our default implementation of ```PartitionGrouper```
* added ```KafkaStreamingPartitionAssignor```
* We always use this as ```PartitionAssignor``` of stream consumers.
* Actual grouping is delegated to ```PartitionGrouper```.
* ```TopologyBuilder```
* added ```topicGroups()```
* This returns groups of related topics according to the topology
* added ```copartitionSources(sourceNodes...)```
* This is used by DSL layer. It asserts the specified source nodes must
be copartitioned.
* added ```copartitionGroups()``` which returns groups of copartitioned
topics
* KStream layer
* keep track of source nodes to determine copartition sources when steams
are joined
* source nodes are set to null when partitioning property is not preserved
(ex. ```map()```, ```transform()```), and this indicates the stream is no
longer joinable
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka grouping
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/353.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #353
----
commit 708718c1be23fad25fa6206f665cbb619c1b5097
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-19T19:38:06Z
partition grouping
commit d2bae046b5509022e2821a2c5eb08853d228e791
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-19T20:19:54Z
wip
commit 86fa8110b23ee1992fbd19daa08c63a4b427448e
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-20T20:01:37Z
long task id
commit 4f4f9ac642ebe0eae33a5c8464309106e9239f2e
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-20T20:03:15Z
Merge branch 'trunk' of github.com:apache/kafka into grouping
commit e4ecf39b9ab0b0f4c915a4f43cfe771b1de69f7f
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-21T19:33:05Z
joinability
commit 37d72a691173a8fe878ac3d99e8973e72f5675c6
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-21T19:33:48Z
Merge branch 'trunk' of github.com:apache/kafka into grouping
commit f68723bab83c3a3f1c15872f4f24bc932df8198f
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-22T18:21:31Z
partition assignor
commit 457cf270222139eae89750781d09abaa07120932
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-22T18:21:40Z
Merge branch 'trunk' of github.com:apache/kafka into grouping
commit 13f3ad703960581229d511287f27345c567b5d3e
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-22T18:34:52Z
complete undoing long taskid
commit 98f3bcc1896fd159ccbbd37fc65b1d9d6f568bb9
Author: Yasuhiro Matsuda <[email protected]>
Date: 2015-10-22T18:45:38Z
fix a test
----
> Incorporate the new consumer protocol with partition-group interface
> --------------------------------------------------------------------
>
> Key: KAFKA-2652
> URL: https://issues.apache.org/jira/browse/KAFKA-2652
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Guozhang Wang
> Assignee: Yasuhiro Matsuda
> Fix For: 0.9.0.1
>
>
> After KAFKA-2464 is checked in, we need to incorporate the new protocol along
> with a partition-group interface.
> The first step maybe a couple of pre-defined partitioning scheme that can be
> chosen by user from some configs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
