[
https://issues.apache.org/jira/browse/FLINK-7?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047039#comment-15047039
]
ASF GitHub Bot commented on FLINK-7:
------------------------------------
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/1255#issuecomment-162935419
Thanks for the fast update!
Your last commit seems to include changes that have already been merged to
the master branch. Can you revert it and rebase the second last commit on the
current master?
I also got feedback from @StephanEwen about the use of the IntDistribution.
He shares your opinion, that it is not really needed and just incurs additional
overhead due to the second binary search. He proposes to use a
`PARTITION_CUSTOM` strategy instead of a `PARTITION_RANGE` strategy with a
`Partitioner` that simply returns the partition id. You basically can replace
`partChannel.setShipStrategy(ShipStrategyType.PARTITION_RANGE, keys,
sortDirection, null, DataExchangeMode.PIPELINED);`
by
`partChannel.setShipStrategy(ShipStrategyType.PARTITION_CUSTOM, keys, new
IdPartitioner(), DataExchangeMode.PIPELINED);`
and provide an implementation for IdPartitioner. This way, we have still
support for generic DataDistribution range partitioning (although not being
used ATM) and a more efficient automatic range partitioning.
On last very small remark. Could you please set the parallelism also for
the `OptimizerNodes` (`sipNode`, `sicNode`, etc.)? Otherwise, the JSON output
will have some `"parallelism": "default"` values.
I think we're almost there. :-)
Thanks a lot, Fabian
> [GitHub] Enable Range Partitioner
> ---------------------------------
>
> Key: FLINK-7
> URL: https://issues.apache.org/jira/browse/FLINK-7
> Project: Flink
> Issue Type: Sub-task
> Components: Distributed Runtime
> Reporter: GitHub Import
> Assignee: Chengxiang Li
> Fix For: pre-apache
>
>
> The range partitioner is currently disabled. We need to implement the
> following aspects:
> 1) Distribution information, if available, must be propagated back together
> with the ordering property.
> 2) A generic bucket lookup structure (currently specific to PactRecord).
> Tests to re-enable after fixing this issue:
> - TeraSortITCase
> - GlobalSortingITCase
> - GlobalSortingMixedOrderITCase
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/7
> Created by: [StephanEwen|https://github.com/StephanEwen]
> Labels: core, enhancement, optimizer,
> Milestone: Release 0.4
> Assignee: [fhueske|https://github.com/fhueske]
> Created at: Fri Apr 26 13:48:24 CEST 2013
> State: open
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
