[GitHub] flink pull request: [FLINK-2997] Support range partition with user...

2016-03-09 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1776#issuecomment-194697700 Just a minor comment, mostly looks good to me. @fhueske , do you want to take a look at this PR? --- If your project is set up for it, you can reply to this email

[GitHub] flink pull request: [FLINK-2997] Support range partition with user...

2016-03-09 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1776#discussion_r55639603 --- Diff: flink-tests/src/test/java/org/apache/flink/test/javaApiOperators/CustomDistributionITCase.java --- @@ -0,0 +1,137 @@ +/* + * Licensed

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-08 Thread ChengXiangLi
Github user ChengXiangLi closed the pull request at: https://github.com/apache/flink/pull/1746 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-07 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1746#issuecomment-193636206 merging this PR --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-07 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1746#issuecomment-193631851 FLINK-3586 is created to track Long AVG issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-06 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1746#issuecomment-192826952 Add null value handling in aggregate functions, some code refactor. Besides, i found the previous Double/Float average function does not work in partial

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-05 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1746#discussion_r55134609 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/runtime/aggregate/AvgAggregate.scala --- @@ -17,115 +17,144

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-03 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1746#issuecomment-191698778 Thanks, @fhueske , i've updated the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-02 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1746#issuecomment-191621309 Hi, @fhueske , i found 2 test failure due to the input data of `AggregateMapFunction` is `Tuple` instead of `Row` while table config is `EFFICIENT`. I thought

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-02 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1746#discussion_r54838441 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/runtime/aggregate/AggregateReduceGroupFunction.scala --- @@ -0,0 +1,198

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-02 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1746#discussion_r54838416 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/runtime/aggregate/Aggregate.scala --- @@ -17,26 +17,77

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-02 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1746#issuecomment-191186632 Hi, @fhueske , i've update the PR based on your comments, except object reusing in `AggregateGroupCombineFunction`. I found `GroupReduceOperator` would tran

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-03-02 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1746#discussion_r54706319 --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/runtime/aggregate/Aggregate.scala --- @@ -17,26 +17,77

[GitHub] flink pull request: [FLINK-3474] support partial aggregate

2016-02-29 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1746 [FLINK-3474] support partial aggregate This PR includes: * New aggregate interface which support partial aggregate. * Simple aggregate function implementation, such as SUM/AVG/COUNT/MIN

[GitHub] flink pull request: [Flink-3226] Translate logical plan into physi...

2016-02-13 Thread ChengXiangLi
Github user ChengXiangLi closed the pull request at: https://github.com/apache/flink/pull/1567 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [Flink-3226] Translate logical plan into physi...

2016-01-31 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1567#issuecomment-177831170 @twalthr and @fhueske , please help to review. For other rules, `DataSetScanRule` and `DataSetUnionRule` should be ready yet, and the missed Flink UDF in

[GitHub] flink pull request: [Flink-3226] Translate logical plan into physi...

2016-01-31 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1567 [Flink-3226] Translate logical plan into physical plan This PR includes: * `DataSetAggregateRule` and `DataSetJoinRule` implementation. * Several common SQL aggregate function

[GitHub] flink pull request: [Flink-3226] Implement project and filter tran...

2016-01-31 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1556#issuecomment-177541447 I got it, as most of the work is done by code generation, current DataSetProjectRule/DataSetFilterRule/DataSetCalRule implementation should be ready. Most part of

[GitHub] flink pull request: [FLINK-3225] Implemented optimization of Table...

2016-01-31 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1559#issuecomment-177429493 Yes, i think we should do that, my Skype is chengxiang.li...@gmail.com. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] flink pull request: [Flink-3226] Implement project and filter tran...

2016-01-29 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1556#issuecomment-176894471 Thanks for the comments, i would create a new PR based on Fabian's work. Besides, @twalthr , what's the scope of your code generation, previous Table

[GitHub] flink pull request: [FLINK-3225] Implemented optimization of Table...

2016-01-29 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1559#issuecomment-176891835 Yes, @fhueske , i saw you rewrite the DataSetRelNodes on scala, i would continue my work based on your PR. --- If your project is set up for it, you can reply to

[GitHub] flink pull request: [Flink-3226] Implement project and filter tran...

2016-01-28 Thread ChengXiangLi
Github user ChengXiangLi closed the pull request at: https://github.com/apache/flink/pull/1556 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [FLINK-3282] Add FlinkRelNode interface.

2016-01-28 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1544#issuecomment-176144759 I didn't prefer Java or Scala, the reason why i writen this in Java is that, this part logic would be translated from Calcite RelNodes(which is in Java

[GitHub] flink pull request: [Flink-3226] Implement project and filter tran...

2016-01-28 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1556 [Flink-3226] Implement project and filter translator. This is just prototype implementation to translate Calcite Project and Filter into Flink RelNode. It still miss several parts, which could

[GitHub] flink pull request: [FLINK-3282] Add FlinkRelNode interface.

2016-01-27 Thread ChengXiangLi
Github user ChengXiangLi closed the pull request at: https://github.com/apache/flink/pull/1544 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [FLINK-3282] Add FlinkRelNode interface.

2016-01-26 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1544#issuecomment-175365963 Just curious, about the merging the PR, the way i know is that, checkout the tableOnCalcite, apply the patch, commit and push to flink repo, is this the way how

[GitHub] flink pull request: [FLINK-3282] Add FlinkRelNode interface.

2016-01-26 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1544#issuecomment-175360681 @fhueske , yes, i just remove the method in `FlinkRelNode` as java interface only allow public method, just forgot to change the methods in implemented

[GitHub] flink pull request: [FLINK-3282] Add FlinkRelNode interface.

2016-01-25 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1544#issuecomment-174760809 Thanks for the comments, @fhueske and @twalthr . i've updated the PR. --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] flink pull request: [FLINK-3282] Add FlinkRelNode interface.

2016-01-25 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1544#issuecomment-174479220 Hi, @fhueske , What about `Project` and `Agggregate`? they are translated to `FlinkMap` and `FlinkReduce` which includes `MapFunction` and `GroupReduceFunction

[GitHub] flink pull request: Add FlinkRelNode interface.

2016-01-25 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1544 Add FlinkRelNode interface. Add FlinkRelNode interface as it's depended by both FLINK-3226 and FLINK-3227, the implementation would be added in FLINK-3226. You can merge this pull re

[GitHub] flink pull request: Enable range partition with custom data distri...

2016-01-21 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1540#discussion_r50505552 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/distributions/DataDistribution.java --- @@ -47,6 +49,12 @@ */ Object

[GitHub] flink pull request: Enable range partition with custom data distri...

2016-01-21 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1540#discussion_r50505395 --- Diff: flink-core/src/main/java/org/apache/flink/api/common/operators/GenericDataSinkBase.java --- @@ -176,6 +182,31 @@ public void setLocalOrder

[GitHub] flink pull request: Enable range partition with custom data distri...

2016-01-21 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1540#discussion_r50505273 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/DataSet.java --- @@ -1253,9 +1257,11 @@ public long count() throws Exception

[GitHub] flink pull request: Enable range partition with custom data distri...

2016-01-21 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1540#discussion_r50505138 --- Diff: flink-core/pom.xml --- @@ -70,6 +70,12 @@ under the License. test

[GitHub] flink pull request: [FLINK-2871] support outer join for hash join ...

2016-01-21 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1469#discussion_r50495814 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/MutableHashTable.java --- @@ -1558,7 +1641,209 @@ public void reset

[GitHub] flink pull request: [FLINK-2871] support outer join for hash join ...

2016-01-21 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1469#discussion_r50495621 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/MutableHashTable.java --- @@ -486,11 +518,43 @@ protected boolean

[GitHub] flink pull request: [FLINK-2871] support outer join for hash join ...

2016-01-20 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1469#issuecomment-173151704 I did simple regression test based on `HashVsSortMiniBenchmark`, the result looks like: Test | Before | After

[GitHub] flink pull request: [FLINK-2871] support outer join for hash join ...

2016-01-20 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1469#discussion_r50232198 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/MutableHashTable.java --- @@ -1558,7 +1641,209 @@ public void reset

[GitHub] flink pull request: [FLINK-2871] support outer join for hash join ...

2016-01-19 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1469#discussion_r50220219 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/operators/util/HashVsSortMiniBenchmark.java --- @@ -184,7 +184,7 @@ public void

[GitHub] flink pull request: [FLINK-2871] support outer join for hash join ...

2016-01-19 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1469#discussion_r50220144 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/operators/util/HashVsSortMiniBenchmark.java --- @@ -130,11 +130,11 @@ public void

[GitHub] flink pull request: [FLINK-2871] support outer join for hash join ...

2016-01-19 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1469#discussion_r50219555 --- Diff: flink-runtime/src/test/java/org/apache/flink/runtime/operators/hash/HashTableITCase.java --- @@ -1534,6 +1534,106 @@ public void

[GitHub] flink pull request: [FLINK-2871] support outer join for hash join ...

2016-01-19 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1469#discussion_r50218331 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/MutableHashTable.java --- @@ -486,11 +518,43 @@ protected boolean

[GitHub] flink pull request: [FLINK-2871] support outer join for hash join ...

2016-01-19 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1469#discussion_r50217624 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/MutableHashTable.java --- @@ -486,11 +518,43 @@ protected boolean

[GitHub] flink pull request: [FLINK-3192] Add explain support to print ast ...

2016-01-07 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1477#discussion_r49161542 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/explain/PlanJsonParser.java --- @@ -0,0 +1,142

[GitHub] flink pull request: [FLINK-3192] Add explain support to print ast ...

2016-01-07 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1477#discussion_r49161453 --- Diff: pom.xml --- @@ -796,6 +796,7 @@ under the License. flink-tests/src/test/resources/testdata

[GitHub] flink pull request: [FLINK-3192] Add explain support to print ast ...

2016-01-07 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1477#discussion_r49161417 --- Diff: flink-staging/flink-table/pom.xml --- @@ -94,6 +94,12 @@ under the License. test

[GitHub] flink pull request: [FLINK-3192] Add explain support to print ast ...

2016-01-07 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1477#discussion_r49161249 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/Table.scala --- @@ -267,5 +271,24 @@ case class Table(private[flink] val

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-12-21 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-166297616 Sorry, @fhueske , i misunderstood your test data, the keys should be skewed on some value, while in my previous test, the keys are now skewed. it's complica

[GitHub] flink pull request: [FLINK-2971] support outer join for hash join ...

2015-12-17 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1469 [FLINK-2971] support outer join for hash join on build side. 1. There are 4 reserved bytes left in bucket header of `MutableHashTable`, as there are only 9 elements in each bucket, This PR

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-12-17 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-165652344 Hi, @fhueske , For the partition part, i think it's normal that `RangePartition` is slower than `HashParition`, as you've mentioned, `RangePartition`

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-12-10 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-163533029 Thanks, @fhueske . I've updated the PR to use `PARTITION_CUSTOM`. The fail test should be unrelated. --- If your project is set up for it, you can reply to

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-12-06 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r46792344 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/traversals/RangePartitionRewriter.java --- @@ -0,0 +1,194

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-12-03 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r46642181 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/traversals/RangePartitionRewriter.java --- @@ -0,0 +1,194

[GitHub] flink pull request: [FLINK-2549] add topK operator to DataSet.

2015-12-02 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1161#issuecomment-161501441 Hi, @StephanEwen , is there any progress of Managed Memory Allocation abstractions for UDF? Not only about TopK operator, i think it's also very importan

[GitHub] flink pull request: [FLINK-2115] [Table API] support no aggregatio...

2015-12-02 Thread ChengXiangLi
Github user ChengXiangLi closed the pull request at: https://github.com/apache/flink/pull/1377 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [FLINK-3087] [Table-API] support multi count i...

2015-12-02 Thread ChengXiangLi
Github user ChengXiangLi closed the pull request at: https://github.com/apache/flink/pull/1414 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [FLINK-2115] [Table API] support no aggregatio...

2015-12-01 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1377#issuecomment-161190371 Hi, @aljoscha , i guess you may forget this PR :smile: , could you help to move this forward? it should be ready yet. --- If your project is set up for it, you

[GitHub] flink pull request: [FLINK-3098] [Table API] Cast Date to Long thr...

2015-12-01 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1431 [FLINK-3098] [Table API] Cast Date to Long throw compile error. While cast Date to Long, the generated code has compile error. You can merge this pull request into a Git repository by running

[GitHub] flink pull request: [FLINK-3087] [Table-API] support multi count i...

2015-12-01 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1414#issuecomment-161153580 Let's reuse the count aggregation now, as it's better for performance now. I would revisit this after NULL value handling is enabled. --- If your proj

[GitHub] flink pull request: [FLINK-3087] [Table-API] support multi count i...

2015-11-30 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1414#issuecomment-160822216 `a.count` and `b.count` may not the same all the times(for example is `a` contains NULL values), that's why i try to aggregate it twice. But NULL value i

[GitHub] flink pull request: [FLINK-3087] [Table-API] support multi count i...

2015-11-26 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1414 [FLINK-3087] [Table-API] support multi count in aggregation The `Literal(1)` is used as the IntermediateField of `Count` aggregation, so multi `Count` looks the same in `ExpandAggregations

[GitHub] flink pull request: [FLINK-2115] [Table API] support no aggregatio...

2015-11-19 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1377#issuecomment-158007073 Ok, i think it could be added in the `Select` operator description in Table API page, it would depends on FLINK-2955. Seems it takes quite long time delay to sync

[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...

2015-11-19 Thread ChengXiangLi
Github user ChengXiangLi closed the pull request at: https://github.com/apache/flink/pull/1318 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] flink pull request: [FLINK-2115] [Table API] support no aggregatio...

2015-11-18 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1377#issuecomment-157932844 @aljoscha , it's a very interesting topic, should we add a limitation that only allow key fields to be no aggregated function field in `select` clause? O

[GitHub] flink pull request: [FLINK-2115] [Table API] support no aggregatio...

2015-11-18 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1377 [FLINK-2115] [Table API] support no aggregation after groupBy. `JavaBatchTranslator` does not support no aggregation `select` after `groupBy` previously, add the logic in this PR. You can

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-11 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r44522180 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/traversals/RangePartitionRewriter.java --- @@ -0,0 +1,194

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-11 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r44521192 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/operators/GroupReduceWithCombineProperties.java --- @@ -121,6 +121,9 @@ public

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-10 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-155403470 Thanks, @fhueske , I've updated the PR and verified in test environment that the deadlock issue has fixed on latest code. --- If your project is set up f

[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...

2015-11-09 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1318#discussion_r44366143 --- Diff: docs/libs/table.md --- @@ -123,13 +123,205 @@ DataSet result = tableEnv.toDataSet(filtered, WC.class); When using Java, the embedded DSL

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-06 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r44125013 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/traversals/RangePartitionRewriter.java --- @@ -0,0 +1,188

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-06 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r44124427 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/plan/Channel.java --- @@ -134,6 +134,10 @@ public Channel(PlanNode sourceNode

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-05 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43992361 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/traversals/RangePartitionRewriter.java --- @@ -0,0 +1,188

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-05 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43991097 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/operators/PartitionOperator.java --- @@ -148,9 +147,6 @@ else if (pKeys instanceof

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-05 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43990006 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/functions/RangeBoundaryBuilder.java --- @@ -0,0 +1,71 @@ +/* + * Licensed to the

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-05 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43989719 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/functions/AssignRangeIndex.java --- @@ -0,0 +1,55 @@ +/* + * Licensed to the

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-05 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43987723 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/traversals/RangePartitionRewriter.java --- @@ -0,0 +1,188

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-05 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43986746 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/operators/PartitionOperator.java --- @@ -148,9 +147,6 @@ else if (pKeys instanceof

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-05 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43985314 --- Diff: flink-optimizer/src/main/java/org/apache/flink/optimizer/traversals/RangePartitionRewriter.java --- @@ -0,0 +1,188

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-04 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43982452 --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/shipping/OutputEmitter.java --- @@ -135,7 +116,7 @@ public OutputEmitter

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-04 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43973217 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/functions/RangeBoundaryBuilder.java --- @@ -0,0 +1,71 @@ +/* + * Licensed to the

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-11-04 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43971562 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/functions/AssignRangeIndex.java --- @@ -0,0 +1,55 @@ +/* + * Licensed to the

[GitHub] flink pull request: [FLINK-2951] [Table API] add union operator to...

2015-11-04 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1315#discussion_r43971472 --- Diff: flink-test-utils/src/main/java/org/apache/flink/test/util/TestBaseUtils.java --- @@ -310,6 +310,20 @@ public static void

[GitHub] flink pull request: [FLINK-2951] [Table API] add union operator to...

2015-11-04 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1315#discussion_r43971393 --- Diff: flink-staging/flink-table/src/main/scala/org/apache/flink/api/table/Table.scala --- @@ -243,5 +243,29 @@ case class Table(private[flink] val

[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...

2015-11-02 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1318 [FLINK-2955] Add operators description in Table API page On the Table API page, there is no formal introduction of current supported operations, it should be nice to have it. You can merge

[GitHub] flink pull request: [FLINK-2951] [Table API] add union operator to...

2015-11-02 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1315#issuecomment-153220339 Thanks, @twalthr , i've updated the UT to use `collect()` instead of writing to disk. Besides, about the documentation, i found there is no other ope

[GitHub] flink pull request: [FLINK-2951] [Table API] add union operator to...

2015-11-02 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1315 [FLINK-2951] [Table API] add union operator to Table API. Currently, union operation is supported by DataSet/DataStream API, not available in Table API. To union two tables, user has to

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-10-29 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-152108364 I add the `RangePartitionRewriter` at `Optimizer.comple()` finally, as it rewrite `OptimizedPlan`, and utilize the `JavaApiPostPass`. --- If your project is set

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-10-27 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43220047 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/DataSet.java --- @@ -1223,6 +1230,51 @@ public long count() throws Exception

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-10-26 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1255#discussion_r43088529 --- Diff: flink-java/src/main/java/org/apache/flink/api/java/DataSet.java --- @@ -1223,6 +1230,51 @@ public long count() throws Exception

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-10-21 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-149849817 Implemented the range partitioner with KeySelector based on broadcast data distribution, it's not fully finished yet, @fhueske , for the range partitioner

[GitHub] flink pull request: [FLINK-2853] [tests] Apply JMH on MutableHashT...

2015-10-20 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1267#issuecomment-149763186 LGTM, +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] flink pull request: [FLINK-2853] [tests] Apply JMH on MutableHashT...

2015-10-19 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1267#discussion_r42452733 --- Diff: flink-benchmark/src/test/java/org/apache/flink/benchmark/runtime/operates/hash/MutableHashTablePerformanceBenchmark.java --- @@ -0,0 +1,361

[GitHub] flink pull request: [FLINK-2853] [tests] Apply JMH on MutableHashT...

2015-10-19 Thread ChengXiangLi
Github user ChengXiangLi commented on a diff in the pull request: https://github.com/apache/flink/pull/1267#discussion_r42452678 --- Diff: flink-runtime/pom.xml --- @@ -210,7 +210,7 @@ under the License. ${curator.version} test

[GitHub] flink pull request: [FLINK-2853] [tests] Apply JMH on MutableHashT...

2015-10-18 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1261#issuecomment-149077635 Hi, @gallenvara , i think this PR has bounded to FLINK-2848, while actually it should link to FLINK-2853, you may need to close this one and open another new PR

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-10-18 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-149074329 Thanks, @fhueske , it's definitely great that if we can only execute the pre-sample logic just once, i would update the code later. --- If your project is s

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-10-14 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1255#issuecomment-148290945 Thanks for the review in advance, @fhueske . Something i want to mention here is that, the range partitioner implementation in this PR follow the existed

[GitHub] flink pull request: [FLINK-7] [Runtime] Enable Range Partitioner.

2015-10-12 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1255 [FLINK-7] [Runtime] Enable Range Partitioner. This PR enable range partitioner for Flink follow the path of existing other partitioners. It depends on the sample operator to random sample data

[GitHub] flink pull request: [FLINK-2743] Add XORShfitRandom and use it in ...

2015-10-03 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1170#issuecomment-145249544 Yes, it should be, @StephanEwen , I'm on National Day vocation now, i would update it when i come back to work at Oct 08. --- If your project is set up f

[GitHub] flink pull request: [FLINK-2754] Add a new module for micro benchm...

2015-09-24 Thread ChengXiangLi
Github user ChengXiangLi commented on the pull request: https://github.com/apache/flink/pull/1177#issuecomment-143097828 Yes, the XORShfit benchmark is ready to merge, and i would lead the work about other benchmark refactor. No one should depends on this module, so i think there

[GitHub] flink pull request: [FLINK-2754] FixedLengthRecordSorter can not w...

2015-09-23 Thread ChengXiangLi
GitHub user ChengXiangLi opened a pull request: https://github.com/apache/flink/pull/1178 [FLINK-2754] FixedLengthRecordSorter can not write to output cross MemorySegments. The issue happens while call FixedLengthRecordSorter::WriteToOutput with non-zero offset and write record

  1   2   >