questions about Flink's HashJoin performance

2017-05-13 Thread weijie tong
I has a test case to use Flink's MutableHashTable class to do a hash join on a local machine with 64g memory, 64cores. The test case is one build table with 14w rows ,one probe table with 320w rows ,the matched result rows is 12 w. It takes 2.2 seconds to complete the join.The performance seems ba

Re: [DISCUSS] Expose State Backend Interface for UDAGG

2017-05-13 Thread Shaoxuan Wang
Radu, Thanks for the feedback. ProcessFunction is a lower level execution operator which is not able to be accessed by tableAPI and SQL users. FLINK-6544 is trying to create a generic interface to let tableAPI & SQL users access backend state via UDAGG. It will be eventually code-generated to the p

[jira] [Created] (FLINK-6577) Expand supported types for ConfigOptions

2017-05-13 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-6577: --- Summary: Expand supported types for ConfigOptions Key: FLINK-6577 URL: https://issues.apache.org/jira/browse/FLINK-6577 Project: Flink Issue Type: Wish

[jira] [Created] (FLINK-6576) Allow ConfigOptions to validate the configured value

2017-05-13 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-6576: --- Summary: Allow ConfigOptions to validate the configured value Key: FLINK-6576 URL: https://issues.apache.org/jira/browse/FLINK-6576 Project: Flink Issu

[jira] [Created] (FLINK-6575) Disable all tests on Windows that use HDFS

2017-05-13 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-6575: --- Summary: Disable all tests on Windows that use HDFS Key: FLINK-6575 URL: https://issues.apache.org/jira/browse/FLINK-6575 Project: Flink Issue Type: Im