Steve Carlin has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/22408 )
Change subject: IMPALA-13520: Support in clause coercing ...................................................................... IMPALA-13520: Support in clause coercing Calcite has special processing for any in clause. It has a callback function that allows all the parameters to be coerced into its proper type. While we do have a mechanism to do coercion, it only handles functions, and the in clause is handled specially within Calcite. So we use the Calcite mechanism to derive a common Impala type and coerce all the parameters. The CombineValueNodesRule is also needed for this change. There is a test case in test_exprs.test where an in clause contains 10,000 params in side the IN clause (e.g. int_col IN (1, 2, 3, ..., 10000). In this case, Calcite creates 10,000 Values RelNodes which takes way too long to process on the execution side. The rule combines all the Values RelNodes into one Values RelNode with many tuples, which Impala handles quickly when converted into the physical Impala PlanNode. Change-Id: I492845d623766b9182bca5eeca22eb3352ef2f3d --- A java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/CombineValueNodesRule.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteOptimizer.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java A java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeCoercionFactory.java A java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeCoercionImpl.java 5 files changed, 282 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/22408/3 -- To view, visit http://gerrit.cloudera.org:8080/22408 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I492845d623766b9182bca5eeca22eb3352ef2f3d Gerrit-Change-Number: 22408 Gerrit-PatchSet: 3 Gerrit-Owner: Steve Carlin <scar...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Fang-Yu Rao <fangyu....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com>
