Steve Carlin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22408 )
Change subject: IMPALA-13520: Support in clause coercing ...................................................................... IMPALA-13520: Support in clause coercing Calcite has special processing for any in clause. It has a callback function that allows all the parameters to be coerced into its proper type. While there exists a mechanism to do coercion, in the CoerceNodes class, it only handles functions, and the in clause is handled in a special way in Calcite. So we use the Calcite mechanism to derive a common Impala type and coerce all the parameters. The CombineValuesNodesRule is also needed for this change. There is a test case in test_exprs.test where an in clause contains 10,000 params in side the IN clause (e.g. int_col IN (1, 2, 3, ..., 10000). In this case, Calcite creates 10,000 Values RelNodes which takes way too long to process on the execution side. The rule combines all the Values RelNodes into one Values RelNode with many tuples, which Impala handles quickly when converted into the physical Impala PlanNode. Change-Id: I492845d623766b9182bca5eeca22eb3352ef2f3d Reviewed-on: http://gerrit.cloudera.org:8080/22408 Reviewed-by: Joe McDonnell <joemcdonn...@cloudera.com> Reviewed-by: Aman Sinha <amsi...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- A java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/CombineValuesNodesRule.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteOptimizer.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java A java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeCoercionFactory.java A java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeCoercionImpl.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java 6 files changed, 298 insertions(+), 8 deletions(-) Approvals: Joe McDonnell: Looks good to me, but someone else must approve Aman Sinha: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/22408 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I492845d623766b9182bca5eeca22eb3352ef2f3d Gerrit-Change-Number: 22408 Gerrit-PatchSet: 12 Gerrit-Owner: Steve Carlin <scar...@cloudera.com> Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com> Gerrit-Reviewer: Fang-Yu Rao <fangyu....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com> Gerrit-Reviewer: Steve Carlin <scar...@cloudera.com>
