This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
The following commit(s) were added to refs/heads/master by this push:
new 141f38197 IMPALA-12935: First pass on Calcite planner functions
141f38197 is described below
commit 141f38197be2ca23757cb8b3f283cdb9dd62de47
Author: Steve Carlin <[email protected]>
AuthorDate: Thu Mar 21 18:45:24 2024 -0700
IMPALA-12935: First pass on Calcite planner functions
This commit handles the first pass on getting functions to work
through the Calcite planner. Only basic functions will work with
this commit. Implicit conversions for parameters are not yet supported.
Custom UDFs are also not supported yet.
The ImpalaOperatorTable is used at validation time to check for
existence of the function name for Impala. At first, it will check
Calcite operators for the existence of the function name (A TODO,
IMPALA-13096, is that we need to remove non-supported names from the
parser file). It is preferable to use the Calcite Operator since
Calcite does some optimizations based on the Calcite Operator class.
If the name is not found within the Calcite Operators, a check is done
within the BuiltinsDb (TODO: IMPALA-13095 handle UDFs) for the function.
If found, and SqlOperator class is generated on the fly to handle this
function.
The validation process for Calcite includes a call into the operator
method "inferReturnType". This method will validate that there exists
a function that will handle the operands, and if so, return the "return
type" of the function. In this commit, we will assume that the Calcite
operators will match Impala functionality. In later commits, there
will be overrides where we will use Impala validation for operators
where Calcite's validation isn't good enough.
After validation is complete, the functions will be in a Calcite format.
After the rest of compilation (relnode conversion, optimization) is
complete, the function needs to be converted back into Impala form (the
Expr object) to eventually get it into its thrift request.
In this commit, all functions are converted into Expr starting in the
ImpalaProjectRel, since this is the RelNode where functions do their
thing. The RexCallConverter and RexLiteralConverter get called via the
CreateExprVisitor for this conversion.
Since Calcite is providing the analysis portion of the planning, there
is no need to go through Impala's Analyzer object. However, the Impala
planner requires Expr objects to be analyzed. To get around this, the
AnalyzedFunctionCallExpr and AnalyzedNullLiteral objects exist which
analyze the expression in the constructor. While this could potentially
be combined with the existing FunctionCallExpr and NullLiteral objects,
this fits in with the general plan to avoid changing "fe" Impala code
as much as we can until much later in the commit cycle. Also, there
will be other Analyzed*Expr classes created in the future, but this
commit is intended for basic function call expressions only.
One minor change to the parser is added with this commit. Calcite parser
does not have acknowledge the "string" datatype, so this has been
added here in Parser.jj and config.fmpp.
Change-Id: I2dd4e402d69ee10547abeeafe893164ffd789b88
Reviewed-on: http://gerrit.cloudera.org:8080/21357
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
java/calcite-planner/src/main/codegen/config.fmpp | 1 +
.../src/main/codegen/templates/Parser.jj | 7 +
.../functions/AnalyzedFunctionCallExpr.java | 117 +++++++++++++
.../calcite/functions/AnalyzedNullLiteral.java | 58 +++++++
.../impala/calcite/functions/FunctionResolver.java | 67 ++++++++
.../impala/calcite/functions/RexCallConverter.java | 70 ++++++++
.../calcite/functions/RexLiteralConverter.java | 126 ++++++++++++++
.../impala/calcite/operators/ImpalaOperator.java | 149 +++++++++++++++++
.../calcite/operators/ImpalaOperatorTable.java | 103 ++++++++++++
.../impala/calcite/rel/node/ImpalaProjectRel.java | 5 +-
.../impala/calcite/rel/util/CreateExprVisitor.java | 42 ++++-
.../impala/calcite/service/CalciteJniFrontend.java | 8 +
.../impala/calcite/service/CalciteValidator.java | 3 +-
.../impala/calcite/type/ImpalaTypeConverter.java | 185 ++++++++++++++++++++-
.../queries/QueryTest/calcite.test | 35 ++++
15 files changed, 962 insertions(+), 14 deletions(-)
diff --git a/java/calcite-planner/src/main/codegen/config.fmpp
b/java/calcite-planner/src/main/codegen/config.fmpp
index b66528157..0a609de35 100644
--- a/java/calcite-planner/src/main/codegen/config.fmpp
+++ b/java/calcite-planner/src/main/codegen/config.fmpp
@@ -29,6 +29,7 @@ parser: {
# List of new keywords. Example: "DATABASES", "TABLES". If the keyword is
# not a reserved keyword, add it to the 'nonReservedKeywords' section.
keywords: [
+ "STRING"
]
# List of keywords from "keywords" section that are not reserved.
diff --git a/java/calcite-planner/src/main/codegen/templates/Parser.jj
b/java/calcite-planner/src/main/codegen/templates/Parser.jj
index 8dddb1a99..ce018c0ae 100644
--- a/java/calcite-planner/src/main/codegen/templates/Parser.jj
+++ b/java/calcite-planner/src/main/codegen/templates/Parser.jj
@@ -6000,6 +6000,8 @@ SqlTypeNameSpec CharacterTypeName(Span s) :
int precision = -1;
final SqlTypeName sqlTypeName;
String charSetName = null;
+ // Added Impala code to handle String type
+ boolean isString = false;
}
{
(
@@ -6011,6 +6013,9 @@ SqlTypeNameSpec CharacterTypeName(Span s) :
)
|
<VARCHAR> { s.add(this); sqlTypeName = SqlTypeName.VARCHAR; }
+ |
+ // Added Impala code to handle String type
+ <STRING> { s.add(this); sqlTypeName = SqlTypeName.VARCHAR; isString =
true;}
)
precision = PrecisionOpt()
[
@@ -6018,6 +6023,8 @@ SqlTypeNameSpec CharacterTypeName(Span s) :
charSetName = Identifier()
]
{
+ // Added Impala code to handle String type
+ if (isString) precision = Integer.MAX_VALUE;
return new SqlBasicTypeNameSpec(sqlTypeName, precision, charSetName,
s.end(this));
}
}
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedFunctionCallExpr.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedFunctionCallExpr.java
new file mode 100644
index 000000000..f13be70f2
--- /dev/null
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedFunctionCallExpr.java
@@ -0,0 +1,117 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.impala.calcite.functions;
+
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.impala.analysis.Analyzer;
+import org.apache.impala.analysis.Expr;
+import org.apache.impala.analysis.FunctionCallExpr;
+import org.apache.impala.analysis.FunctionParams;
+import org.apache.impala.catalog.Function;
+import org.apache.impala.catalog.Type;
+import org.apache.impala.common.AnalysisException;
+import org.apache.impala.common.ImpalaException;
+
+import java.util.List;
+
+/**
+ * A FunctionCallExpr that is always in an analyzed state
+ *
+ * The analysis for Calcite expressions can be done in the constructor
+ * rather than issuing a separate call to "analyze" after the object
+ * is constructed.
+ *
+ * For this class, we also want to override the "analyzeImpl" call since
+ * the "fn_" member is passed in rather than deduced at analysis time.
+ *
+ */
+public class AnalyzedFunctionCallExpr extends FunctionCallExpr {
+
+ // Need to save the function because it is known at constructor time. The
+ // resetAnalyzeState() method can be called at various points which could
+ // set the fn_ member to null. So we save the function in the savedFunction_
+ // variable so it can be properly set in analyzeImpl()
+ private final Function savedFunction_;
+
+ private final Analyzer analyzer_;
+
+ // c'tor that takes a list of Exprs that eventually get converted to
FunctionParams
+ public AnalyzedFunctionCallExpr(Function fn, List<Expr> params,
+ RexCall rexCall, Type retType, Analyzer analyzer) throws ImpalaException
{
+ super(fn.getFunctionName(), params);
+ this.savedFunction_ = fn;
+ this.type_ = retType;
+ this.analyze(analyzer);
+ this.analyzer_ = analyzer;
+ }
+
+ // c'tor which does not depend on Calcite's RexCall but is used when Impala's
+ // FunctionParams are created or there is some modifications to it
+ public AnalyzedFunctionCallExpr(Function fn, FunctionParams funcParams,
+ Type retType, Analyzer analyzer) throws ImpalaException {
+ super(fn.getFunctionName(), funcParams);
+ this.savedFunction_ = fn;
+ this.type_ = retType;
+ this.analyze(analyzer);
+ this.analyzer_ = analyzer;
+ }
+
+ public AnalyzedFunctionCallExpr(AnalyzedFunctionCallExpr other) {
+ super(other);
+ this.savedFunction_ = other.savedFunction_;
+ this.type_ = other.type_;
+ this.analyzer_ = other.analyzer_;
+ try {
+ this.analyze(this.analyzer_);
+ } catch (ImpalaException e) {
+ //TODO: IMPALA-13097: Don't throw runtime exception
+ throw new RuntimeException(e);
+ }
+ }
+
+ @Override
+ protected void analyzeImpl(Analyzer analyzer) throws AnalysisException {
+ // Functions have already gone through the analysis phase in Calcite so the
+ // analyzeImpl method is overridden. However, the FunctionName object
+ // still needs to be analyzed. This allows Expr.toSql() to display the
names
+ // correctly in the explain plan.
+ getFnName().analyze(analyzer);
+ this.fn_ = savedFunction_;
+ }
+
+ @Override
+ protected float computeEvalCost() {
+ // TODO: IMPALA-13098: need to implement
+ return UNKNOWN_COST;
+ }
+
+ @Override
+ public Expr clone() { return new AnalyzedFunctionCallExpr(this); }
+
+ @Override
+ public FunctionCallExpr cloneWithNewParams(FunctionParams params) {
+ try {
+ return new AnalyzedFunctionCallExpr(this.getFn(), params,
+ this.type_, analyzer_);
+ } catch (ImpalaException e) {
+ throw new IllegalStateException(e);
+ }
+ }
+
+}
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedNullLiteral.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedNullLiteral.java
new file mode 100644
index 000000000..3a3b74199
--- /dev/null
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedNullLiteral.java
@@ -0,0 +1,58 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.impala.calcite.functions;
+
+import org.apache.impala.analysis.Analyzer;
+import org.apache.impala.analysis.Expr;
+import org.apache.impala.analysis.NullLiteral;
+import org.apache.impala.catalog.Type;
+import org.apache.impala.common.AnalysisException;
+import org.apache.impala.common.ImpalaException;
+
+/**
+ * A NulLLiteral that is always in analyzed state
+ */
+public class AnalyzedNullLiteral extends NullLiteral {
+ private final Analyzer analyzer_;
+
+ private final Type savedType_;
+
+ public AnalyzedNullLiteral(Analyzer analyzer, Type type) throws
ImpalaException {
+ this.analyzer_ = analyzer;
+ savedType_ = type;
+ this.analyze(analyzer);
+ }
+
+ public AnalyzedNullLiteral(AnalyzedNullLiteral other) {
+ super(other);
+ this.analyzer_ = other.analyzer_;
+ this.savedType_ = other.savedType_;
+ }
+
+ @Override
+ public Expr clone() {
+ return new AnalyzedNullLiteral(this);
+ }
+
+ @Override
+ protected void analyzeImpl(Analyzer analyzer) throws AnalysisException {
+ // resetAnalyzedState will remove the type so we use the savedType
+ // from the constructor
+ uncheckedCastTo(savedType_);
+ }
+}
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionResolver.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionResolver.java
new file mode 100644
index 000000000..00b0980ba
--- /dev/null
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionResolver.java
@@ -0,0 +1,67 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.impala.calcite.functions;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import com.google.common.collect.Lists;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.impala.analysis.FunctionName;
+import org.apache.impala.calcite.type.ImpalaTypeConverter;
+import org.apache.impala.catalog.BuiltinsDb;
+import org.apache.impala.catalog.Db;
+import org.apache.impala.catalog.Function;
+import org.apache.impala.catalog.ScalarFunction;
+import org.apache.impala.catalog.Type;
+
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The FunctionResolver is a wrapper around the Impala Function Resolver (via
the
+ * (Db.getFunction() method). In this current iteration, only exact matches are
+ * resolved. TODO: IMPALA-13022: change this comment when implicit conversion
is handled.
+ */
+public class FunctionResolver {
+ protected static final Logger LOG =
+ LoggerFactory.getLogger(FunctionResolver.class.getName());
+
+ public static Function getFunction(String name, List<RelDataType> argTypes) {
+ String lowercaseName = name.toLowerCase();
+
+ List<Type> impalaArgTypes =
ImpalaTypeConverter.getNormalizedImpalaTypes(argTypes);
+ Function searchDesc = new Function(new FunctionName(BuiltinsDb.NAME,
lowercaseName),
+ impalaArgTypes, Type.INVALID, false);
+
+ Function fn = BuiltinsDb.getInstance().getFunction(searchDesc,
+ Function.CompareMode.IS_INDISTINGUISHABLE);
+
+ if (fn == null) {
+ LOG.debug("Failed to find function " + lowercaseName);
+ }
+
+ return fn;
+ }
+}
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexCallConverter.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexCallConverter.java
new file mode 100644
index 000000000..66bdf85d1
--- /dev/null
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexCallConverter.java
@@ -0,0 +1,70 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.impala.calcite.functions;
+
+import com.google.common.collect.Lists;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexNode;
+import org.apache.impala.analysis.Analyzer;
+import org.apache.impala.analysis.Expr;
+import org.apache.impala.calcite.type.ImpalaTypeConverter;
+import org.apache.impala.catalog.Function;
+import org.apache.impala.catalog.Type;
+import org.apache.impala.common.AnalysisException;
+import org.apache.impala.common.ImpalaException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.List;
+
+/**
+ * Static Helper class that returns Exprs for RexCall nodes.
+ */
+public class RexCallConverter {
+ protected static final Logger LOG =
+ LoggerFactory.getLogger(RexCallConverter.class.getName());
+
+ /*
+ * Returns the Impala Expr object for RexCallConverter.
+ */
+ public static Expr getExpr(RexCall rexCall, List<Expr> params,
+ RexBuilder rexBuilder, Analyzer analyzer) throws ImpalaException {
+
+ String funcName = rexCall.getOperator().getName().toLowerCase();
+
+ Function fn = getFunction(funcName, rexCall.getOperands(),
rexCall.getType());
+
+ Type impalaRetType =
ImpalaTypeConverter.createImpalaType(fn.getReturnType(),
+ rexCall.getType().getPrecision(), rexCall.getType().getScale());
+
+ return new AnalyzedFunctionCallExpr(fn, params, rexCall, impalaRetType,
analyzer);
+ }
+
+ private static Function getFunction(String name, List<RexNode> args,
+ RelDataType retType) throws ImpalaException {
+ List<RelDataType> argTypes = Lists.transform(args, RexNode::getType);
+ Function fn = FunctionResolver.getFunction(name, argTypes);
+ if (fn == null) {
+ throw new AnalysisException("Could not find function \"" + name + "\" in
Impala "
+ + "with args " + argTypes + " and return type " + retType);
+ }
+ return fn;
+ }
+}
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexLiteralConverter.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexLiteralConverter.java
new file mode 100644
index 000000000..5e5d706c3
--- /dev/null
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexLiteralConverter.java
@@ -0,0 +1,126 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.impala.calcite.functions;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Lists;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexLiteral;
+import org.apache.calcite.sql.type.SqlTypeName;
+import org.apache.calcite.util.DateString;
+import org.apache.calcite.util.TimestampString;
+import org.apache.impala.analysis.Analyzer;
+import org.apache.impala.analysis.Expr;
+import org.apache.impala.analysis.BoolLiteral;
+import org.apache.impala.analysis.DateLiteral;
+import org.apache.impala.analysis.NumericLiteral;
+import org.apache.impala.analysis.StringLiteral;
+import org.apache.impala.calcite.type.ImpalaTypeConverter;
+import org.apache.impala.catalog.Function;
+import org.apache.impala.catalog.PrimitiveType;
+import org.apache.impala.catalog.ScalarType;
+import org.apache.impala.catalog.Type;
+import org.apache.impala.common.AnalysisException;
+import org.apache.impala.common.ImpalaException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.base.Preconditions;
+
+import java.math.BigDecimal;
+import java.util.List;
+
+/**
+ * Static Helper class that returns Exprs for RexLiteral nodes.
+ */
+public class RexLiteralConverter {
+ protected static final Logger LOG =
+ LoggerFactory.getLogger(RexLiteralConverter.class.getName());
+
+ /*
+ * Returns Expr object for ImpalaRexLiteral
+ */
+ public static Expr getExpr(RexLiteral rexLiteral, Analyzer analyzer)
+ throws ImpalaException {
+ if (SqlTypeName.INTERVAL_TYPES.contains(rexLiteral.getTypeName())) {
+ return new NumericLiteral(
+ new BigDecimal(rexLiteral.getValueAs(Long.class)), Type.BIGINT);
+ }
+ switch (rexLiteral.getTypeName()) {
+ case NULL:
+ Type type = ImpalaTypeConverter.createImpalaType(rexLiteral.getType());
+ return new AnalyzedNullLiteral(null, type);
+ case BOOLEAN:
+ Expr boolExpr = new BoolLiteral(rexLiteral.getValueAs(Boolean.class));
+ boolExpr.analyze(null);
+ return boolExpr;
+ case BIGINT:
+ case DECIMAL:
+ case DOUBLE:
+ Expr numericExpr = new
NumericLiteral(rexLiteral.getValueAs(BigDecimal.class),
+ ImpalaTypeConverter.createImpalaType(rexLiteral.getType()));
+ numericExpr.analyze(null);
+ return numericExpr;
+ case CHAR:
+ case VARCHAR:
+ ScalarType charType = rexLiteral.getType().getSqlTypeName() ==
SqlTypeName.VARCHAR
+ ? Type.STRING
+ :
ScalarType.createCharType(rexLiteral.getValueAs(String.class).length());
+ if (charType.getPrimitiveType() == PrimitiveType.CHAR) {
+ // ensure no wildcards or char length 0 which will crash impalad
+ Preconditions.checkState(charType.getLength() > 0);
+ }
+ Expr charExpr = new StringLiteral(rexLiteral.getValueAs(String.class),
+ charType, false);
+ charExpr.analyze(null);
+ return charExpr;
+ case DATE:
+ DateString dateStringClass = rexLiteral.getValueAs(DateString.class);
+ String dateString = (dateStringClass == null) ? null :
dateStringClass.toString();
+ Expr dateExpr = new DateLiteral(rexLiteral.getValueAs(Integer.class),
dateString);
+ dateExpr.analyze(null);
+ return dateExpr;
+ case TIMESTAMP:
+ return createCastTimestampExpr(rexLiteral, analyzer);
+ default:
+ throw new AnalysisException("Unsupported RexLiteral: "
+ + rexLiteral.getTypeName());
+ }
+ }
+
+ /**
+ * Create a cast timestamp expression from a String to a Timestamp.
+ * The only way to create a TimestampLiteral directly in Impala is by
accessing
+ * the backend. This will normally be done earlier in Calcite via constant
folding.
+ * If constant folding was not allowed, it means we did not have access to
the backend
+ * and thus need to do a cast in order to support conversion to a Timestamp.
+ */
+ private static Expr createCastTimestampExpr(RexLiteral rexLiteral,
+ Analyzer analyzer)
+ throws ImpalaException {
+ List<RelDataType> typeNames =
+ ImmutableList.of(ImpalaTypeConverter.getRelDataType(Type.STRING));
+
+ String timestamp = rexLiteral.getValueAs(TimestampString.class).toString();
+ List<Expr> argList =
+ Lists.newArrayList(new StringLiteral(timestamp, Type.STRING, false));
+ Function castFunc = FunctionResolver.getFunction("casttotimestamp",
typeNames);
+ return new AnalyzedFunctionCallExpr(castFunc, argList, null,
Type.TIMESTAMP,
+ analyzer);
+ }
+}
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaOperator.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaOperator.java
new file mode 100644
index 000000000..1024d7818
--- /dev/null
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaOperator.java
@@ -0,0 +1,149 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.impala.calcite.operators;
+
+
+import com.google.common.base.Preconditions;
+import org.apache.commons.lang.StringUtils;
+import org.apache.impala.calcite.type.ImpalaTypeSystemImpl;
+import org.apache.calcite.jdbc.JavaTypeFactoryImpl;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rel.type.RelDataTypeFactory;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.sql.SqlCallBinding;
+import org.apache.calcite.sql.SqlFunction;
+import org.apache.calcite.sql.SqlFunctionCategory;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.SqlOperandCountRange;
+import org.apache.calcite.sql.SqlOperatorBinding;
+import org.apache.calcite.sql.SqlSyntax;
+import org.apache.calcite.sql.type.SqlOperandCountRanges;
+import org.apache.impala.analysis.FunctionName;
+import org.apache.impala.calcite.functions.FunctionResolver;
+import org.apache.impala.calcite.type.ImpalaTypeConverter;
+import org.apache.impala.catalog.BuiltinsDb;
+import org.apache.impala.catalog.Function;
+import org.apache.impala.catalog.Type;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * ImpalaOperator is a custom Calcite operator that handles all generic
functions
+ * that are not defined by Calcite. It is preferable to use a Calcite operator
+ * if possible because Calcite has optimizations that are based on the operator
+ * class.
+ */
+public class ImpalaOperator extends SqlFunction {
+ protected static final Logger LOG =
+ LoggerFactory.getLogger(ImpalaOperator.class.getName());
+
+ // Allow any count because this is used for all functions. Validation for
specific
+ // number of parameters will be done when Impala function resolving is done.
+ public static SqlOperandCountRange ANY_COUNT_RANGE =
SqlOperandCountRanges.any();
+
+ public ImpalaOperator(String name) {
+ super(name.toUpperCase(), SqlKind.OTHER, null, null, null,
+ SqlFunctionCategory.USER_DEFINED_FUNCTION);
+ }
+
+ @Override
+ public RelDataType inferReturnType(SqlOperatorBinding opBinding) {
+ final List<RelDataType> operandTypes = getOperandTypes(opBinding);
+
+ RexBuilder rexBuilder =
+ new RexBuilder(new JavaTypeFactoryImpl(new ImpalaTypeSystemImpl()));
+ RelDataTypeFactory factory = rexBuilder.getTypeFactory();
+
+ // Resolve Impala function through Impala method.
+ // TODO: IMPALA-13022: Right now, CompareMode is INDISTINGUISHABLE because
this
+ // commit only deals with exact matches. This will change in a future
commit.
+ Function fn = FunctionResolver.getFunction(getName(), operandTypes);
+
+ if (fn == null) {
+ throw new IllegalArgumentException("Cannot infer return type for "
+ + getName() + "; operand types: " + operandTypes);
+ }
+
+ RelDataType returnType =
+ ImpalaTypeConverter.getRelDataType(fn.getReturnType());
+ return isNullable(operandTypes)
+ ? returnType
+ : factory.createTypeWithNullability(returnType, true);
+ }
+
+ @Override
+ public boolean checkOperandTypes(SqlCallBinding callBinding, boolean
throwOnFailure) {
+ // Validation for operand types are done when checking for signature in
+ // the inferReturnType method.
+ return true;
+ }
+
+ @Override
+ public SqlOperandCountRange getOperandCountRange() {
+ // Validation for operand count ranges are done when checking for
signature in
+ // the inferReturnType method.
+ return ANY_COUNT_RANGE;
+ }
+
+ /**
+ * getAllowedSignatures used for error messages.
+ */
+ @Override
+ public String getAllowedSignatures(String opNameToUse) {
+ // TODO: IMPALA-13099 leave blank for now since this might be hard to
derive because
+ // of implicit type support.
+ return "";
+ }
+
+ @Override
+ public SqlSyntax getSyntax() {
+ return SqlSyntax.FUNCTION;
+ }
+
+ private List<RelDataType> getOperandTypes(SqlOperatorBinding opBinding) {
+ Preconditions.checkState(opBinding instanceof SqlCallBinding);
+ SqlCallBinding callBinding = (SqlCallBinding) opBinding;
+
+ List<RelDataType> operandTypes = new ArrayList<>();
+ for (int i = 0; i < callBinding.getOperandCount(); ++i) {
+ if (callBinding.isOperandNull(i, false)) {
+ operandTypes.add(ImpalaTypeConverter.getRelDataType(Type.NULL));
+ } else {
+ operandTypes.add(callBinding.getOperandType(i));
+ }
+ }
+ return operandTypes;
+ }
+
+ // return false if all operand types are not nullable. Else return true.
+ private boolean isNullable(List<RelDataType> operandTypes) {
+ for (RelDataType rdt : operandTypes) {
+ if (rdt.isNullable()) {
+ return true;
+ }
+ }
+ return false;
+ }
+}
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaOperatorTable.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaOperatorTable.java
new file mode 100644
index 000000000..6e2540e94
--- /dev/null
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaOperatorTable.java
@@ -0,0 +1,103 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.impala.calcite.operators;
+
+import com.google.common.base.Preconditions;
+import org.apache.calcite.sql.SqlFunctionCategory;
+import org.apache.calcite.sql.SqlIdentifier;
+import org.apache.calcite.sql.validate.SqlNameMatcher;
+import org.apache.calcite.sql.SqlOperator;
+import org.apache.calcite.sql.SqlSyntax;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.calcite.sql.util.ReflectiveSqlOperatorTable;
+import org.apache.impala.calcite.functions.FunctionResolver;
+import org.apache.impala.catalog.BuiltinsDb;
+import org.apache.impala.catalog.Db;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The ImpalaOperatorTable is used to hold all the operators and to resolve
+ * the functions that are being validated.
+ *
+ * The main method used to resolve the operators is lookupOperatorOverloads.
+ * The method mutates the operatorList and fills it with the matching operator.
+ * If Calcite contains an operator that matches the name passed in, the Calcite
+ * operator is the one that is used. It is preferable to use the Calcite
operator
+ * since Calcite performs optimizations based on the operator class.
+ *
+ * If a Calcite operator is not found, we check the Impala functions to see if
+ * the function name is known by Impala. If so, an ImpalaOperator class is
generated
+ * on the fly.
+ *
+ * TODO: IMPALA-13095: Handle UDFs
+ */
+public class ImpalaOperatorTable extends ReflectiveSqlOperatorTable {
+ protected static final Logger LOG =
+ LoggerFactory.getLogger(ImpalaOperatorTable.class.getName());
+
+ private static ImpalaOperatorTable INSTANCE;
+
+ /**
+ * lookupOperatorOverloads: See class comment above for details.
+ */
+ @Override
+ public void lookupOperatorOverloads(SqlIdentifier opName,
SqlFunctionCategory category,
+ SqlSyntax syntax, List<SqlOperator> operatorList, SqlNameMatcher
nameMatcher) {
+
+
+ // Check Calcite operator table for existence.
+ SqlStdOperatorTable.instance().lookupOperatorOverloads(opName, category,
syntax,
+ operatorList, nameMatcher);
+ Preconditions.checkState(operatorList.size() <= 1);
+ if (operatorList.size() == 1) {
+ return;
+ }
+
+ // There shouldn't be more than one opName with our usage, so throw an
exception
+ // if this happens.
+ if (opName.names.size() > 1) {
+ throw new RuntimeException("Cannot handle identifier with more than one
name: " +
+ opName);
+ }
+
+ // Check Impala Builtins for existence: TODO: IMPALA-13095: handle UDFs
+ if (!BuiltinsDb.getInstance().containsFunction(opName.getSimple())) {
+ return;
+ }
+
+ operatorList.add(new ImpalaOperator(opName.getSimple()));
+ }
+
+ public static synchronized void create(Db db) {
+ if (INSTANCE != null) {
+ return;
+ }
+ INSTANCE = new ImpalaOperatorTable();
+ }
+
+ public static ImpalaOperatorTable getInstance() {
+ return INSTANCE;
+ }
+}
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java
index fc9706fe8..e0e54e0d2 100644
---
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java
@@ -68,7 +68,6 @@ public class ImpalaProjectRel extends Project
@Override
public NodeWithExprs getPlanNode(ParentPlanRelContext context) throws
ImpalaException {
-
NodeWithExprs inputWithExprs = getChildPlanNode(context);
// get the output exprs for this node that are needed by the parent node.
@@ -88,8 +87,8 @@ public class ImpalaProjectRel extends Project
throws ImpalaException {
ImpalaPlanRel inputRel = (ImpalaPlanRel) getInput(0);
- CreateExprVisitor visitor =
- new CreateExprVisitor(inputNodeWithExprs.outputExprs_);
+ CreateExprVisitor visitor = new
CreateExprVisitor(getCluster().getRexBuilder(),
+ inputNodeWithExprs.outputExprs_, basicAnalyzer);
ImmutableList.Builder<Expr> builder = new ImmutableList.Builder();
for (RexNode rexNode : getProjects()) {
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/CreateExprVisitor.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/CreateExprVisitor.java
index e8884741e..aec0295cd 100644
---
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/CreateExprVisitor.java
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/CreateExprVisitor.java
@@ -17,6 +17,8 @@
package org.apache.impala.calcite.rel.util;
+import com.google.common.collect.Lists;
+import org.apache.calcite.rex.RexBuilder;
import org.apache.calcite.rex.RexCall;
import org.apache.calcite.rex.RexCorrelVariable;
import org.apache.calcite.rex.RexDynamicParam;
@@ -31,23 +33,34 @@ import org.apache.calcite.rex.RexRangeRef;
import org.apache.calcite.rex.RexSubQuery;
import org.apache.calcite.rex.RexTableInputRef;
import org.apache.calcite.rex.RexVisitorImpl;
+
+import org.apache.impala.analysis.Analyzer;
import org.apache.impala.analysis.Expr;
import org.apache.impala.common.AnalysisException;
import org.apache.impala.common.ImpalaException;
+import org.apache.impala.calcite.functions.RexCallConverter;
+import org.apache.impala.calcite.functions.RexLiteralConverter;
+import java.util.ArrayList;
import java.util.List;
/**
* CreateExprVisitor will generate Impala expressions for function calls and
literals.
- * TODO: In this iteration, it only handles input refs.
*/
public class CreateExprVisitor extends RexVisitorImpl<Expr> {
+ private final RexBuilder rexBuilder_;
+
private final List<Expr> inputExprs_;
- public CreateExprVisitor(List<Expr> inputExprs) {
+ private final Analyzer analyzer_;
+
+ public CreateExprVisitor(RexBuilder rexBuilder, List<Expr> inputExprs,
+ Analyzer analyzer) {
super(false);
this.inputExprs_ = inputExprs;
+ this.rexBuilder_ = rexBuilder;
+ this.analyzer_ = analyzer;
}
@Override
@@ -57,12 +70,24 @@ public class CreateExprVisitor extends RexVisitorImpl<Expr>
{
@Override
public Expr visitCall(RexCall rexCall) {
- throw new RuntimeException("Not supported");
+ try {
+ List<Expr> params = Lists.newArrayList();
+ for (RexNode operand : rexCall.getOperands()) {
+ params.add(operand.accept(this));
+ }
+ return RexCallConverter.getExpr(rexCall, params, rexBuilder_, analyzer_);
+ } catch (ImpalaException e) {
+ throw new RuntimeException(e);
+ }
}
@Override
public Expr visitLiteral(RexLiteral rexLiteral) {
- throw new RuntimeException("Not supported");
+ try {
+ return RexLiteralConverter.getExpr(rexLiteral, analyzer_);
+ } catch (ImpalaException e) {
+ throw new RuntimeException(e);
+ }
}
@Override
@@ -122,4 +147,13 @@ public class CreateExprVisitor extends
RexVisitorImpl<Expr> {
throw new AnalysisException(e);
}
}
+
+ public static List<Expr> getExprs(CreateExprVisitor visitor, List<RexNode>
operands)
+ throws ImpalaException {
+ List<Expr> exprs = new ArrayList<>();
+ for (RexNode operand : operands) {
+ exprs.add(getExpr(visitor, operand));
+ }
+ return exprs;
+ }
}
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java
index af5c8225f..54b582a94 100644
---
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java
@@ -23,8 +23,11 @@ import
org.apache.calcite.rel.metadata.DefaultRelMetadataProvider;
import org.apache.calcite.rel.metadata.JaninoRelMetadataProvider;
import org.apache.calcite.rel.metadata.RelMetadataQuery;
import org.apache.calcite.sql.SqlNode;
+import org.apache.impala.calcite.functions.FunctionResolver;
+import org.apache.impala.calcite.operators.ImpalaOperatorTable;
import org.apache.impala.calcite.rel.node.NodeWithExprs;
import org.apache.impala.calcite.rel.node.ImpalaPlanRel;
+import org.apache.impala.catalog.BuiltinsDb;
import org.apache.impala.common.ImpalaException;
import org.apache.impala.common.InternalException;
import org.apache.impala.common.JniUtil;
@@ -68,6 +71,7 @@ public class CalciteJniFrontend extends JniFrontend {
public CalciteJniFrontend(byte[] thriftBackendConfig, boolean isBackendTest)
throws ImpalaException, TException {
super(thriftBackendConfig, isBackendTest);
+ loadCalciteImpalaFunctions();
}
/**
@@ -191,6 +195,10 @@ public class CalciteJniFrontend extends JniFrontend {
}
}
+ private static void loadCalciteImpalaFunctions() {
+ ImpalaOperatorTable.create(BuiltinsDb.getInstance());
+ }
+
public static class QueryContext {
private final TQueryCtx queryCtx_;
private final String stmt_;
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java
index 7c455f97e..7ca3ddbb4 100644
---
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java
@@ -24,6 +24,7 @@ import org.apache.calcite.sql.SqlNode;
import org.apache.calcite.sql.fun.SqlStdOperatorTable;
import org.apache.calcite.sql.validate.SqlValidator;
import org.apache.calcite.sql.validate.SqlValidatorUtil;
+import org.apache.impala.calcite.operators.ImpalaOperatorTable;
import org.apache.impala.calcite.type.ImpalaTypeSystemImpl;
import org.apache.impala.calcite.validate.ImpalaConformance;
@@ -52,7 +53,7 @@ public class CalciteValidator implements CompilerStep {
this.catalogReader = mdHandler.getCalciteCatalogReader();
this.sqlValidator = SqlValidatorUtil.newValidator(
- SqlStdOperatorTable.instance(),
+ ImpalaOperatorTable.getInstance(),
catalogReader, typeFactory,
SqlValidator.Config.DEFAULT
.withConformance(ImpalaConformance.INSTANCE)
diff --git
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java
index 8a93f6541..0f6573a68 100644
---
a/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java
+++
b/java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java
@@ -23,17 +23,11 @@ import org.apache.calcite.jdbc.JavaTypeFactoryImpl;
import org.apache.calcite.rel.type.RelDataType;
import org.apache.calcite.rel.type.RelDataTypeFactory;
import org.apache.calcite.rex.RexBuilder;
-import org.apache.calcite.rex.RexLiteral;
-import org.apache.calcite.rex.RexNode;
-import org.apache.calcite.sql.SqlCollation;
import org.apache.calcite.sql.type.SqlTypeName;
-import org.apache.calcite.util.ConversionUtil;
-import org.apache.impala.catalog.PrimitiveType;
import org.apache.impala.catalog.ScalarType;
import org.apache.impala.catalog.Type;
import org.apache.impala.thrift.TPrimitiveType;
-import java.nio.charset.Charset;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@@ -77,7 +71,10 @@ public class ImpalaTypeConverter {
map.put(Type.DOUBLE, factory.createSqlType(SqlTypeName.DOUBLE));
map.put(Type.TIMESTAMP, factory.createSqlType(SqlTypeName.TIMESTAMP));
map.put(Type.DATE, factory.createSqlType(SqlTypeName.DATE));
+ map.put(Type.DECIMAL, factory.createSqlType(SqlTypeName.DECIMAL));
map.put(Type.BINARY, factory.createSqlType(SqlTypeName.BINARY));
+ map.put(Type.CHAR, factory.createSqlType(SqlTypeName.CHAR, 1));
+ map.put(Type.VARCHAR, factory.createSqlType(SqlTypeName.VARCHAR, 1));
map.put(Type.STRING, factory.createSqlType(SqlTypeName.VARCHAR,
Integer.MAX_VALUE));
map.put(Type.NULL, factory.createSqlType(SqlTypeName.NULL));
@@ -118,6 +115,18 @@ public class ImpalaTypeConverter {
}
}
+ /**
+ * Get the normalized RelDataType given an impala type.
+ */
+ public static RelDataType getRelDataType(Type impalaType) {
+ if (impalaType == null) {
+ return null;
+ }
+ TPrimitiveType primitiveType = impalaType.getPrimitiveType().toThrift();
+ Type normalizedImpalaType = getImpalaType(primitiveType);
+ return impalaToCalciteMap.get(normalizedImpalaType);
+ }
+
/**
* Create Impala types given primitive types.
* Primitive types should not be exposed outside of this class.
@@ -163,4 +172,168 @@ public class ImpalaTypeConverter {
throw new RuntimeException("Unknown type " + argType);
}
}
+
+ /**
+ * Create a new impala type given a relDataType
+ */
+ public static Type createImpalaType(RelDataType relDataType) {
+ // First retrieve the normalized impala type
+ if (relDataType.getSqlTypeName() == SqlTypeName.VARCHAR &&
+ ((relDataType.getPrecision() == Integer.MAX_VALUE) ||
+ (relDataType.getPrecision() == -1))) {
+ return Type.STRING;
+ }
+ Type impalaType = getType(relDataType.getSqlTypeName());
+ // create the impala type given the normalized type, precision, and scale.
+ return createImpalaType(impalaType, relDataType.getPrecision(),
+ relDataType.getScale());
+ }
+
+ public static Type createImpalaType(Type impalaType, int precision, int
scale) {
+ TPrimitiveType primitiveType = impalaType.getPrimitiveType().toThrift();
+ // Char, varchar, decimal, and fixed_uda_intermediate contain precisions
and need to
+ // be treated separately.
+ switch (primitiveType) {
+ case CHAR:
+ return ScalarType.createCharType(precision);
+ case VARCHAR:
+ return (precision == Integer.MAX_VALUE || precision == -1)
+ ? Type.STRING
+ : ScalarType.createVarcharType(precision);
+ case DECIMAL:
+ if (precision == -1) {
+ return Type.DECIMAL;
+ }
+ return ScalarType.createDecimalType(precision, scale);
+ case FIXED_UDA_INTERMEDIATE:
+ return ScalarType.createFixedUdaIntermediateType(precision);
+ default:
+ return impalaType;
+ }
+ }
+
+ public static Type getType(SqlTypeName calciteTypeName) {
+ switch (calciteTypeName) {
+ case TINYINT:
+ return Type.TINYINT;
+ case SMALLINT:
+ return Type.SMALLINT;
+ case INTEGER:
+ return Type.INT;
+ case INTERVAL_YEAR:
+ case INTERVAL_MONTH:
+ case INTERVAL_DAY:
+ case INTERVAL_HOUR:
+ case INTERVAL_MINUTE:
+ case INTERVAL_SECOND:
+ case BIGINT:
+ return Type.BIGINT;
+ case VARCHAR:
+ return Type.VARCHAR;
+ case BOOLEAN:
+ return Type.BOOLEAN;
+ case REAL:
+ case FLOAT:
+ return Type.FLOAT;
+ case DOUBLE:
+ return Type.DOUBLE;
+ case DECIMAL:
+ return Type.DECIMAL;
+ case CHAR:
+ return Type.CHAR;
+ case TIMESTAMP:
+ return Type.TIMESTAMP;
+ case DATE:
+ return Type.DATE;
+ case NULL:
+ return Type.NULL;
+ case BINARY:
+ return Type.BINARY;
+ default:
+ throw new RuntimeException("Type " + calciteTypeName + " not
supported yet.");
+ }
+ }
+
+ // helper function to handle translation of lists.
+ public static List<Type> getNormalizedImpalaTypes(List<RelDataType>
relDataTypes) {
+ return Lists.transform(relDataTypes,
ImpalaTypeConverter::getNormalizedImpalaType);
+ }
+
+ /**
+ * Return the default impala type given a reldatatype that potentially has
precision.
+ */
+ public static ScalarType getNormalizedImpalaType(RelDataType relDataType) {
+ SqlTypeName sqlTypeName = relDataType.getSqlTypeName();
+ if (SqlTypeName.INTERVAL_TYPES.contains(sqlTypeName)) {
+ return Type.BIGINT;
+ }
+ switch (sqlTypeName) {
+ case VARCHAR:
+ return relDataType.getPrecision() == Integer.MAX_VALUE
+ ? Type.STRING : Type.VARCHAR;
+ case CHAR:
+ return Type.CHAR;
+ case DECIMAL:
+ return Type.DECIMAL;
+ case BOOLEAN:
+ return Type.BOOLEAN;
+ case TINYINT:
+ return Type.TINYINT;
+ case SMALLINT:
+ return Type.SMALLINT;
+ case INTEGER:
+ return Type.INT;
+ case BIGINT:
+ return Type.BIGINT;
+ case FLOAT:
+ case REAL:
+ return Type.FLOAT;
+ case DOUBLE:
+ return Type.DOUBLE;
+ case TIMESTAMP:
+ return Type.TIMESTAMP;
+ case DATE:
+ return Type.DATE;
+ case BINARY:
+ return Type.BINARY;
+ case SYMBOL:
+ return null;
+ case NULL:
+ return Type.NULL;
+ default:
+ throw new RuntimeException("Unknown SqlTypeName " + sqlTypeName +
+ " to convert to Impala.");
+ }
+ }
+
+ public static List<RelDataType> createRelDataTypes(List<Type> impalaTypes) {
+ List<RelDataType> result = Lists.newArrayList();
+ for (Type t : impalaTypes) {
+ result.add(createRelDataType(t));
+ }
+ return result;
+ }
+
+ /**
+ * Create a new RelDataType given the Impala type.
+ */
+ public static RelDataType createRelDataType(Type impalaType) {
+ if (impalaType == null) {
+ return null;
+ }
+ TPrimitiveType primitiveType = impalaType.getPrimitiveType().toThrift();
+ if (primitiveType == TPrimitiveType.DECIMAL) {
+ ScalarType scalarType = (ScalarType) impalaType;
+ RexBuilder rexBuilder =
+ new RexBuilder(new JavaTypeFactoryImpl(new ImpalaTypeSystemImpl()));
+ RelDataTypeFactory factory = rexBuilder.getTypeFactory();
+ RelDataType decimalDefinedRetType =
factory.createSqlType(SqlTypeName.DECIMAL,
+ scalarType.decimalPrecision(), scalarType.decimalScale());
+ return factory.createTypeWithNullability(decimalDefinedRetType, true);
+ }
+ // for all other arguments besides decimal, we just normalize the datatype
and return
+ // the previously created RelDataType.
+ Type normalizedImpalaType = getImpalaType(primitiveType);
+ return impalaToCalciteMap.get(normalizedImpalaType);
+ }
}
diff --git a/testdata/workloads/functional-query/queries/QueryTest/calcite.test
b/testdata/workloads/functional-query/queries/QueryTest/calcite.test
index b05f3089a..9c6e0cf07 100644
--- a/testdata/workloads/functional-query/queries/QueryTest/calcite.test
+++ b/testdata/workloads/functional-query/queries/QueryTest/calcite.test
@@ -115,3 +115,38 @@ int,string,binary
---- RUNTIME_PROFILE
row_regex: .*PlannerType: CalcitePlanner.*
====
+---- QUERY
+# Tiny test for Calcite. At the point of this commit, very few functions work.
This
+# is a test that includes one of the functions that does.
+# The ultimate goal is to include all tests in the testing framework, so there
is
+# no need to be extensive about testing in this file.
+select bigint_col, abs(cast(-3 as bigint)), abs(-3000000000) from
functional.alltypestiny;
+---- RESULTS
+0,3,3000000000
+10,3,3000000000
+0,3,3000000000
+10,3,3000000000
+0,3,3000000000
+10,3,3000000000
+0,3,3000000000
+10,3,3000000000
+---- TYPES
+bigint,bigint,bigint
+====
+---- QUERY
+# Tiny test for Calcite. At the point of this commit, very few functions work.
This
+# is a test that includes one of the functions that does.
+# The ultimate goal is to include all tests in the testing framework, so there
is
+# no need to be extensive about testing in this file.
+select cast(cast('2005-12-13 08:00:00' as string) AS TIMESTAMP) from
functional.alltypestiny;
+---- RESULTS
+2005-12-13 08:00:00
+2005-12-13 08:00:00
+2005-12-13 08:00:00
+2005-12-13 08:00:00
+2005-12-13 08:00:00
+2005-12-13 08:00:00
+2005-12-13 08:00:00
+2005-12-13 08:00:00
+---- TYPES
+timestamp
