Hello Quanlong Huang, Riza Suminto, Xuebin Su, Peter Rozsa, Csaba Ringhofer,
Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23709
to look at the new patch set (#2).
Change subject: IMPALA-13552: Add support for size()/length() for array/map
types
......................................................................
IMPALA-13552: Add support for size()/length() for array/map types
This patch enables the UDF interface to accept CollectionVal arguments
by extending the symbol mangling code to handle ARRAY and MAP types.
This architectural improvement allows all collection functions to be
implemented as built-in functions rather than special expression types.
Changes:
- Extended symbols-util.cc to mangle TYPE_ARRAY and TYPE_MAP as
CollectionVal in function signatures
- Implemented size() and length() as UDFs in collection-functions.{h,cc}
- Registered functions in CollectionFunctionsBuiltins.java
- Both functions return the number of elements in a collection or NULL
TODO:
IMPALA-9519: Add support for accepting complex types as UDF parameters
by making the CollectionVal struct public. This requires designing a
public API for complex types, implementing the memory management and
security layer.
Testing:
- Backend unit tests for NULL, empty, and non-empty collections
- Integration tests covering SELECT, WHERE, HAVING, aggregations
- Tests for both size() and length() (aliases)
- Tests for nested arrays and maps
Change-Id: I85b50386790657ea035addb35eed959d71810ab2
---
M be/src/codegen/impala-ir.cc
M be/src/exprs/CMakeLists.txt
M be/src/exprs/anyval-util.cc
M be/src/exprs/anyval-util.h
A be/src/exprs/collection-functions-ir.cc
A be/src/exprs/collection-functions-test.cc
A be/src/exprs/collection-functions.cc
A be/src/exprs/collection-functions.h
M be/src/exprs/scalar-expr-evaluator.cc
M be/src/udf/udf.h
M be/src/util/symbols-util.cc
A fe/src/main/java/org/apache/impala/analysis/CollectionFunctionsBuiltins.java
M fe/src/main/java/org/apache/impala/catalog/ArrayType.java
M fe/src/main/java/org/apache/impala/catalog/BuiltinsDb.java
M fe/src/main/java/org/apache/impala/catalog/MapType.java
M fe/src/main/java/org/apache/impala/util/FunctionUtils.java
A
testdata/workloads/functional-query/queries/QueryTest/collection-functions.test
A tests/query_test/test_collection_functions.py
18 files changed, 860 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/09/23709/2
--
To view, visit http://gerrit.cloudera.org:8080/23709
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I85b50386790657ea035addb35eed959d71810ab2
Gerrit-Change-Number: 23709
Gerrit-PatchSet: 2
Gerrit-Owner: Arnab Karmakar <[email protected]>
Gerrit-Reviewer: Arnab Karmakar <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Peter Rozsa <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
Gerrit-Reviewer: Xuebin Su <[email protected]>
