Hello Zihao Ye, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21492
to look at the new patch set (#2).
Change subject: WIP: IMPALA-12370: allow converting timestamps to UTC when
writing Kudu
......................................................................
WIP: IMPALA-12370: allow converting timestamps to UTC when writing Kudu
Before this commit only read support was implemented
(convert_kudu_utc_timestamps=true). This change adds write support:
if write_kudu_utc_timestamps=true, then timestamps are converted
from local time to utc during INSERT/UPSERT to Kudu. In case of
ambigious conversions (DST changes) the earlier possible UTC
timestamp is written.
To be able to read back Kudu tables written by Impala correctly
convert_kudu_utc_timestamps and write_kudu_utc_timestamps need to
have the same value.
The conversion is implemented by adding to_utc_timestamp() to inserted
timestamp expressions during planning. This allows doing the same
conversion during the pre-insert sorting and partitioning.
Read support is implemented differently - in that case the plan is not
changed and the scanner does the conversion.
Other changes:
- Before this change, verification of tests with TIMESTAMP results
were skipped when the file format is Kudu. This shouldn't be
necessary so the skipping was removed.
Haven't run the full test suite yet, for this reason the change is WIP.
Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1
---
M be/src/service/query-options.cc
M be/src/service/query-options.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java
M fe/src/main/java/org/apache/impala/util/ExprUtil.java
M
testdata/workloads/functional-query/queries/QueryTest/kudu_predicate_with_timestamp_conversion.test
M
testdata/workloads/functional-query/queries/QueryTest/kudu_runtime_filter_with_timestamp_conversion.test
M
testdata/workloads/functional-query/queries/QueryTest/kudu_timestamp_conversion.test
M tests/common/test_result_verifier.py
M tests/query_test/test_kudu.py
11 files changed, 95 insertions(+), 32 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/21492/2
--
To view, visit http://gerrit.cloudera.org:8080/21492
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibb4995a64e042e7bb261fcc6e6bf7ffce61e9bd1
Gerrit-Change-Number: 21492
Gerrit-PatchSet: 2
Gerrit-Owner: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Zihao Ye <[email protected]>
