[
https://issues.apache.org/jira/browse/HIVE-26529?focusedWorklogId=813307&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-813307
]
ASF GitHub Bot logged work on HIVE-26529:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Sep/22 11:43
Start Date: 29/Sep/22 11:43
Worklog Time Spent: 10m
Work Description: szlta commented on code in PR #3607:
URL: https://github.com/apache/hive/pull/3607#discussion_r983436133
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java:
##########
@@ -177,9 +178,19 @@ public boolean shouldSkipCombine(Path path, Configuration
conf) {
@Override
public VectorizedSupport.Support[] getSupportedFeatures() {
- // disabling VectorizedSupport.Support.DECIMAL_64 as Parquet doesn't
support it, and we have no way of telling
- // beforehand what kind of file format we're going to hit later
- return new VectorizedSupport.Support[]{ };
+ throw new UnsupportedOperationException("This overload of
getSupportedFeatures should never be called");
+ }
+
+ @Override
+ public VectorizedSupport.Support[] getSupportedFeatures(HiveConf hiveConf,
Properties properties) {
+ // disabling VectorizedSupport.Support.DECIMAL_64 for Parquet as it
doesn't support it
+ boolean isORCOnly =
Boolean.parseBoolean(properties.getProperty(HiveIcebergMetaHook.ORC_FILES_ONLY));
+ if (!isORCOnly) {
+ HiveConf.setVar(hiveConf,
HiveConf.ConfVars.HIVE_VECTORIZED_INPUT_FORMAT_SUPPORTS_ENABLED, "");
+ return new VectorizedSupport.Support[] {};
+ }
+ HiveConf.setVar(hiveConf,
HiveConf.ConfVars.HIVE_VECTORIZED_INPUT_FORMAT_SUPPORTS_ENABLED, "decimal_64");
Review Comment:
While in line 189 we do need to set this to empty string to prevent LLAP
from picking up decimal64 as supported feature in case
HiveIcebergInputformat.getSupportedFeatures is [] we don't want to set this
true explicitly here. A user may want to have this disabled, and while the
Inputformat could support it we don't want to force the usage of this feature.
Issue Time Tracking
-------------------
Worklog Id: (was: 813307)
Time Spent: 4h 20m (was: 4h 10m)
> Fix VectorizedSupport support for DECIMAL_64 in HiveIcebergInputFormat
> -----------------------------------------------------------------------
>
> Key: HIVE-26529
> URL: https://issues.apache.org/jira/browse/HIVE-26529
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
> Reporter: Rajesh Balamohan
> Assignee: Ayush Saxena
> Priority: Major
> Labels: perfomance, pull-request-available
> Attachments: iceberg_table_with_HiveDecimal.png,
> regular_tables_with_decimal64.png
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> For supporting vectored reads in parquet, DECIMAL_64 support in ORC has been
> disabled in HiveIcebergInputFormat. This causes regressions in queries.
> [https://github.com/apache/hive/blob/master/iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java#L182]
> It will be good to restore DECIMAL_64 support in iceberg input format.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
