[
https://issues.apache.org/jira/browse/HIVE-25276?focusedWorklogId=625136&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-625136
]
ASF GitHub Bot logged work on HIVE-25276:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Jul/21 10:10
Start Date: 20/Jul/21 10:10
Worklog Time Spent: 10m
Work Description: marton-bod commented on a change in pull request #2419:
URL: https://github.com/apache/hive/pull/2419#discussion_r672164500
##########
File path:
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java
##########
@@ -233,15 +237,21 @@ public void
preAlterTable(org.apache.hadoop.hive.metastore.api.Table hmsTable, E
preAlterTableProperties.tableLocation = sd.getLocation();
preAlterTableProperties.format = sd.getInputFormat();
preAlterTableProperties.schema = schema(catalogProperties, hmsTable);
- preAlterTableProperties.spec = spec(conf,
preAlterTableProperties.schema, catalogProperties, hmsTable);
preAlterTableProperties.partitionKeys = hmsTable.getPartitionKeys();
context.getProperties().put(HiveMetaHook.ALLOW_PARTITION_KEY_CHANGE,
"true");
// If there are partition keys specified remove them from the HMS table
and add them to the column list
- if (hmsTable.isSetPartitionKeys()) {
+ if (hmsTable.isSetPartitionKeys() &&
!hmsTable.getPartitionKeys().isEmpty()) {
+ List<PartitionTransformSpec> spec =
PartitionTransform.getPartitionTransformSpec(hmsTable.getPartitionKeys());
+ if (!SessionStateUtil.addResource(conf,
hive_metastoreConstants.PARTITION_TRANSFORM_SPEC, spec)) {
+ throw new MetaException("Query state attached to Session state must
be not null. " +
+ "Partition transform metadata cannot be saved.");
+ }
hmsTable.getSd().getCols().addAll(hmsTable.getPartitionKeys());
hmsTable.setPartitionKeysIsSet(false);
}
+ preAlterTableProperties.spec = spec(conf,
preAlterTableProperties.schema, hmsTable);
Review comment:
Is this needed here? Or if we use it only for the validation inside
`spec()`, then maybe remove the local variable, or better yet, extract the
validation logic into a different method we can call here?
##########
File path:
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java
##########
@@ -233,15 +237,21 @@ public void
preAlterTable(org.apache.hadoop.hive.metastore.api.Table hmsTable, E
preAlterTableProperties.tableLocation = sd.getLocation();
preAlterTableProperties.format = sd.getInputFormat();
preAlterTableProperties.schema = schema(catalogProperties, hmsTable);
- preAlterTableProperties.spec = spec(conf,
preAlterTableProperties.schema, catalogProperties, hmsTable);
preAlterTableProperties.partitionKeys = hmsTable.getPartitionKeys();
context.getProperties().put(HiveMetaHook.ALLOW_PARTITION_KEY_CHANGE,
"true");
// If there are partition keys specified remove them from the HMS table
and add them to the column list
- if (hmsTable.isSetPartitionKeys()) {
+ if (hmsTable.isSetPartitionKeys() &&
!hmsTable.getPartitionKeys().isEmpty()) {
+ List<PartitionTransformSpec> spec =
PartitionTransform.getPartitionTransformSpec(hmsTable.getPartitionKeys());
+ if (!SessionStateUtil.addResource(conf,
hive_metastoreConstants.PARTITION_TRANSFORM_SPEC, spec)) {
Review comment:
Where was this part before that we saved it into the session conf?
##########
File path:
iceberg/iceberg-handler/src/test/results/positive/vectorized_iceberg_read.q.out
##########
@@ -129,17 +129,17 @@ Stage-0
Stage-1
Reducer 2 vectorized
File Output Operator [FS_11]
- Select Operator [SEL_10] (rows=1 width=564)
+ Select Operator [SEL_10] (rows=1 width=372)
Review comment:
Out of curiosity: do you know why the width has descreased so much?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 625136)
Time Spent: 4.5h (was: 4h 20m)
> Enable automatic statistics generation for Iceberg tables
> ---------------------------------------------------------
>
> Key: HIVE-25276
> URL: https://issues.apache.org/jira/browse/HIVE-25276
> Project: Hive
> Issue Type: Improvement
> Reporter: Peter Vary
> Assignee: Peter Vary
> Priority: Major
> Labels: pull-request-available
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> During inserts we should have calculate the column statistics
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
