[
https://issues.apache.org/jira/browse/HIVE-30021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikram Ahuja updated HIVE-30021:
--------------------------------
Description:
1. CREATE TABLE fails outright.
ArrayWritableObjectInspector.getObjectInspector(
(ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java)
has no case for this type, so ParquetHiveSerDe.initialize() — called during
Table.checkValidity() — throws immediately: UnsupportedOperationException:
Unknown field type: timestamp with local time zone('US/Pacific'). So a table
with this column type can never even be created with STORED AS PARQUET.
2. Schema generation has no case for it either.
HiveSchemaConverter.convertType()
(.../io/parquet/convert/HiveSchemaConverter.java) — which builds the physical
Parquet MessageType at write time — falls through to throw new
IllegalArgumentException("Unknown type: " + typeInfo).
3. Write path has no case either. DataWritableWriter.createWriter()
(.../io/parquet/write/DataWritableWriter.java) explicitly rejects it:
IllegalArgumentException: Unsupported primitive data type: TIMESTAMPLOCALTZ
4. Read path silently returns the wrong type. ETypeConverter
(.../io/parquet/convert/ETypeConverter.java) has a case that matches
TIMESTAMPLOCALTZ_TYPE_NAME for the INT64 encoding, but shares it with plain
TIMESTAMP and always constructs a TimestampWritableV2 — never a
TimestampLocalTZWritable. The INT96 converter (EINT96_TIMESTAMP_CONVERTER) has
no type-specific handling for it at all, same default-to-TimestampWritableV2
problem.
*Steps to reproduce:*
{code:java}
CREATE EXTERNAL TABLE t (
tsval timestamp with local time zone
)
stored as parquet;{code}
> TIMESTAMP WITH LOCAL TIME ZONE columns cannot be created, written, or read
> correctly on Parquet tables
> ------------------------------------------------------------------------------------------------------
>
> Key: HIVE-30021
> URL: https://issues.apache.org/jira/browse/HIVE-30021
> Project: Hive
> Issue Type: Bug
> Reporter: Vikram Ahuja
> Assignee: Vikram Ahuja
> Priority: Major
>
> 1. CREATE TABLE fails outright.
> ArrayWritableObjectInspector.getObjectInspector(
> (ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java)
> has no case for this type, so ParquetHiveSerDe.initialize() — called during
> Table.checkValidity() — throws immediately: UnsupportedOperationException:
> Unknown field type: timestamp with local time zone('US/Pacific'). So a table
> with this column type can never even be created with STORED AS PARQUET.
> 2. Schema generation has no case for it either.
> HiveSchemaConverter.convertType()
> (.../io/parquet/convert/HiveSchemaConverter.java) — which builds the physical
> Parquet MessageType at write time — falls through to throw new
> IllegalArgumentException("Unknown type: " + typeInfo).
> 3. Write path has no case either. DataWritableWriter.createWriter()
> (.../io/parquet/write/DataWritableWriter.java) explicitly rejects it:
> IllegalArgumentException: Unsupported primitive data type: TIMESTAMPLOCALTZ
> 4. Read path silently returns the wrong type. ETypeConverter
> (.../io/parquet/convert/ETypeConverter.java) has a case that matches
> TIMESTAMPLOCALTZ_TYPE_NAME for the INT64 encoding, but shares it with plain
> TIMESTAMP and always constructs a TimestampWritableV2 — never a
> TimestampLocalTZWritable. The INT96 converter (EINT96_TIMESTAMP_CONVERTER)
> has no type-specific handling for it at all, same
> default-to-TimestampWritableV2 problem.
>
> *Steps to reproduce:*
> {code:java}
> CREATE EXTERNAL TABLE t (
> tsval timestamp with local time zone
> )
> stored as parquet;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
