[
https://issues.apache.org/jira/browse/HIVE-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitalii Diravka updated HIVE-19069:
-----------------------------------
Description:
Parquet supports several minor types for Decimal ligical data type:
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal
But Hive supports only "fixed_len_byte_array":
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java#L335
After creating parquet external table and quering it via Hive:
{code}
hive> select * from decimal_parquet;
OK
Failed with exception
java.io.IOException:org.apache.parquet.io.ParquetDecodingException: Can not
read value at 1 in block 0 in file maprfs:///tmp/decimal_parquet/0_0_0.parquet
{code}
The sample of parquet file with decimal int32 values is added to the jira:
{code}
vitalii@vitalii-pc:~$ java -jar
parquet-tools/parquet-mr/parquet-tools/target/parquet-tools-1.6.0rc3-SNAPSHOT.jar
schema /tmp/decimal_parquet/0_0_0.parquet
message root {
optional binary a (UTF8);
optional int32 b (DECIMAL(7,2));
}
vitalii@vitalii-pc:~$ java -jar
parquet-tools/parquet-mr/parquet-tools/target/parquet-tools-1.6.0rc3-SNAPSHOT.jar
cat /tmp/md4107_par/0_0_0.parquet
a = a
b = 100
{code}
was:
Parquet supports several minor types for Decimal ligical data type:
https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal
But Hive supports only "fixed_len_byte_array":
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java#L423
After creating parquet external table and quering it via Hive:
{code}
hive> select * from decimal_parquet;
OK
Failed with exception
java.io.IOException:org.apache.parquet.io.ParquetDecodingException: Can not
read value at 1 in block 0 in file maprfs:///tmp/decimal_parquet/0_0_0.parquet
{code}
The sample of parquet file with decimal int32 values is added to the jira:
{code}
vitalii@vitalii-pc:~$ java -jar
parquet-tools/parquet-mr/parquet-tools/target/parquet-tools-1.6.0rc3-SNAPSHOT.jar
schema /tmp/decimal_parquet/0_0_0.parquet
message root {
optional binary a (UTF8);
optional int32 b (DECIMAL(7,2));
}
vitalii@vitalii-pc:~$ java -jar
parquet-tools/parquet-mr/parquet-tools/target/parquet-tools-1.6.0rc3-SNAPSHOT.jar
cat /tmp/md4107_par/0_0_0.parquet
a = a
b = 100
{code}
> Hive can't read int32 and int64 Parquet decimal values
> ------------------------------------------------------
>
> Key: HIVE-19069
> URL: https://issues.apache.org/jira/browse/HIVE-19069
> Project: Hive
> Issue Type: Improvement
> Components: Types
> Affects Versions: 2.3.2
> Reporter: Vitalii Diravka
> Priority: Major
> Labels: decimal, parquet
> Attachments: 0_0_0.parquet
>
>
> Parquet supports several minor types for Decimal ligical data type:
> https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#decimal
> But Hive supports only "fixed_len_byte_array":
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java#L335
> After creating parquet external table and quering it via Hive:
> {code}
> hive> select * from decimal_parquet;
> OK
> Failed with exception
> java.io.IOException:org.apache.parquet.io.ParquetDecodingException: Can not
> read value at 1 in block 0 in file maprfs:///tmp/decimal_parquet/0_0_0.parquet
> {code}
> The sample of parquet file with decimal int32 values is added to the jira:
> {code}
> vitalii@vitalii-pc:~$ java -jar
> parquet-tools/parquet-mr/parquet-tools/target/parquet-tools-1.6.0rc3-SNAPSHOT.jar
> schema /tmp/decimal_parquet/0_0_0.parquet
> message root {
> optional binary a (UTF8);
> optional int32 b (DECIMAL(7,2));
> }
> vitalii@vitalii-pc:~$ java -jar
> parquet-tools/parquet-mr/parquet-tools/target/parquet-tools-1.6.0rc3-SNAPSHOT.jar
> cat /tmp/md4107_par/0_0_0.parquet
> a = a
> b = 100
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
