[
https://issues.apache.org/jira/browse/HIVE-6210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Chu updated HIVE-6210:
---------------------------
Description:
In Hive 10 when I create a table in RCFile, the serde is
org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe
In Hive 12 when I do the same thing, the serde becomes
org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe
Similarly, in Hive 12, when I set FILEFORMAT to RCFILE, the serde will become
LazyBinaryColumnarSerDe, as opposed to ColumnarSerDe in previous versions. What
is the reason behind this change? This seems like a regression bug to me.
Normally, we can work around the issue by explicitly setting the table serde to
be org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe. However, this causes a
problem for our migration to ORC. Specifically, we have a partitioned table for
which we want the new partitions to have locations pointing to ORC partitions,
and the old partitions to have locations pointing to RCFILE partitions.
Moreover, we need the ability to change the location of a partition to point to
RCFILE partition. For this we'd do so by doing SET FILEFORMAT RCFILE. However,
b/c of this serde problem the RCFile partition in an ORC table will have the
wrong serde, and ALTER TABLE doesn't allow us to set serde for a partition.
was:
In Hive 10 when I create a table in RCFile, the serde is
org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe
In Hive 12 when I do the same thing, the serde becomes
org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe
Similarly, in Hive 12, when I set FILEFORMAT to RCFILE, the serde will become
LazyBinaryColumnarSerDe, as opposed to ColumnarSerDe in previous versions. What
is the reason behind a change? This seems like a regression bug to me.
Normally, we can work around the issue by explicitly setting the table serde to
be org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe. However, this causes a
problem for our migration to ORC. Specifically, we have a partitioned table for
which we want the new partitions to have locations pointing to ORC partitions,
and the old partitions to have locations pointing to RCFILE partitions.
Moreover, we need the ability to change the location of a partition to point to
RCFILE partition. For this we'd do so by doing SET FILEFORMAT RCFILE. However,
b/c of this serde problem the RCFile partition in an ORC table will have the
wrong serde, and ALTER TABLE doesn't allow us to set serde for a partition.
> Default serde for RCFile has changed
> ------------------------------------
>
> Key: HIVE-6210
> URL: https://issues.apache.org/jira/browse/HIVE-6210
> Project: Hive
> Issue Type: Bug
> Components: File Formats
> Affects Versions: 0.12.0
> Reporter: Eric Chu
>
> In Hive 10 when I create a table in RCFile, the serde is
> org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe
> In Hive 12 when I do the same thing, the serde becomes
> org.apache.hadoop.hive.serde2.columnar.LazyBinaryColumnarSerDe
> Similarly, in Hive 12, when I set FILEFORMAT to RCFILE, the serde will become
> LazyBinaryColumnarSerDe, as opposed to ColumnarSerDe in previous versions.
> What is the reason behind this change? This seems like a regression bug to me.
> Normally, we can work around the issue by explicitly setting the table serde
> to be org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe. However, this
> causes a problem for our migration to ORC. Specifically, we have a
> partitioned table for which we want the new partitions to have locations
> pointing to ORC partitions, and the old partitions to have locations pointing
> to RCFILE partitions. Moreover, we need the ability to change the location of
> a partition to point to RCFILE partition. For this we'd do so by doing SET
> FILEFORMAT RCFILE. However, b/c of this serde problem the RCFile partition in
> an ORC table will have the wrong serde, and ALTER TABLE doesn't allow us to
> set serde for a partition.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
