[
https://issues.apache.org/jira/browse/HIVE-14509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Wagner updated HIVE-14509:
-------------------------------
Attachment: avro-tinyint.demo.patch
Here's a patch for a q test which demonstrates the problem. The flow goes like
this:
* AvroSerde is initialized with the right columns
* Avro doesn't have a tinyint type, so it is stored as int. Internally, the
AvroSerde translates the column types to an Avro schema, then generates an OI
from that. This OI then has a IntegerColumnInspector in it instead of a
TinyintColumnInspector.
* Table.getColsInternal gets to this section:
{code}
try {
// Do the lightweight check for general case.
if (hasMetastoreBasedSchema(SessionState.getSessionConf(),
serializationLib)) {
return tTable.getSd().getCols();
} else if (forMs && !shouldStoreFieldsInMetastore(
SessionState.getSessionConf(), serializationLib,
tTable.getParameters())) {
return Hive.getFieldsFromDeserializerForMsStorage(this,
getDeserializer());
} else {
return MetaStoreUtils.getFieldsFromDeserializer(getTableName(),
getDeserializer());
}
{code}
which dutifully sets the columns according to the OI returned by the Serde.
> AvroSerde mutates tinyint and smallint columns when specifying native columns
> -----------------------------------------------------------------------------
>
> Key: HIVE-14509
> URL: https://issues.apache.org/jira/browse/HIVE-14509
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 2.2.0
> Reporter: Mark Wagner
> Attachments: avro-tinyint.demo.patch
>
>
> tinyint and smallint go in, int comes out:
> {noformat}
> string1 string
> int1 int
> tinyint1 int
> smallint1 int
> bigint1 bigint
> boolean1 boolean
> float1 float
> double1 double
> list1 array<string>
> map1 map<string,int>
> struct1 struct<sint:int,sboolean:boolean,sstring:string>
>
> enum1 string
> nullableint int
> bytes1 binary
> fixed1 binary
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
