Yuming Wang created HIVE-24436:
----------------------------------
Summary: Fix Avro NULL_DEFAULT_VALUE compatibility issue
Key: HIVE-24436
URL: https://issues.apache.org/jira/browse/HIVE-24436
Project: Hive
Issue Type: Improvement
Components: Avro
Affects Versions: 2.3.8
Reporter: Yuming Wang
Exception1:
{noformat}
- create hive serde table with Catalog
*** RUN ABORTED ***
java.lang.NoSuchMethodError: 'void
org.apache.avro.Schema$Field.<init>(java.lang.String, org.apache.avro.Schema,
java.lang.String, org.codehaus.jackson.JsonNode)'
at
org.apache.hadoop.hive.serde2.avro.TypeInfoToSchema.createAvroField(TypeInfoToSchema.java:76)
at
org.apache.hadoop.hive.serde2.avro.TypeInfoToSchema.convert(TypeInfoToSchema.java:61)
at
org.apache.hadoop.hive.serde2.avro.AvroSerDe.getSchemaFromCols(AvroSerDe.java:170)
at org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:114)
at org.apache.hadoop.hive.serde2.avro.AvroSerDe.initialize(AvroSerDe.java:83)
at
org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:533)
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:450)
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:437)
at
org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:281)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:263)
{noformat}
Exception2:
{noformat}
- alter hive serde table add columns -- partitioned - AVRO *** FAILED ***
org.apache.spark.sql.AnalysisException:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.avro.AvroRuntimeException: Unknown datum class: class
org.codehaus.jackson.node.NullNode;
at
org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:112)
at
org.apache.spark.sql.hive.HiveExternalCatalog.createTable(HiveExternalCatalog.scala:245)
at
org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.createTable(ExternalCatalogWithListener.scala:94)
at
org.apache.spark.sql.catalyst.catalog.SessionCatalog.createTable(SessionCatalog.scala:346)
at
org.apache.spark.sql.execution.command.CreateTableCommand.run(tables.scala:166)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
at org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:228)
at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:3680)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
