Hi, guys,
I need to extend some array UDF in our project. and I designed a
GenericUDF. However, it could not work as I expect.
here is the clause for creating table:
CREATE EXTERNAL TABLE doc PARTITIONED BY (doctype string)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.thrift.ThriftSerDe'
WITH SERDEPROPERTIES (
"serialization.class" = "com.test.proto.Doc",
"serialization.format" = "org.apache.thrift.protocol.TBinaryProtocol")
I think I had specified the necessary param. it works well for any other
hql stats.
but when my udf ran, the serde altered, to LazySimpleSerde. here is
output snippet of job tracker:
Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Can
not set java.util.List field com.test.proto.doc.participant to java.lang.String
at
org.apache.hadoop.hive.serde2.objectinspector.ReflectionStructObjectInspector.getStructFieldsDataAsList(ReflectionStructObjectInspector.java:160)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:492)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serializeField(LazySimpleSerDe.java:396)
at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:383)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:564)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.LimitOperator.processOp(LimitOperator.java:47)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:78)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
any clue?
BR.
-Bing
