Does that mean I need to override UDTFExplode ?
In that case, every nested struct /list serde needs to implement this ?
Correct if I m wrong
-Sagar
On May 13, 2011, at 6:51 AM, Edward Capriolo wrote:
> You must have a mistake in your object inspector. I am doing something
> similar with nested structs in arrays. This is a struct that has a few
> top level properties, and then one property is a list of structs one
> level of nesting only.
>
> public ObjectInspector initialize(ObjectInspector[] arguments)
> throws UDFArgumentException {
> ret = new Object[2];
> oi=arguments;
>
> List<String> fieldNames = new ArrayList<String>();
> List<ObjectInspector> fieldInspectors = new
> ArrayList<ObjectInspector>();
>
>
> List<String> structFieldNames = new ArrayList<String>();
> List<ObjectInspector> structFieldInspectors = new
> ArrayList<ObjectInspector>();
> structFieldNames.add("id");
> structFieldNames.add("acount");
> structFieldNames.add("bcount");
>
>
> structFieldInspectors.add(PrimitiveObjectInspectorFactory.javaLongObjectInspector);
>
> structFieldInspectors.add(PrimitiveObjectInspectorFactory.javaLongObjectInspector);
>
> structFieldInspectors.add(PrimitiveObjectInspectorFactory.javaLongObjectInspector);
>
> fieldNames.add("redirectList");
> fieldInspectors.add(
>
> ObjectInspectorFactory.getStandardListObjectInspector(
>
> ObjectInspectorFactory.getStandardStructObjectInspector(structFieldNames,
> structFieldInspectors)
> )
> );
>
> //sl.getRecordCount()
> fieldNames.add("cCount");
>
> fieldInspectors.add(PrimitiveObjectInspectorFactory.javaLongObjectInspector);
>
> StructObjectInspector soi =
> ObjectInspectorFactory.getStandardStructObjectInspector(fieldNames,
> fieldInspectors);
> return soi;
>
> I hope this helps.
>
> On Thu, May 12, 2011 at 9:45 PM, Sagar Naik <sn...@attributor.com> wrote:
>> Hi ,
>>
>> I have a four column external table.
>> One of the col is array of struct of five fields.
>> I use a custom serde, to read out hadoop seq file and populate values on to
>> hive row.
>> The array of struct is ArrayList of Object[5].
>>
>> All queries run fine.
>> Except the explode query which gives a class cast exception and saying
>> object array is being casted as string
>>
>>
>> I dont knw wht is the issue, but the other serde along same lines works
>> super fine.
>>
>> pl guide me
>>
>>
>> at
>> org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaStringObjectInspector.getPrimitiveWritableObject(JavaStringObjectInspector.java:35)
>> at
>> org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:206)
>> at
>> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:480)
>> at
>> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:426)
>> at
>> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:555)
>> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
>> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
>> at
>> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
>> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
>> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
>> at
>> org.apache.hadoop.hive.ql.exec.LateralViewJoinOperator.processOp(LateralViewJoinOperator.java:133)
>> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:457)
>> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:697)
>> at
>> org.apache.hadoop.hive.ql.exec.UDTFOperator.forwardUDTFOutput(UDTFOperator.java:111)
>> at
>> org.apache.hadoop.hive.ql.udf.generic.UDTFCollector.collect(UDTFCollector.java:44)
>> at
>> org.apache.hadoop.hive.ql.udf.generic.GenericUDTF.forward(GenericUDTF.java:82)
>> at
>> org.apache.hadoop.hive.ql.udf.generic.GenericUDTFExplode.process(GenericUDTFExplode.java:75)
>>
>>
>>
>> -Sagar
