Hi,
I am a software engineer from China. I’m learning hive now.
These days I encounter a problem, when I update one of the columns in
a table, such as ALTER TABLE cl_Play CHANGE `TVSection` `TVSection` String;
The raw type of TVSection is
array<struct<dim_sectionid:int,sectionid:int,title:string>>
After the type changed, I execute hql(select count(1),dt from cl_play where
dt=131202 and hour=17 group by dt ), an exception is thrown below:
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazyListObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:136)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.<init>(ObjectInspectorConverters.java:301)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:138)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:274)
at
org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:486)
I find out it caused by the different type of tvsection between table level
metadata and partition level metadata. The partition level metadata in
column_v2 keeps the raw type of tvsection with array type.
It confused me a lot. Why do we design like this? Is there a solution update
the table metadata and table’s partition metadata at the same time?
Best Regards.
Jin.
