Thank you Mich, I tried the cast function too, but it still failed with the below exception :
Failed with exception java.io.IOException:java.io.IOException: Unknown encoding kind: DICTIONARY_V2 dictionarySize: 7 in column 219 Table Layout : Old Partition Metadata : Col1 String, Col2 Struct<c1:smallint,c2:string> . New Partition Metadata : Col1 String, Col2 Struct<c1:int,c2:string> . If I query the Col1, It's fetching the data, but getting failed for the Col2. On Sat, Jan 20, 2018 at 5:21 AM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Your Hive metadata reflects the new data type of the column changed from > type A to type B. > > The old partitions have data stored as type A. These have not changed. > > If you run the query on the old partition you would probably need to use > CAST function to change the column from B to A so it is compatible with > what is stored in the old partitions. That I believe will work. > > HTH > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 19 January 2018 at 21:37, Raghuraman Murugaiyan < > ragu.hive.engin...@gmail.com> wrote: > >> Hi All, >> >> I have a table partitioned on a date column. We have changed the data >> type of one of the field for the older partitions and the data type of the >> newer partitions will remain the same as the table. When I tried to run a >> simple Select query on the older partitions , the query is failing with the >> below error : >> >> >> 2018-01-19 16:17:58,766 FATAL [IPC Server handler 23 on 38645] >> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: >> attempt_1497186993127_459726_m_000112_0 - exited : java.io.IOException: >> java.lang.reflect.InvocationTargetException >> at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handle >> RecordReaderCreationException(HiveIOExceptionHandlerChain.java:97) >> at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleR >> ecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57) >> at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRe >> cordReader.initNextRecordReader(HadoopShimsSecure.java:269) >> at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRe >> cordReader.<init>(HadoopShimsSecure.java:216) >> at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileIn >> putFormatShim.getRecordReader(HadoopShimsSecure.java:343) >> at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecor >> dReader(CombineHiveInputFormat.java:681) >> at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>( >> MapTask.java:169) >> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:422) >> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGro >> upInformation.java:1656) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) >> Caused by: java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >> at sun.reflect.NativeConstructorAccessorImpl.newInstance(Native >> ConstructorAccessorImpl.java:62) >> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(De >> legatingConstructorAccessorImpl.java:45) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) >> at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRe >> cordReader.initNextRecordReader(HadoopShimsSecure.java:255) >> ... 11 more >> Caused by: java.io.IOException: Unknown encoding kind: DICTIONARY_V2 >> dictionarySize: 6 >> in column 219 >> >> >> Can you help me debug this error ? I am using Hive 2.2. >> >> Regards, >> Raghu M >> > >
