Thanks Matthew, Currently we are in Hive 1.2 version only, Is there any setting like "hive.metastore.disallow.incompatible.col.type.changes=false;" in Hive 1.2 or any around apart for reloading entire table data. For Quick workaround, we are reloading entire data. Can you please share with us Jira for Schema Evolution.
@Mich : Currently we have only primitive types. But I'm also interested to know "how the behavior will be in complex types" /Mahender On 7/18/2016 3:55 PM, Mich Talebzadeh wrote: Hi Mathew, In layman's term if I create the source ORC table column as INT and then create a target ORC table but that column has now been defined as STRING and do an INSERT/SELECT from source table how data is internally stored? Is it implicitly converted into new format using CAST function or it is stored as is and just masked? The version of Hive I am using is 2 and it works OK for primitive data types (insert/select from INT to String) However, I believe Mahender is referring to Complex types? Thanks Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw http://talebzadehmich.wordpress.com Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On 18 July 2016 at 22:31, Matthew McCline <mmccl...@hortonworks.com<mailto:mmccl...@hortonworks.com>> wrote: Hi Mahender, Schema Evolution is available on the latest recent version of Hive. For example, if you set hive.metastore.disallow.incompatible.col.type.changes=false; on master (i.e. hive2) it will support INT to STRING conversion. If you need to remain on an older version, then you are out of luck. Thanks, Matt ________________________________ From: Mahender Sarangam <mahender.bigd...@outlook.com<mailto:mahender.bigd...@outlook.com>> Sent: Monday, July 18, 2016 1:59 PM To: user@hive.apache.org<mailto:user@hive.apache.org> Subject: Re: ORC does not support type conversion from INT to STRING. Hi Mich, Sorry for delay in responding. here is the scenario, We have created new cluster and we have moved all ORC File data into new cluster. We have re-created table pointing to ORC location. We have modified data type of ORC table from INT to String. From then onward, we were unable to fire select statement against this ORC table, hive keep throwing exception, "Orc table select. Unable to convert Int to String". Looks like it is bug in ORC table only. Where in we modify the datatype from int to string, is causing problem with ORC reading/select statement, it throws exceptio. Please let me know if there are any workaround for this scenario. Is this behavior expected previously also. /Mahender On 6/14/2016 11:47 AM, Mich Talebzadeh wrote: you must excuse my ignorance can you please elaborate on this as there seems something has gone wrong somewhere? Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw http://talebzadehmich.wordpress.com<http://talebzadehmich.wordpress.com/> On 14 June 2016 at 19:42, Mahender Sarangam <mahender.bigd...@outlook.com<mailto:mahender.bigd...@outlook.com>> wrote: Yes Mich. We have restored cluster from metastore. On 6/14/2016 11:35 AM, Mich Talebzadeh wrote: Hi Mahendar, Did you load the meta-data DB/schema from backup and now seeing this error Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw http://talebzadehmich.wordpress.com On 14 June 2016 at 19:04, Mahender Sarangam <mahender.bigd...@outlook.com<mailto:mahender.bigd...@outlook.com>> wrote: ping. On 6/13/2016 1:19 PM, Mahender Sarangam wrote: Hi, We are facing issue while reading data from ORC table. We have created ORC table and dumped data into it. We have deleted cluster due to some reason. When we recreated cluster (using Metastore) and table pointing to same location. When we perform reading from ORC table. We see below error. SELECT col2, Col1, reflect("java.util.UUID", "randomUUID") AS ID, Source, 1 , SDate, EDate FROM Table ORC JOIN Table2 _surr; ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1465411930667_0212_1_01, diagnostics=[Task failed, taskId=task_1465411930667_0212_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: ORC does not support type conversion from INT to STRING. I think issue is reflect("java.util.UUID", "randomUUID") AS ID I know there is Bug raised while reading data from ORC table. Is there any workaround apart from reloading data. -MS
