Re: Schema evolution for parquet table

2015-10-09 Thread Sergio Pena
Thanks Mohammad. I reviewed the HIVE-6784 patch and it has performance penalties because the way is implemented. I think we can do the promotion in a different way to avoid such penalties. If you take a look at ParquetStringInspector.java, this class gets a String value from different string writ

Re: Schema evolution for parquet table

2015-10-07 Thread Mohammad Islam
Hi Sergio, Thanks for your reply. I found one such effort : https://issues.apache.org/jira/browse/HIVE-6784 I consider to try it differently. Similar stuffs in ORC: https://issues.apache.org/jira/browse/HIVE-10591 Regards, Mohammad On Wednesday, October 7, 2015 8:58 AM, Sergio Pena wrote

Re: Schema evolution for parquet table

2015-10-07 Thread Sergio Pena
Hi Mohammad, Currently, Hive + Parquet does not support auto casting for wider types. That is be a very good idea to implement in Hive. I'll investigate the hive + parquet code, and see if it is something we can add in a future release. - Sergio On Tue, Oct 6, 2015 at 7:23 PM, Mohammad Islam w

Re: Schema evolution for parquet table

2015-10-06 Thread Mohammad Islam
Any hive+parquet user/dev to address this? Regards, Mohammad On Monday, October 5, 2015 3:41 PM, Mohammad Islam wrote: Hi, Does the parquet table support auto casting to wider data types? For example, If I have a parquet table where some parquet data files which have "int" as data type a