Re: adding parquet partitions to external table (msck repair table not working)

2019-01-15 Thread Colin Williams
Hello, Sorry for the late response. Just got back from vacation. Regarding Hive version: 2.3.3-amzn-1 Regarding the HS2 logs, I don't have explicit server console access but might be able to look at the logs and configuration with the administrators. I've just implemented the manual alter table

Re: adding parquet partitions to external table (msck repair table not working)

2019-01-02 Thread Vihang Karajgaonkar
What version of Hive are you using? You should look at the HS2 logs to see if there were any errors from msck command which ignored such partitions. Also, would be worth to take a look at hive.msck.path.validation configuration in case it is set to "ignore" which silently ignores invalidate partiti

adding parquet partitions to external table (msck repair table not working)

2018-12-30 Thread Colin Williams
Hello, I have a parquet datasource that I'm writing via spark. For some reason this particular source will not pick up added partitions with msck repair table. However if I alter table tablename / add partition (key=value) then it works. However this is more cumbersome than msck repair table. Is