> On Feb. 3, 2017, 2:42 p.m., Aihua Xu wrote: > > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, > > line 1679 > > <https://reviews.apache.org/r/56118/diff/2/?file=1623348#file1623348line1679> > > > > With this approach, I feel it will place Hive in worse situation since > > now it could fail during the data deletion and it failed, metastore is > > rolled back but part of the data is removed. The table is in unusable > > conditition. > > > > Without this change, we roll back when the metadata drop has issues. > > After roll back, the table is still consistent. > > > > If the metadata drop succeeds, then we continue deleting the data. It's > > not that good that the data could not be purged, but it won't cause any > > inconsistency since metadata is gone and the data is not visiable to the > > caller.
Okay, I see, let's stick to removing the metadata as well then. Although I still suggest we _do not return success_ to the client - if some file/dir couldn't have been deleted then the drop_table command clearly didn't succeeded entirely and the user should know about this. So I think the exception throwing changes should be kept (along with the configurability of this strict mode). Also, it should be logged additionally what path could not have been deleted, and what (partition) paths remained on disk because of this. This all will ultimately give the user: (1) notification that due to an error some data is remained; (2) what paths have to be considered for retrying to delete them manually (can't retry from Hive itself, since we drop the metadata regardless). So - having much more experience in Hive, what do you think about this approach? - Adam ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56118/#review164121 ----------------------------------------------------------- On Feb. 3, 2017, 2:22 p.m., Adam Szita wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56118/ > ----------------------------------------------------------- > > (Updated Feb. 3, 2017, 2:22 p.m.) > > > Review request for hive, Aihua Xu, Peter Vary, and Sergio Pena. > > > Bugs: HIVE-14181 > https://issues.apache.org/jira/browse/HIVE-14181 > > > Repository: hive-git > > > Description > ------- > > Failure during table drop doesn't throw errors and results in success - some > times data resides in warehouse, but table (meta data) is removed from > metastore resulting in incosistency > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java > 53b9b0c6962c9b1cd2eef1cb71687ec0245cfac3 > > itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java > af125c38236582ba532f5e3de3d2ba724f38b101 > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java > f8c3c4e48db0df9d6c18801bcd61f9e5dc6eb7c2 > > Diff: https://reviews.apache.org/r/56118/diff/ > > > Testing > ------- > > -Added test case > -Tested on cluster > > > Thanks, > > Adam Szita > >
