> On June 3, 2016, 12:42 a.m., Sergey Shelukhin wrote: > > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java, > > line 2357 > > <https://reviews.apache.org/r/48159/diff/2/?file=1405580#file1405580line2357> > > > > cleanup code needs to be updated below. As far as I can tell, when this > > throws there can be completed operations yet to be added to the set, unless > > they are decisively terminated or allowed to finish > > Rajesh Balamohan wrote: > Same approach is followed as in earlier case (where it was all or none > gets updated). Plz let me know if i am missing anything here. > > Sergey Shelukhin wrote: > addedPartitions set is used to determine which directories to delete on > clean up. > In the patch, the access to addedPartitions itself doesn't appear to be > thread safe. If it was, there's a question of tasks that are somewhere > between creating the directory and adding to the map. > Actually, I don't think addedPartitions.put inside the callable is thread > safe.
Agreed. Fixed this in latest patch. In corner case, it would end up invoking deleteDir which isn't created yet, which is not harmful. - Rajesh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/48159/#review136018 ----------------------------------------------------------- On June 3, 2016, 3:03 a.m., Rajesh Balamohan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/48159/ > ----------------------------------------------------------- > > (Updated June 3, 2016, 3:03 a.m.) > > > Review request for hive and Ashutosh Chauhan. > > > Bugs: HIVE-13901 > https://issues.apache.org/jira/browse/HIVE-13901 > > > Repository: hive-git > > > Description > ------- > > Depending on FS, creating external tables & adding partitions can be > expensive (e.g msck which adds all partitions). > > > Diffs > ----- > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9cc8fbe > metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java > 94dd72e > > Diff: https://reviews.apache.org/r/48159/diff/ > > > Testing > ------- > > > Thanks, > > Rajesh Balamohan > >
