> On Sept. 12, 2017, 6:18 a.m., Ashutosh Chauhan wrote: > > metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java > > Lines 275 (patched) > > <https://reviews.apache.org/r/62228/diff/1/?file=1819749#file1819749line283> > > > > Here we are listing all partitions for table and than we immediately > > aggr stats for all partitions. Another (better) way is to not retrieve > > partNames and do a sql query to aggr stats for partitions by > > partFilterExpr. Essentially get_aggr_stats_for(dbName, tblName, > > partFilterExpr). > > Here, partFilterExpr = * > > That will allow many roundtrips to backend DB.
I'll create a follow-up jira for this (near term item) if you're ok with it. > On Sept. 12, 2017, 6:18 a.m., Ashutosh Chauhan wrote: > > metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java > > Lines 295 (patched) > > <https://reviews.apache.org/r/62228/diff/1/?file=1819749#file1819749line303> > > > > And here it will be partFilterExpr = partNames not in (defaultPartition) I'll create a follow-up jira for this (near term item) if you're ok with it. > On Sept. 12, 2017, 6:18 a.m., Ashutosh Chauhan wrote: > > metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java > > Line 1536 (original), 1570 (patched) > > <https://reviews.apache.org/r/62228/diff/1/?file=1819749#file1819749line1578> > > > > This if condition will always be true for cachedstore prewarm > > invocation. > > can you please add comments for that. Modified prewarm to use the mergeColStatsForPartitions call directly which would also avoid the extra cache lookup we were doing in prewarm. - Vaibhav ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62228/#review185136 ----------------------------------------------------------- On Sept. 11, 2017, 9:25 p.m., Vaibhav Gumashta wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62228/ > ----------------------------------------------------------- > > (Updated Sept. 11, 2017, 9:25 p.m.) > > > Review request for hive, Ashutosh Chauhan and Thejas Nair. > > > Bugs: HIVE-17495 > https://issues.apache.org/jira/browse/HIVE-17495 > > > Repository: hive-git > > > Description > ------- > > https://issues.apache.org/jira/browse/HIVE-17495 > > > Diffs > ----- > > > itests/hcatalog-unit/src/test/java/org/apache/hive/hcatalog/listener/DummyRawStoreFailEvent.java > 8d861e4 > metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java > dc1245e > metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java > bbe13fd > metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java > 3053dcb > metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 71982a0 > metastore/src/java/org/apache/hadoop/hive/metastore/cache/CachedStore.java > 3ba81ce > metastore/src/java/org/apache/hadoop/hive/metastore/cache/SharedCache.java > 80b17e0 > > metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/BinaryColumnStatsAggregator.java > e6c836b > > metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/BooleanColumnStatsAggregator.java > a34bc9f > > metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/ColumnStatsAggregator.java > a52e5e5 > > metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/ColumnStatsAggregatorFactory.java > dfae708 > > metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DateColumnStatsAggregator.java > ee95396 > > metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DecimalColumnStatsAggregator.java > 284c12c > > metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/DoubleColumnStatsAggregator.java > bb4a725 > > metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/LongColumnStatsAggregator.java > 5b1145e > > metastore/src/java/org/apache/hadoop/hive/metastore/columnstats/aggr/StringColumnStatsAggregator.java > 1b29f92 > > metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java > 4db203d > > metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java > fb16cfc > > > Diff: https://reviews.apache.org/r/62228/diff/1/ > > > Testing > ------- > > > Thanks, > > Vaibhav Gumashta > >
