> On Sept. 25, 2015, 6:55 p.m., John Pullokkaran wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java, > > line 316 > > <https://reviews.apache.org/r/38768/diff/1/?file=1084857#file1084857line316> > > > > Also why is this only at the else condition? > > Are we assuming that if the requested cols & returned cols size matches > > then order is preserved. > > > > This seems rather shaky assumptions. > > Its better to fix stats API to guarantee the order. > > Chaoyu Tang wrote: > No, I did not assume that. The reasons I put the logic in this else > conditiion are because: > 1. only when the requested cols & returned cols size matches, the stats > is mapped to an index and put into the hiveColStatsMap, see following code in > updateColStats (line 363 -367) > == > if (hiveColStats != null && hiveColStats.size() == > nonPartColNamesThatRqrStats.size()) { > for (int i = 0; i < hiveColStats.size(); i++) { > // the columns in > nonPartColIndxsThatRqrStats/nonPartColNamesThatRqrStats/hiveColStats > // are in same order > hiveColStatsMap.put(nonPartColIndxsThatRqrStats.get(i), > hiveColStats.get(i)); > } > } > == > > 2. For partitioned table, the hiveColStats is already built in the order > of nonPartColNamesThatRqrStats. There is not necessary to reorder it. > == > for (String c : nonPartColNamesThatRqrStats) { > ColStatistics cs = stats.getColumnStatisticsFromColName(c); > if (cs != null) { > hiveColStats.add(cs); > } else { > colNamesFailedStats.add(c); > } > } > == > > 3. HMS stats API might not be a proper place to enforce the order as I > explained in last comment.
ok - John ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38768/#review100655 ----------------------------------------------------------- On Sept. 25, 2015, 6:37 p.m., Chaoyu Tang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/38768/ > ----------------------------------------------------------- > > (Updated Sept. 25, 2015, 6:37 p.m.) > > > Review request for hive. > > > Bugs: HIVE-11964 > https://issues.apache.org/jira/browse/HIVE-11964 > > > Repository: hive-git > > > Description > ------- > > The stats in hiveColStats should be ordered in that of the columns in > nonPartColIndxsThatRqrStats/nonPartColNamesThatRqrStats before being put in > the hiveColStatsMap > === > for (int i = 0; i < hiveColStats.size(); i++) { > hiveColStatsMap.put(nonPartColIndxsThatRqrStats.get(i), > hiveColStats.get(i)); > } > > Since hiveColStats = StatsUtils.getTableColumnStats(hiveTblMetadata, > hiveNonPartitionCols, nonPartColNamesThatRqrStats); might not return the > stats in same order as those in nonPartColNamesThatRqrStats. > > > Diffs > ----- > > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java > 6c0bd25 > > Diff: https://reviews.apache.org/r/38768/diff/ > > > Testing > ------- > > Precommit tests > > > Thanks, > > Chaoyu Tang > >
