----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55661/#review162184 -----------------------------------------------------------
Can this be considered a S3 performance only? or is it fine to have many RPC calls in case the path is on HDFS? ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java (line 2992) <https://reviews.apache.org/r/55661/#comment233436> 1. what happens if numThread is a negative number? 2. do we need to execute a threadpool if numThreads is just 1? ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java (line 3016) <https://reviews.apache.org/r/55661/#comment233439> Can this happen? I followed the workflow when file is null, and found that createDummyFileForEmptyPartition() may fail when attempts to call path.toString(). Is that right? What can we do to avoid a NPE there? - Sergio Pena On Jan. 18, 2017, 3:17 a.m., Sahil Takiar wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55661/ > ----------------------------------------------------------- > > (Updated Jan. 18, 2017, 3:17 a.m.) > > > Review request for hive and Sergio Pena. > > > Bugs: HIVE-15546 > https://issues.apache.org/jira/browse/HIVE-15546 > > > Repository: hive-git > > > Description > ------- > > HIVE-15546: Optimize Utilities.getInputPaths() so each listStatus of a > partition is done in parallel > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 0161c20 > ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java bd067aa > > Diff: https://reviews.apache.org/r/55661/diff/ > > > Testing > ------- > > Unit tests added > > > Thanks, > > Sahil Takiar > >
