> On Dec. 13, 2017, 10:40 p.m., Prasanth_J wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java > > Lines 790 (patched) > > <https://reviews.apache.org/r/64324/diff/4/?file=1912930#file1912930line806> > > > > This exclusion of LLAP jars happen only for task localization or even > > for AM. > > If for AM as well, then AM would need hive-llap-tez and > > hadoop-yarn-registry jars right?
It already has them as part of the epic jar I think. The logic w.r.t. these jars is not changed, I just added a comment. > On Dec. 13, 2017, 10:40 p.m., Prasanth_J wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java > > Lines 1525 (patched) > > <https://reviews.apache.org/r/64324/diff/4/?file=1912930#file1912930line1548> > > > > even in uber mode? This is moved code, old comment > On Dec. 13, 2017, 10:40 p.m., Prasanth_J wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java > > Line 471 (original), 459 (patched) > > <https://reviews.apache.org/r/64324/diff/4/?file=1912932#file1912932line473> > > > > Why is the boolean flipped? If its reopen do we need to recreate tez > > scratch dirs? Not anymore; the Hive files directory is "taken away" from the session before close, via HiveResources. The Tez scratchdir doesn't actually need to be preserved, it was only preserved for Hive jars. open will re-create the scratchdir if needed > On Dec. 13, 2017, 10:40 p.m., Prasanth_J wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java > > Line 241 (original), 236 (patched) > > <https://reviews.apache.org/r/64324/diff/4/?file=1912934#file1912934line251> > > > > Handle duplicates in additionalFilesNotFromConf since Set<String> is > > now moved to String[]? Not sure what you mean? > On Dec. 13, 2017, 10:40 p.m., Prasanth_J wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java > > Line 287 (original), 282 (patched) > > <https://reviews.apache.org/r/64324/diff/4/?file=1912934#file1912934line299> > > > > nit: RegistryOperations.class? Will check if that is easy to change. This is existing code that used string for some reason. > On Dec. 13, 2017, 10:40 p.m., Prasanth_J wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java > > Line 578 (original), 575 (patched) > > <https://reviews.apache.org/r/64324/diff/4/?file=1912934#file1912934line594> > > > > Question: > > For jars, does it track the file name or file name + sha? If this is > > just file names how does it refresh when the sha changes? File name only for non-exec jar. The files are specified as local files and not shared, so I think there's no expectation of consistency if local files change during process lifetime, behavior is undefined (reopen is also not a defined away). Exec jar uses sha because it's shared between multiple users. - Sergey ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64324/#review193743 ----------------------------------------------------------- On Dec. 12, 2017, 2:10 a.m., Sergey Shelukhin wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64324/ > ----------------------------------------------------------- > > (Updated Dec. 12, 2017, 2:10 a.m.) > > > Review request for hive, Prasanth_J and Siddharth Seth. > > > Repository: hive-git > > > Description > ------- > > see jira > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 88a75edd35 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 6c1afa6555 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPool.java > 3bcf657ac4 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java > 8417ebb7d5 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolSession.java > b3ccd24fd6 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java > dd879fc5e8 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java 8795cfcee1 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java > dbdbbf25db > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezJobMonitor.java > 9726af1506 > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java > 4148a8aa3a > ql/src/test/org/apache/hadoop/hive/ql/exec/tez/SampleTezSessionState.java > 52484540ff > ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezSessionPool.java > 829ea8cecc > ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezTask.java 47aa936845 > ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java > c58e4507f2 > > > Diff: https://reviews.apache.org/r/64324/diff/4/ > > > Testing > ------- > > > Thanks, > > Sergey Shelukhin > >
