----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37156/#review94354 -----------------------------------------------------------
Ship it! ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java (line 2667) <https://reviews.apache.org/r/37156/#comment148929> Add brief comment on what this function is doing. ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java (line 2672) <https://reviews.apache.org/r/37156/#comment148931> Might be clearer to move this line to the top of the function and just have something like: // Check if FileSystems are different if (srcFs.getClass().equals(destFs.getClass())) { return false; } // Check encryption zones ... Also - we might also need to a move not just if the file system type if different (that works for fixing S3) but also if the file system authority is different. For examople, you have a src destination on cluster1, destination on cluster 2 but both are HDFS. Maybe add a TODO to think about this more? - Lenni Kuff On Aug. 6, 2015, 1:32 a.m., Szehon Ho wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/37156/ > ----------------------------------------------------------- > > (Updated Aug. 6, 2015, 1:32 a.m.) > > > Review request for hive. > > > Bugs: HIVE-7476 > https://issues.apache.org/jira/browse/HIVE-7476 > > > Repository: hive-git > > > Description > ------- > > Currently, CTAS is broken when target is on S3 and source tables are not, or > more generally, where source and target tables are on different file systems. > > > Mainly the issues was that during the Move operation (last stage of CTAS), it > was using the destination FileSystem object to run the operations on both the > source/dest files, thus error when running on a source. The fix is to use > the source FileSystem to run operations on the source file, and the dest > FileSystem to run operations on the dest File. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java 0a466e4 > ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 5840802 > > Diff: https://reviews.apache.org/r/37156/diff/ > > > Testing > ------- > > Manually ran CTAS to create a table on S3. > > > Thanks, > > Szehon Ho > >
