----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68891/#review209340 -----------------------------------------------------------
src/org/apache/pig/backend/hadoop/executionengine/tez/util/TezInputHelper.java Line 51 (original), 49 (patched) <https://reviews.apache.org/r/68891/#comment293663> * This method creates input splits similar to * {@link org.apache.tez.mapreduce.hadoop.MRInputHelpers#generateInputSplitsToMem} * but only does it for mapreduce API and does not do grouping of splits or create * {@link org.apache.tez.mapreduce.protos.MRRuntimeProtos.MRSplitsProto} * which is an expensive operation. src/org/apache/pig/backend/hadoop/executionengine/tez/util/TezInputHelper.java Lines 56-60 (original), 54-58 (patched) <https://reviews.apache.org/r/68891/#comment293659> To be removed as we don't do groups src/org/apache/pig/backend/hadoop/executionengine/tez/util/TezInputHelper.java Line 62 (original), 60 (patched) <https://reviews.apache.org/r/68891/#comment293660> To be removed as we only do mapreduce which is used to determine * whether the mapred of mapreduce API is being used src/org/apache/pig/backend/hadoop/executionengine/tez/util/TezInputHelper.java Line 144 (original), 139 (patched) <https://reviews.apache.org/r/68891/#comment293664> new ArrayList<>(newFormatSplits.length); src/org/apache/pig/backend/hadoop/executionengine/tez/util/TezJobSplitWriter.java Lines 137 (patched) <https://reviews.apache.org/r/68891/#comment293666> serialize -> serialized - Rohini Palaniswamy On Oct. 8, 2018, 10:39 p.m., Satish Saley wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68891/ > ----------------------------------------------------------- > > (Updated Oct. 8, 2018, 10:39 p.m.) > > > Review request for pig. > > > Repository: pig-git > > > Description > ------- > > [PIG-5359] Reduce time spent in split serialization > > > Diffs > ----- > > src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java > f292487f0 > > src/org/apache/pig/backend/hadoop/executionengine/tez/plan/optimizer/LoaderProcessor.java > 7a12df784 > > src/org/apache/pig/backend/hadoop/executionengine/tez/util/MRToTezHelper.java > b604d9f18 > > src/org/apache/pig/backend/hadoop/executionengine/tez/util/SerializationInfo.java > PRE-CREATION > > src/org/apache/pig/backend/hadoop/executionengine/tez/util/TezInputHelper.java > PRE-CREATION > > src/org/apache/pig/backend/hadoop/executionengine/tez/util/TezJobSplitWriter.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/68891/diff/2/ > > > Testing > ------- > > > Thanks, > > Satish Saley > >
