Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review180895 --- Ship it! Ship It! - kelly zhang On July 18, 2017, 11:38 a.m.

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

> On July 18, 2017, 7:16 a.m., kelly zhang wrote: > > src/org/apache/pig/backend/hadoop/executionengine/spark/Spark1Shims.java > > Line 133 (original), 133 (patched) > > > > > > How about change > > org.apache.p

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated July 18, 2017, 11:38 a.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review180799 --- src/org/apache/pig/backend/hadoop/executionengine/spark/Spark1Shi

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated July 17, 2017, 3:01 p.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review180657 --- Fix it then Ship it src/org/apache/pig/backend/hadoop/executione

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 28, 2017, 8:55 a.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 27, 2017, 3:33 p.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 23, 2017, 12:18 p.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 23, 2017, 9:49 a.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review178716 --- Ship it! Ship It! - Rohini Palaniswamy On June 22, 2017, 2:3

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 22, 2017, 2:38 p.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 22, 2017, 2:07 p.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

> On June 20, 2017, 8:43 p.m., Rohini Palaniswamy wrote: > > build.xml > > Line 795 (original), 795 (patched) > > > > > > Instead of changing all references of jar to jar-simple, you can just > > compile Spark 1 a

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 21, 2017, 9:06 p.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review178421 --- build.xml Line 795 (original), 795 (patched)

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review178236 --- Ship it! Ship It! - Adam Szita On June 19, 2017, 9:41 a.m.,

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

> On June 16, 2017, 9:33 p.m., Rohini Palaniswamy wrote: > > build.xml > > Lines 795 (patched) > > > > > > Can you rename the current "jar" target to "jar-simple" and actually > > make this one part of the jar tar

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

> On June 16, 2017, 9:33 p.m., Rohini Palaniswamy wrote: > > build.xml > > Lines 795 (patched) > > > > > > Can you rename the current "jar" target to "jar-simple" and actually > > make this one part of the jar tar

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

> On June 15, 2017, 10:18 a.m., Adam Szita wrote: > > build.xml > > Lines 795 (patched) > > > > > > This will leave the jar inside build directory having Spark2 classes > > only. We should fix that, so there is no d

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 19, 2017, 9:41 a.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review178126 --- Looks good. Just minor changes for class names and ant target. b

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review177990 --- build.xml Lines 245 (patched)

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 14, 2017, 9:13 a.m.) Review request for pig, liyun zhang, Rohini

RE: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

: Tuesday, June 13, 2017 3:52 PM To: Rohini Palaniswamy ; Zhang, Liyun ; Adam Szita Cc: Zhang, Liyun ; pig ; Nandor Kollar Subject: Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0 > On June 13, 2017, 7:33 a.m., kelly zhang wrote: > > src/org/apache/pig/tools/pigst

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

> On June 13, 2017, 7:33 a.m., kelly zhang wrote: > > src/org/apache/pig/backend/hadoop/executionengine/spark/JobMetricsListener.java > > Line 82 (original), 83 (patched) > > > > > > I guess in spark2, the comment in

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

> On June 13, 2017, 7:33 a.m., kelly zhang wrote: > > src/org/apache/pig/tools/pigstats/spark/SparkJobStats1.java > > Lines 60-63 (patched) > > > > > > why > > inputMetricExists,outputMetricExist,shuffleReadMetricExi

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review177699 --- src/org/apache/pig/backend/hadoop/executionengine/spark/JobMetric

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 12, 2017, 9:20 p.m.) Review request for pig, liyun zhang, Rohini

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated June 2, 2017, 3:20 p.m.) Review request for pig, liyun zhang, Rohini P

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review176232 --- Ship it! Ship It! - Rohini Palaniswamy On May 25, 2017, 7:13

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/ --- (Updated May 25, 2017, 7:13 a.m.) Review request for pig, liyun zhang, Rohini P

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

> On May 24, 2017, 9:21 p.m., Rohini Palaniswamy wrote: > > build.xml > > Lines 251 (patched) > > > > > > Can just load one property file for spark2, leaving the spark 1.6 > > version in libraries.properties AFAIK

Re: Review Request 59530: PIG-5157 Upgrade to Spark 2.0

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59530/#review175998 --- build.xml Lines 243 (patched)