[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37570651 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13162/ --- If your project

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37570650 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10576136 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -222,4 +232,19 @@ private[spark] object HadoopRDD { def putCachedMetadat

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10576150 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -222,4 +232,19 @@ private[spark] object HadoopRDD { def putCachedMetadat

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37563131 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10576332 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -222,4 +232,19 @@ private[spark] object HadoopRDD { def putCachedMetada

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37563133 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10576162 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -222,4 +232,19 @@ private[spark] object HadoopRDD { def putCachedMetadat

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10576154 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -222,4 +232,19 @@ private[spark] object HadoopRDD { def putCachedMetadat

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37558032 Hi, @aarondav , thank you very much for the comments, I think it's ready for the further review --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37547667 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13161/ --- If your project

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37547666 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37540078 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37540073 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37499630 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37499631 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13147/ --- If your project

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37499281 Thanks for updating this, I think changing the package is a very good cleanup. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10552386 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -222,4 +232,27 @@ private[spark] object HadoopRDD { def putCachedMetadat

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10552362 --- Diff: core/src/main/scala/org/apache/spark/SparkHadoopWriter.scala --- @@ -15,18 +15,19 @@ * limitations under the License. */ -packa

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10552358 --- Diff: core/src/main/scala/org/apache/spark/SparkHadoopWriter.scala --- @@ -15,18 +15,19 @@ * limitations under the License. */ -packa

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10552253 --- Diff: core/src/main/scala/org/apache/spark/SparkHadoopWriter.scala --- @@ -15,18 +15,19 @@ * limitations under the License. */ -packa

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37498220 Hi, @kayousterhout and @aarondav , Thank you for your comments, I addressed them One potential issue is that, to call the function in HadoopRDD, I moved SparkH

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37497545 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37497546 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37494136 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13144/ --- If your p

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37494135 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37493979 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37493980 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10548780 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theS

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10548392 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theSplit

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37485195 @aarondav Thank you for your comments, I will address them --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10548369 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theSplit

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10548362 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theSplit

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10548255 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theSplit

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37480611 Thanks, this looks like a good bug fix. I just had some questions since I'm not super familiar with the nature of these Hadoop JobIds and such, and some minor stylistic c

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10546476 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theSplit:

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10546480 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theSplit:

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10546472 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theSplit:

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10546478 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theSplit:

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10546468 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -34,6 +39,7 @@ import org.apache.spark.broadcast.Broadcast import org.apache.spa

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10546475 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -165,12 +174,29 @@ class HadoopRDD[K, V]( override def compute(theSplit:

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10546469 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -111,6 +117,9 @@ class HadoopRDD[K, V]( protected val inputFormatCacheKe

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-12 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37451940 ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-09 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37150695 anyone would like to review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37100326 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37100327 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13072/ --- If your project

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37098778 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37098779 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37089803 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37089804 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13062/ --- If your project

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37088861 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37088862 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37087799 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13059/ --- If your p

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37087798 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37086647 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37086648 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37084833 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13056/ --- If your project

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37084832 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37082402 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37082403 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-07 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/spark/pull/101 SPARK-1128: set hadoop task properties when constructing HadoopRDD The task properties are set when constructing HadoopRDD in current implementation, this may limit the implementation based on