[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/43#issuecomment-36327170 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12929/ --- If your pr

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/43#issuecomment-36327168 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1134 pyspark only uses ipython if starte...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36326993 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1134 pyspark only uses ipython if starte...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36326995 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12928/ --- If your project i

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/43#discussion_r10156359 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -84,12 +84,27 @@ private class DiskStore(blockManager: BlockManager, diskManager:

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/43#discussion_r10156343 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala --- @@ -146,6 +146,12 @@ object BlockFetcherIterator { }

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/43#discussion_r10156309 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala --- @@ -146,6 +146,12 @@ object BlockFetcherIterator { }

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/43#discussion_r10156300 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala --- @@ -146,6 +146,12 @@ object BlockFetcherIterator { }

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/43#issuecomment-36326084 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/43#issuecomment-36326074 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/43#issuecomment-36326073 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-27 Thread pwendell
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/43 SPARK-1145: Memory mapping with many small blocks can cause JVM allocation failures This includes some minor code clean-up as well. The main change is that small files are not memory mapped. There i

[GitHub] spark pull request: SPARK-1134 pyspark only uses ipython if starte...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36325879 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1134 pyspark only uses ipython if starte...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36325871 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1134 pyspark only uses ipython if starte...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36325872 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: SPARK-1134 pyspark only uses ipython if starte...

2014-02-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36325759 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36325079 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12927/ --- If your project i

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36325078 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/39 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36324015 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36323971 thanks I've merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36324006 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36324005 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36323841 Thanks @pwendell -- fixed the style! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/39#discussion_r10155351 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala --- @@ -233,7 +238,16 @@ object BlockFetcherIterator { logDeb

How best to ensure partitioners behave?

2014-02-27 Thread Bryn Keller
Hi Folks, I just filed https://spark-project.atlassian.net/browse/SPARK-1149 - I'm happy to fix it, but I'd like input about how best to go about it. The problem is, if a partitioner misbehaves by, say, returning a negative partition number, Spark hangs. This is easier to do than it sounds. I'd l

[GitHub] spark pull request: Removed reference to incubation in Spark user ...

2014-02-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled

[GitHub] spark pull request: SPARK-1134 pyspark only uses ipython if starte...

2014-02-27 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36322600 Jenkins, this is okay to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-1102] Create a saveAsNewAPIHadoopDatase...

2014-02-27 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/12#issuecomment-36321849 this is a re-opened PR, in the old PR, https://github.com/apache/incubator-spark/pull/636, all test cases have passed Can anyone verify that and make further revi

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36315930 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12926/ --- If your pr

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36315929 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/39#discussion_r10152493 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala --- @@ -233,7 +238,16 @@ object BlockFetcherIterator { logDebug("G

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/39#discussion_r10152243 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala --- @@ -44,9 +44,14 @@ import org.apache.spark.util.Utils */

Re: Discussion on SPARK-1139

2014-02-27 Thread ligq
You can make the patch so that everyone review On Wednesday, February 26, 2014 at 8:23 AM, Nan Zhu wrote: > Hi, all > > I just created a JIRA https://spark-project.atlassian.net/browse/SPARK-1139 . > The issue discusses that: > > the new Hadoop API based Spark APIs are actually a mixture

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/33#issuecomment-36315155 Hey Tom, This patch is looking really great. I took a pretty thorough look through this and there were really only two high level things: 1. Configuration

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10152124 --- Diff: docs/security.md --- @@ -0,0 +1,14 @@ +--- +layout: global +title: Spark Security +--- + +Spark currently supports authenticat

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36314544 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36314537 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36314536 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10151846 --- Diff: core/src/main/scala/org/apache/spark/network/SecurityMessage.scala --- @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36313840 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12925/ --- If your pr

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36313839 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10151554 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -41,56 +46,103 @@ private[spark] object JettyUtils extends Logging { type Respo

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36313780 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36313766 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36313763 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-27 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/42 [WIP] [SPARK-1132] Persisting Web UI through refactoring the SparkListener interface The fleeting nature of the Spark Web UI has long been a problem reported by many users: The existing Web UI dis

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36313360 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36313361 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12924/ --- If your project i

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36311620 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36311619 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36311628 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-27 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36311392 Oops, actually pushed it this time. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10150334 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -41,56 +46,103 @@ private[spark] object JettyUtils extends Logging { type Respo

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10150320 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -41,56 +46,103 @@ private[spark] object JettyUtils extends Logging { type Respo

[GitHub] spark pull request: Update io.netty from 4.0.13 Final to 4.0.17.Fi...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/41#issuecomment-36308968 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Update io.netty from 4.0.13 Final to 4.0.17.Fi...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/41#issuecomment-36308969 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12923/ --- If your project i

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10149280 --- Diff: core/src/main/scala/org/apache/spark/network/ConnectionManager.scala --- @@ -557,7 +754,54 @@ private[spark] class ConnectionManager(port: Int, conf

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10149145 --- Diff: core/src/main/scala/org/apache/spark/network/ConnectionId.scala --- @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36307824 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12922/ --- If your project i

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36307822 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Update io.netty from 4.0.13 Final to 4.0.17.Fi...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/41#issuecomment-36306883 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Update io.netty from 4.0.13 Final to 4.0.17.Fi...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/41#issuecomment-36306872 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: Update io.netty from 4.0.13 Final to 4.0.17.Fi...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/41#issuecomment-36306871 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Update io.netty from 4.0.13 Final to 4.0.17.Fi...

2014-02-27 Thread ngbinh
GitHub user ngbinh opened a pull request: https://github.com/apache/spark/pull/41 Update io.netty from 4.0.13 Final to 4.0.17.Final This update contains a lot of bug fixes and some new perf improvements. It is also binary compatible with the current 4.0.13.Final For more

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/40#issuecomment-36306198 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-27 Thread dbtsai
GitHub user dbtsai opened a pull request: https://github.com/apache/spark/pull/40 Initialized the regVal for first iteration in SGD optimizer Ported from https://github.com/apache/incubator-spark/pull/633 In runMiniBatchSGD, the regVal (for 1st iter) should be initialized

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/40#issuecomment-36306177 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36305801 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36305816 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/39#issuecomment-36305800 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Remote BlockFetchTracker trait

2014-02-27 Thread kayousterhout
GitHub user kayousterhout opened a pull request: https://github.com/apache/spark/pull/39 Remote BlockFetchTracker trait This trait seems to have been created a while ago when there were multiple implementations; now that there's just one, I think it makes sense to merge it in

[GitHub] spark pull request: [HOTFIX] Patching maven build after #6 (SPARK-...

2014-02-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/37 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] spark pull request: [HOTFIX] Patching maven build after #6 (SPARK-...

2014-02-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/37#issuecomment-36303359 Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: SPARK-1134

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36301198 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark pull request: SPARK-1134

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36301181 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark pull request: SPARK-1134

2014-02-27 Thread dianacarroll
GitHub user dianacarroll opened a pull request: https://github.com/apache/spark/pull/38 SPARK-1134 bug with ipython: startup scripts are called in both interactive and non-interactive mode. This prevents non-interactive use with spark because the spark startup script and user sc

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10144869 --- Diff: docs/configuration.md --- @@ -477,6 +505,21 @@ Apart from these, the following properties are also available, and may be useful Whet

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10144822 --- Diff: core/src/main/scala/org/apache/spark/network/ConnectionManager.scala --- @@ -53,6 +54,10 @@ private[spark] class ConnectionManager(port: Int, conf:

[GitHub] spark pull request: [HOTFIX] Patching maven build after #6 (SPARK-...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/37#issuecomment-36297711 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12921/ --- If your project i

[GitHub] spark pull request: [HOTFIX] Patching maven build after #6 (SPARK-...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/37#issuecomment-36297710 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10144775 --- Diff: core/src/main/scala/org/apache/spark/network/Connection.scala --- @@ -18,25 +18,27 @@ package org.apache.spark.network import org.apac

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10144726 --- Diff: core/src/main/scala/org/apache/spark/network/Connection.scala --- @@ -18,25 +18,27 @@ package org.apache.spark.network import org.apac

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10144358 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerWebUI.scala --- @@ -198,6 +199,6 @@ class WorkerWebUI(val worker: Worker, val workDir: F

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10144156 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -100,17 +152,12 @@ private[spark] object JettyUtils extends Logging { * If the

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10144073 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10143842 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10143746 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: [HOTFIX] Patching maven build after #6 (SPARK-...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/37#issuecomment-36294696 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [HOTFIX] Patching maven build after #6 (SPARK-...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/37#issuecomment-36294679 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [HOTFIX] Patching maven build after #6 (SPARK-...

2014-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/37#issuecomment-36294680 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [HOTFIX] Patching maven build after #6 (SPARK-...

2014-02-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/37#issuecomment-36294278 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [HOTFIX] Patching maven build after #6 (SPARK-...

2014-02-27 Thread pwendell
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/37 [HOTFIX] Patching maven build after #6 (SPARK-1121). That patch removed the Maven avro declaration but didn't remove the actual dependency in core. /cc @scrapcodes You can merge this pull request

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-27 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36293609 No problem. I attached a new patch that does this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [WIP] [SPARK-1146] Vagrant support for Spark

2014-02-27 Thread ngbinh
Github user ngbinh commented on the pull request: https://github.com/apache/spark/pull/26#issuecomment-36292181 @pwendell Yes! You can point to https://github.com/ngbinh/spark-vagrant --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request: [WIP] [SPARK-1146] Vagrant support for Spark

2014-02-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/26#issuecomment-36291938 @ngbinh when we launch a list of contrib projects, would you be interested in being included in it? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10142402 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -135,6 +135,8 @@ class SparkContext( val isLocal = (master == "local" || m

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10141554 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10141281 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: Add Security to Spark - Akka, Http, Connection...

2014-02-27 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/33#discussion_r10141220 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

  1   2   3   >