Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/149#discussion_r10637576
--- Diff: core/src/main/scala/org/apache/spark/Dependency.scala ---
@@ -43,12 +44,13 @@ abstract class NarrowDependency[T](rdd: RDD[T]) extends
Dependency(rdd
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/149#discussion_r10636602
--- Diff: core/src/main/scala/org/apache/spark/Dependency.scala ---
@@ -43,12 +44,13 @@ abstract class NarrowDependency[T](rdd: RDD[T]) extends
Dependency(rdd
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/149#discussion_r10636356
--- Diff: core/src/main/scala/org/apache/spark/Dependency.scala ---
@@ -43,12 +44,13 @@ abstract class NarrowDependency[T](rdd: RDD[T]) extends
Dependency(rdd
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/149#issuecomment-37720234
@marmbrus this is for you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/149
SPARK-1255: Allow user to pass Serializer object instead of class name for
shuffle.
This is more general than simply passing a string name and leaves more room
for performance optimizations
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/141#issuecomment-37717648
I did :
https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=e19044cb1048c3755d1ea2cb43879d2225d49b54
---
If your project is set up for it, you can reply to this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/143#issuecomment-37691023
I was thinking maybe we want a config option for this - which is on by
default, but can be turned off. What do you guys think?
---
If your project is set up for it, you can
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/141#issuecomment-37681568
Merge.d
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/79#issuecomment-37607046
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/135#discussion_r10581086
--- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala ---
@@ -457,6 +457,10 @@ class RDDSuite extends FunSuite with
SharedSparkContext
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/135#discussion_r10579726
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -310,6 +310,9 @@ abstract class RDD[T: ClassTag](
* Return a sampled subset of this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/131#issuecomment-37507055
Thanks. I've merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/126#issuecomment-37501863
If you don't need high performance, why not just put a normal immutable
hashmap so you don't have to worry about concurrency?
---
If your project is set up for i
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/126#issuecomment-37501259
@tdas I haven't finished looking at this (will probably spend more time
after Fri) - but WrappedJavaHashMap is fairly complicated, and it seems like a
recipe for compl
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10553024
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockManagerMessages.scala ---
@@ -35,9 +35,9 @@ private[storage] object BlockManagerMessages
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552981
--- Diff: core/src/main/scala/org/apache/spark/util/BoundedHashMap.scala ---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552941
--- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552859
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockManagerMessages.scala ---
@@ -35,9 +35,9 @@ private[storage] object BlockManagerMessages
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552846
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -20,15 +20,15 @@ package org.apache.spark
import java.io._
import
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552839
--- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552834
--- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552716
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -278,7 +278,7 @@ private[spark] class Executor(
// have left some
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552723
--- Diff: core/src/main/scala/org/apache/spark/network/Connection.scala ---
@@ -206,12 +206,12 @@ class SendingConnection(val address:
InetSocketAddress
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552696
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/StateDStream.scala
---
@@ -64,7 +64,7 @@ class StateDStream[K: ClassTag, V: ClassTag, S
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552705
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala ---
@@ -152,7 +152,7 @@ class InputStreamsSuite extends TestSuiteBase with
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552712
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala ---
@@ -137,7 +137,7 @@ trait ClientBase extends Logging {
} else if
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552681
--- Diff:
graphx/src/main/scala/org/apache/spark/graphx/impl/Serializers.scala ---
@@ -298,7 +298,7 @@ abstract class ShuffleSerializationStream(s:
OutputStream
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552685
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/NetworkInputDStream.scala
---
@@ -128,7 +128,7 @@ abstract class NetworkReceiver[T
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552680
--- Diff:
graphx/src/main/scala/org/apache/spark/graphx/impl/Serializers.scala ---
@@ -391,7 +391,7 @@ abstract class ShuffleDeserializationStream(s
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552671
--- Diff:
examples/src/main/scala/org/apache/spark/examples/SparkHdfsLR.scala ---
@@ -34,8 +34,8 @@ object SparkHdfsLR {
case class DataPoint(x: Vector, y
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552662
--- Diff: examples/src/main/scala/org/apache/spark/examples/LocalALS.scala
---
@@ -53,7 +53,7 @@ object LocalALS {
for (i <- 0 until M; j <- 0 u
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552655
--- Diff: core/src/test/scala/org/apache/spark/CheckpointSuite.scala ---
@@ -432,7 +432,7 @@ object CheckpointSuite {
// This is a custom cogroup function
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552666
--- Diff: examples/src/main/scala/org/apache/spark/examples/SparkALS.scala
---
@@ -54,7 +54,7 @@ object SparkALS {
for (i <- 0 until M; j <- 0 u
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/124#discussion_r10552654
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockManagerMessages.scala ---
@@ -35,9 +35,9 @@ private[storage] object BlockManagerMessages
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/126#discussion_r10552638
--- Diff:
core/src/main/scala/org/apache/spark/util/TimeStampedWeakValueHashMap.scala ---
@@ -0,0 +1,112 @@
+/*
+ * Licensed to the Apache Software
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37483693
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37482277
Ok I pushed a new version with Maven build changes as well. This is ready
to be merged from my perspective.
---
If your project is set up for it, you can reply to this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37377281
That sounds good. @pwendell should make the call here ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37249125
Ok I think the cloduera repo is up. @tgravescs it would be great if you can
try this.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37230554
Yea unfortunately that's been the case for the past few days We should
probably have another external repo to host artifacts only on cloudera repo to
have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37213133
The main reason is that we are investigating upgrading some of the major
dependencies prior to 1.0, after which we won't be able to upgrade for a while.
Some users
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37212561
Actually I was waiting for the pull request builder to come back before
asking you to verify it. @tgravescs do you mind verifying this works for you? I
haven't update
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/113#issuecomment-37158460
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/113#discussion_r10420573
--- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala ---
@@ -120,26 +120,25 @@ private[spark] object JettyUtils extends Logging
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/114#issuecomment-37154805
Thanks. Merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/113
Upgrade Jetty to 9.1.3.v20140225.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rxin/spark jetty9
Alternatively you can review and apply these
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/104#issuecomment-37109170
Ok I merged this.
Not sure about Maven off the top of my head. All these build plugins are
pretty arcane to me.
---
If your project is set up for it, you can
GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/104
Update junitxml plugin to the latest version to avoid recompilation in
every SBT command.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rxin
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/103#issuecomment-37091459
Ok I merged this. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/103
Allow sbt to use more than 1G of heap.
There was a mistake in sbt build file ( introduced by
012bd5fbc97dc40bb61e0e2b9cc97ed0083f37f6 ) in which we set the default to 2048
and the immediately reset it
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/21#issuecomment-36949585
We should use the primitive hashmap - otherwise it is pretty slow
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/21#issuecomment-36949572
Actually - no ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/90#discussion_r10332129
--- Diff: graphx/pom.xml ---
@@ -70,6 +70,10 @@
scalacheck_${scala.binary.version}
test
+
--- End diff --
Yes
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/74#issuecomment-36656962
Thanks. Merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/72#issuecomment-36656843
Thanks. Merged.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/66#issuecomment-36570035
Thanks. I've merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/61#issuecomment-36569896
I merged this. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/67#issuecomment-36562029
Hi @jyotiska
These docs are not meant to be consumed directly as markdown files. They
are meant to be generated using jekyll (run jekyll build in docs folder), and
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/65#issuecomment-36487798
I merged this one too.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/65#issuecomment-36487753
Thanks for doing this.
BTW one thing I noticed is that your git commit's email is different from
the ones you registered on github, so your commits don'
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/64#issuecomment-36487507
Actually you will need to submit another PR. I've already merged this one
(but github is laggy because it is waiting for the asf git bot to synchronize).
Sorry abou
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/55#issuecomment-36487061
Also @ryanlecompte since you changed the implementation to tail recursion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/64#issuecomment-36486817
There's also a typo in the Java version of the doc. If you don't mind
fixing that as well ... :)
---
If your project is set up for it, you can reply to this emai
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/63#issuecomment-36486838
Jenkins, add to whitelist.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/64#issuecomment-36486791
Thanks Aaron. I've merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/55#issuecomment-36473499
Also I think we should reduce the time to run this 4 sec test, but that's
for another PR ...
---
If your project is set up for it, you can reply to this email and have
GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/55
SPARK-1158: Fix flaky RateLimitedOutputStreamSuite.
There was actually a problem with the RateLimitedOutputStream
implementation where the first second doesn't write anything because of in
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/55#issuecomment-36473419
@tdas @pwendell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user rxin opened a pull request:
https://github.com/apache/spark/pull/54
Ignore RateLimitedOutputStreamSuite for now.
This test has been flaky. We can re-enable it after @tdas has a chance to
look at it.
You can merge this pull request into a Git repository by running
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/4#issuecomment-36449554
Thanks. I've merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/51#issuecomment-36449552
Thanks @pwendell I merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/41#issuecomment-36449397
Thanks. I've merged this!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/40#issuecomment-36449243
I think the asf git bot will close this once the change is sync-ed on
github. If it doesn't get closed tomorrow morning, please close this manually.
Thanks!
---
If
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/41#issuecomment-36449234
Sorry @ngbinh you misunderstood me. I think the problem is the git commit
metadata doesn't actually contain the author information. It could be that the
email or the a
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/14#issuecomment-36449214
I've merged this. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/40#issuecomment-36449165
Thanks. I've merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/10#issuecomment-36449099
@ankurdave
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/4#issuecomment-36449083
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/23#issuecomment-36449072
#2 was actually merged. @jyotiska do you mind closing this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/41#issuecomment-36449064
Actually I'm having trouble merging this. I think it's because your git
commit doesn't actually have any author information. Do you mind fixing that?
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/41#issuecomment-36449019
Thanks. I've merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/44#issuecomment-36449005
Hi guys,
I think it is better to make sure Spark doesn't hang when an incorrect
partition index is given, because there will be other code paths to run a job.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/51#issuecomment-36448947
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/40#issuecomment-36413838
It is running now. Let's wait for Jenkins to come back.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as wel
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/40#issuecomment-36413420
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/40#issuecomment-36413377
It's not you. There was somehow two Jenkins pull request builder setup ...
I Just removed one of them.
---
If your project is set up for it, you can reply to this emai
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/40#issuecomment-36413254
Jenkins, add to whitelist.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/40#discussion_r10188555
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala
---
@@ -149,7 +149,14 @@ object GradientDescent extends Logging
88 matches
Mail list logo
