+1 (non-binding)
Tested Scala, SparkSQL, and MLLib on OSX against Hadoop 2.6
On Wed, Apr 8, 2015 at 5:35 PM Joseph Bradley wrote:
> +1 tested ML-related items on Mac OS X
>
> On Wed, Apr 8, 2015 at 7:59 PM, Krishna Sankar
> wrote:
>
> > +1 (non-binding, of course)
> >
> > 1. Compiled OSX 10.10
+1 tested ML-related items on Mac OS X
On Wed, Apr 8, 2015 at 7:59 PM, Krishna Sankar wrote:
> +1 (non-binding, of course)
>
> 1. Compiled OSX 10.10 (Yosemite) OK Total time: 14:16 min
> mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4
> -Dhadoop.version=2.6.0 -Phive -DskipTests -
+1 (non-binding, of course)
1. Compiled OSX 10.10 (Yosemite) OK Total time: 14:16 min
mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4
-Dhadoop.version=2.6.0 -Phive -DskipTests -Dscala-2.11
2. Tested pyspark, mlib - running as well as compare results with 1.3.0
pyspark works well
+1 for this feature
In our use case, we probably wouldn’t use this feature in production, but it
can be useful during prototyping and algorithm development to repeatedly
perform the same streaming operation on a fixed, already existing set of files.
-
jeremyfreeman.net
@
I'll add a note that this is just for ML, not other parts of Spark. (We
can discuss more on the JIRA.)
Thanks!
Joseph
On Mon, Apr 6, 2015 at 9:46 PM, Yu Ishikawa
wrote:
> Hi all,
>
> Joseph proposed an idea about using just builder methods, instead of static
> train()
> methods for Scala/Java.
+1
Built against Hadoop 2.6 and ran some jobs against a pseudo-distributed
YARN cluster.
-Sandy
On Wed, Apr 8, 2015 at 12:49 PM, Patrick Wendell wrote:
> Oh I see - ah okay I'm guessing it was a transient build error and
> I'll get it posted ASAP.
>
> On Wed, Apr 8, 2015 at 3:41 PM, Denny Lee
Hey Nathan, thanks for bringing this up I will look at this within the next
day or two.
2015-04-08 8:03 GMT-07:00 Nathan Kronenfeld
:
> Could I get someone to look at PR 5140 please? It's been languishing more
> than two weeks.
>
Oh I see - ah okay I'm guessing it was a transient build error and
I'll get it posted ASAP.
On Wed, Apr 8, 2015 at 3:41 PM, Denny Lee wrote:
> Oh, it appears the 2.4 bits without hive are there but not the 2.4 bits with
> hive. Cool stuff on the 2.6.
> On Wed, Apr 8, 2015 at 12:30 Patrick Wendell
Oh, it appears the 2.4 bits without hive are there but not the 2.4 bits
with hive. Cool stuff on the 2.6.
On Wed, Apr 8, 2015 at 12:30 Patrick Wendell wrote:
> Hey Denny,
>
> I beleive the 2.4 bits are there. The 2.6 bits I had done specially
> (we haven't merge that into our upstream build scrip
Hey Denny,
I beleive the 2.4 bits are there. The 2.6 bits I had done specially
(we haven't merge that into our upstream build script). I'll do it
again now for RC2.
- Patrick
On Wed, Apr 8, 2015 at 1:53 PM, Timothy Chen wrote:
> +1 Tested on 4 nodes Mesos cluster with fine-grain and coarse-grai
One good way to guarantee your tests will work is to have your server bind to
an ephemeral port and then query it to find the port it is running on. This
ensures that race conditions don’t cause test failures.
Thanks, Hari
On Wed, Apr 8, 2015 at 3:24 AM, Sean Owen wrote:
> Utils.startServi
Tathagata,
Thanks for stating your preference for Approach 2.
My use case and motivation are similar to the concerns raised by others in
SPARK-3276. In previous versions of Spark, e.g. 1.1.x we had the ability
for Spark Streaming applications to process the files in an input directory
that existe
Why is this a bug? Each RDD implementation should know whether they have a
parent or not.
For example, if you are a MapPartitionedRDD, there is always a parent since
it is a unary operator.
On Wed, Apr 8, 2015 at 6:19 AM, Zoltán Zvara wrote:
> Is does not seem to be safe to call RDD.firstParen
+1 Tested on 4 nodes Mesos cluster with fine-grain and coarse-grain mode.
Tim
On Wed, Apr 8, 2015 at 9:32 AM, Denny Lee wrote:
> The RC2 bits are lacking Hadoop 2.4 and Hadoop 2.6 - was that intended
> (they were included in RC1)?
>
>
> On Wed, Apr 8, 2015 at 9:01 AM Tom Graves
> wrote:
>
>> +1
Approach 2 is definitely better :)
Can you tell us more about the use case why you want to do this?
TD
On Wed, Apr 8, 2015 at 1:44 AM, Emre Sevinc wrote:
> Hello,
>
> This is about SPARK-3276 and I want to make MIN_REMEMBER_DURATION (that is
> now a constant) a variable (configurable, with a d
The RC2 bits are lacking Hadoop 2.4 and Hadoop 2.6 - was that intended
(they were included in RC1)?
On Wed, Apr 8, 2015 at 9:01 AM Tom Graves
wrote:
> +1. Tested spark on yarn against hadoop 2.6.
> Tom
>
>
> On Wednesday, April 8, 2015 6:15 AM, Sean Owen
> wrote:
>
>
> Still a +1 from me
+1. Tested spark on yarn against hadoop 2.6.
Tom
On Wednesday, April 8, 2015 6:15 AM, Sean Owen wrote:
Still a +1 from me; same result (except that now of course the
UISeleniumSuite test does not fail)
On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell wrote:
> Please vote on releasin
+1. Tested on Mac OS X and verified that some of the bugs were fixed.
Matei
> On Apr 8, 2015, at 7:13 AM, Sean Owen wrote:
>
> Still a +1 from me; same result (except that now of course the
> UISeleniumSuite test does not fail)
>
> On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell wrote:
>> Ple
Could I get someone to look at PR 5140 please? It's been languishing more
than two weeks.
Is does not seem to be safe to call RDD.firstParent from anywhere, as it
might throw a java.util.NoSuchElementException: "head of empty list". This
seems to be a bug for a consumer of the RDD API.
Zvara Zoltán
mail, hangout, skype: zoltan.zv...@gmail.com
mobile, viber: +36203129543
bank: 1091
Still a +1 from me; same result (except that now of course the
UISeleniumSuite test does not fail)
On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.3.1!
>
> The tag to be voted on is v1.3.1-rc2 (commit 7c4473a):
Utils.startServiceOnPort?
On Wed, Apr 8, 2015 at 6:16 AM, Steve Loughran wrote:
>
> I'm writing some functional tests for the SPARK-1537 JIRA, Yarn timeline
> service integration, for which I need to allocate some free ports.
>
> I don't want to hard code them in as that can lead to unreliable t
I'm writing some functional tests for the SPARK-1537 JIRA, Yarn timeline
service integration, for which I need to allocate some free ports.
I don't want to hard code them in as that can lead to unreliable tests,
especially on Jenkins.
Before I implement the logic myself -Is there a utility cl
Hello,
This is about SPARK-3276 and I want to make MIN_REMEMBER_DURATION (that is
now a constant) a variable (configurable, with a default value). Before
spending effort on developing something and creating a pull request, I
wanted to consult with the core developers to see which approach makes mo
24 matches
Mail list logo
