Re: [Tungsten] NPE in UnsafeShuffleWriter.java

2015-06-19 Thread Josh Rosen
I've filed https://issues.apache.org/jira/browse/SPARK-8498 to fix this error-handling code. On Fri, Jun 19, 2015 at 11:51 AM, Josh Rosen wrote: > Hey Peter, > > I think that this is actually due to an error-handling issue: if you look > at the stack trace that you posted, the NPE is being throw

Re: Workaround for problems with OS X + JIRA Client

2015-06-19 Thread Yu Ishikawa
Hi Sean, That sounds interesting. I didn't know the client. I will try it later. Thank you for sharing the information. Yu - -- Yu Ishikawa -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Workaround-for-problems-with-OS-X-JIRA-Client-tp12799p1280

Re: [mllib] Refactoring some spark.mllib model classes in Python not inheriting JavaModelWrapper

2015-06-19 Thread Yu Ishikawa
Hi Xiangrui I got it. I will try to refactor any model class not inheriting JavaModelWrapper and show you it. Thanks, Yu - -- Yu Ishikawa -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Refactoring-some-spark-mllib-model-classes-in-Python-n

Re: Stats on targets for 1.5.0

2015-06-19 Thread Nicholas Chammas
> I think it would be fantastic if this work was burned down before adding big new chunks of work. The stat is worth keeping an eye on. +1, keeping in mind that burning down work also means just targeting it for a different release or closing it. :) Nick On Fri, Jun 19, 2015 at 3:18 PM Sean Owe

a

2015-06-19 Thread Cubenomy Subscriber

Stats on targets for 1.5.0

2015-06-19 Thread Sean Owen
Quick point of reference for 1.5.0: 226 issues are Fixed for 1.5.0, and 388 are Targeted for 1.5.0. So maybe 36% of things to be done for 1.5.0 are complete, and we're in theory 3 of 8 weeks into the merge window, or 37.5%. That's nicely on track! assuming, of course, that nothing else is targeted

Workaround for problems with OS X + JIRA Client

2015-06-19 Thread Sean Owen
Not sure if many of you use JIRA Client (http://almworks.com/jiraclient/overview.html) to keep tabs on JIRA -- definitely worth it -- but if you're on OS X, I wonder if you too have suddenly been experiencing some type of SSL / keypair error on syncing? It's something to do with a JIRA server updat

Re: [Tungsten] NPE in UnsafeShuffleWriter.java

2015-06-19 Thread Josh Rosen
Hey Peter, I think that this is actually due to an error-handling issue: if you look at the stack trace that you posted, the NPE is being thrown from an error-handling branch of a `finally` block: @Override public void write(scala.collection.Iterator> records) throws IOException { boolean success

[Tungsten] NPE in UnsafeShuffleWriter.java

2015-06-19 Thread Peter Rudenko
Hi want to try new tungsten-sort shuffle manager, but on 1 stage executors start to die with NPE: 15/06/19 17:53:35 WARN TaskSetManager: Lost task 38.0 in stage 41.0 (TID 3176, ip-10-50-225-214.ec2.internal): java.lang.NullPointerException at org.apache.spark.shuffle.unsafe.UnsafeShuff