Re: spark task scheduling delay

Which Resource Manager are you using? 2016-01-20 21:38 GMT-08:00 Renu Yadav : > Any suggestions? > > On Wed, Jan 20, 2016 at 6:50 PM, Renu Yadav wrote: > >> Hi , >> >> I am facing spark task scheduling delay issue in spark 1.4. >> >> suppose I have 1600 tasks running then 1550 tasks runs fine

Re: How Spark utilize low-level architecture features?

Anyone could shed some light on this? Thanks, Boric On Tue, Jan 19, 2016 at 4:12 PM, Boric Tan wrote: > Hi there, > > I am new to Spark, and would like to get some help to understand if Spark > can utilize the underlying architectures for better performance. If so, how > does it do it? > > For

RE: Using CUDA within Spark / boosting linear algebra

Hi Everyone, I’ve updated the benchmark and done experiments with new hardware with 2x Nvidia Tesla K80 (physically 4x Tesla K40) and 2x modern Haswell CPU Intel E5-2650 v3 @ 2.30GHz. This time I computed average and median of 10 runs for each of experiment and approximated FLOPS. Results are

Re: BUILD FAILURE at Spark Project Test Tags for 2.11.7?

On Wed, Jan 20, 2016 at 10:47 PM, Sean Owen wrote: > That's not a Spark problem. Your compiler was not available. It's Spark to use zinc. It starts it for a built, but alas doesn't stop it afterwards. If you happen to upgrade your JDK without restarting zinc, you *will* run into the issue. If you

Re: BUILD FAILURE at Spark Project Test Tags for 2.11.7?

That's not a Spark problem. Your compiler was not available. On Wed, Jan 20, 2016 at 10:44 PM, Jacek Laskowski wrote: > On Wed, Jan 20, 2016 at 8:48 PM, Marcelo Vanzin wrote: >> On Wed, Jan 20, 2016 at 11:46 AM, Jacek Laskowski wrote: >>> /Users/jacek/dev/oss/spark/tags/target/scala-2.11/classe

Re: BUILD FAILURE at Spark Project Test Tags for 2.11.7?

On Wed, Jan 20, 2016 at 8:48 PM, Marcelo Vanzin wrote: > On Wed, Jan 20, 2016 at 11:46 AM, Jacek Laskowski wrote: >> /Users/jacek/dev/oss/spark/tags/target/scala-2.11/classes... >> [error] Cannot run program "javac": error=2, No such file or directory > > That doesn't exactly look like a Spark pr

Re: BUILD FAILURE at Spark Project Test Tags for 2.11.7?

On Wed, Jan 20, 2016 at 8:48 PM, Marcelo Vanzin wrote: > On Wed, Jan 20, 2016 at 11:46 AM, Jacek Laskowski wrote: >> /Users/jacek/dev/oss/spark/tags/target/scala-2.11/classes... >> [error] Cannot run program "javac": error=2, No such file or directory > > That doesn't exactly look like a Spark pr

Re: BUILD FAILURE at Spark Project Test Tags for 2.11.7?

On Wed, Jan 20, 2016 at 11:46 AM, Jacek Laskowski wrote: > /Users/jacek/dev/oss/spark/tags/target/scala-2.11/classes... > [error] Cannot run program "javac": error=2, No such file or directory That doesn't exactly look like a Spark problem. -- Marcelo --

BUILD FAILURE at Spark Project Test Tags for 2.11.7?

Hi, Build is broken again for me :( I build for Scala 2.11.7 and use maven. Is this a known issue? Anyone looking into it? ➜ spark git:(master) ✗ ./build/mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.7.1 -Dscala-2.11 -Phive -Phive-thriftserver -DskipTests clean install ... [INFO] --- scala-maven-pl

Re: Removing the Mesos fine-grained mode

That'd be great, thanks Adam! On Tue, Jan 19, 2016 at 5:41 PM, Adam McElwee wrote: > Sorry, I never got a chance to circle back with the master logs for this. > I definitely can't share the job code, since it's used to build a pretty > core dataset for my company, but let me see if I can pull so

Optimized toIndexedRowMatrix

Hi guys, I've been working on an optimized implementation of the toIndexedRowMatrix of the BlockMatrix. I already created a ticket