Re: short jenkins downtime -- trying to get to the bottom of the git fetch timeouts

2014-10-18 Thread Davies Liu
Cool, the recent 4 build had used the new configs, thanks! Let's run more builds. Davies On Fri, Oct 17, 2014 at 11:06 PM, Josh Rosen wrote: > I think that the fix was applied. Take a look at > https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21874/consoleFull > > Here, I see

Oryx + Spark mllib

2014-10-18 Thread Debasish Das
Hi, Is someone working on a project on integrating Oryx model serving layer with Spark ? Models will be built using either Streaming data / Batch data in HDFS and cross validated with mllib APIs but the model serving layer will give API endpoints like Oryx and read the models may be from hdfs/impa

Re: Oryx + Spark mllib

2014-10-18 Thread Rajiv Abraham
Oryx 2 seems to be geared for Spark https://github.com/OryxProject/oryx 2014-10-18 11:46 GMT-04:00 Debasish Das : > Hi, > > Is someone working on a project on integrating Oryx model serving layer > with Spark ? Models will be built using either Streaming data / Batch data > in HDFS and cross val

Joining the spark dev community

2014-10-18 Thread Saurabh Wadhawan
How can I become a spark contributor. What's the good path that I can follow to become an active code submitter for spark from a newbie. Regards - Saurabh

Re: Raise Java dependency from 6 to 7

2014-10-18 Thread Koert Kuipers
my experience is that there are still a lot of java 6 clusters out there. also distros that bundle spark still support java 6 On Oct 17, 2014 8:01 PM, "Andrew Ash" wrote: > Hi Spark devs, > > I've heard a few times that keeping support for Java 6 is a priority for > Apache Spark. Given that Java

Re: Raise Java dependency from 6 to 7

2014-10-18 Thread Matei Zaharia
I'd also wait a bit until these are gone. Jetty is unfortunately a much hairier topic by the way, because the Hadoop libraries also depend on Jetty. I think it will be hard to update. However, a patch that shades Jetty might be nice to have, if that doesn't require shading a lot of other stuff.

Re: Raise Java dependency from 6 to 7

2014-10-18 Thread Marcelo Vanzin
Hadoop, for better or worse, depends on an ancient version of Jetty (6), that is even on a different package. So Spark (or anyone trying to use a newer Jetty) is lucky on that front... IIRC Hadoop is planning to move to Java 7-only starting with 2.7. Java 7 is also supposed to be EOL some time nex

Submissions open for Spark Summit East 2015

2014-10-18 Thread Matei Zaharia
After successful events in the past two years, the Spark Summit conference has expanded for 2015, offering both an event in New York on March 18-19 and one in San Francisco on June 15-17. The conference is a great chance to meet people from throughout the Spark community and see the latest news,

Re: Oryx + Spark mllib

2014-10-18 Thread Sean Owen
Yes, that is exactly what the next 2.x version does. Still in progress but the recommender app and framework are code - complete. It is not even specific to MLlib and could plug in other model build functions. The current 1.x version will not use MLlib. Neither uses Play but is intended to scale j

Re: Oryx + Spark mllib

2014-10-18 Thread Nick Pentreath
We've built a model server internally, based on Scalatra and Akka Clustering. Our use case is more geared towards serving possibly thousands of smaller models. It's actually very basic, just reads models from S3 as strings (!!) (uses HDFS FileSystem so can read from local, HDFS, S3) and uses Breez