Re: Building Spark AMI

2014-04-11 Thread Jim Ancona
Hi, Right now my use case is setting up a small cluster for prototyping/evaluation. My hope was that I could use the scripts that come with Spark to get things up and running quickly. For a production deploy we would probably roll our own using Puppet. Jim On Fri, Apr 11, 2014 at 7:58 PM, Mayur

Re: Building Spark AMI

2014-04-11 Thread Mayur Rustagi
I am creating one fully configured & synced one. But you still need to send over configuration. Do you plan to use chef for that ? On Apr 10, 2014 6:58 PM, "Jim Ancona" wrote: > Are there scripts to build the AMI used by the spark-ec2 script? > > Alternatively, is there a place to download the A

It seems that jenkins for PR is not working

2014-04-11 Thread DB Tsai
I always got = Could not find Apache license headers in the following files: !? /root/workspace/SparkPullRequestBuilder/python/metastore/db.lck !? /root/workspace/SparkPullRequestBuilder/python/metastore/service.pro

Re: feature selection and sparse vector support

2014-04-11 Thread Ignacio Zendejas
Here's the JIRA: https://issues.apache.org/jira/browse/SPARK-1473 Future discussions should take place in its comments section. Thanks. On Fri, Apr 11, 2014 at 11:26 AM, Ignacio Zendejas < ignacio.zendejas...@gmail.com> wrote: > Thanks for the response, Xiangrui. > > And sounds good, Héctor.

Re: Suggestion

2014-04-11 Thread Sandy Ryza
Hi Priya, Here's a good place to start: https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark -Sandy On Fri, Apr 11, 2014 at 12:05 PM, priya arora wrote: > Hi, > > May I know how one can contribute in this project > http://spark.apache.org/mllib/ or in any other project. I am

Suggestion

2014-04-11 Thread priya arora
Hi, May I know how one can contribute in this project http://spark.apache.org/mllib/ or in any other project. I am very eager to contribute. Do let me know. Thanks & Regards, Priya Arora

Re: feature selection and sparse vector support

2014-04-11 Thread Ignacio Zendejas
Thanks for the response, Xiangrui. And sounds good, Héctor. Look forward to working on this together. A common interface is definitely required. I'll create a JIRA shortly and will explore design options myself to bring ideas to the table. cheers. On Fri, Apr 11, 2014 at 5:44 AM, Héctor Mour

Re: RFC: varargs in Logging.scala?

2014-04-11 Thread David Hall
Another usage that's nice is: logDebug { val timeS = timeMillis/1000.0 s"Time: $timeS" } which can be useful for more complicated expressions. On Thu, Apr 10, 2014 at 5:55 PM, Michael Armbrust wrote: > BTW... > > You can do calculations in string interpolation: > s"Time: ${timeMillis / 1

Re: RFC: varargs in Logging.scala?

2014-04-11 Thread Marcelo Vanzin
On Thu, Apr 10, 2014 at 5:46 PM, Michael Armbrust wrote: > ... all of the suffer from the fact that the log message needs to be built >> even >> though it might not be used. > > This is not true of the current implementation (and this is actually why > Spark has a logging trait instead of just usi

Re: feature selection and sparse vector support

2014-04-11 Thread Héctor Mouriño-Talín
Hi, Regarding the implementation of feature selection techniques, I'm implementing some iterative algorithms based on a paper by Gavin Brown et al. [1]. In this paper, he proposes a common framework for many Information Theory-based criteria, namely those that use relevancy (mutual information bet