Re: sbt-package-bin

> What do folks think of adding the sbt-package-bin plugin to allow easy > creation of a Spark .deb file? I believe it adds all dependency jars into > a single lib/ folder, so in some ways it's even easier to manage than the > assembly. > You might also check out the sbt-native-packager

Re: sbt-package-bin

Whoops! Looks like it was just my brain that was broken. On Tue, Apr 1, 2014 at 6:03 PM, Mark Hamstra wrote: > ...or at least you could do that if the Maven build wasn't broken right > now. > > > On Tue, Apr 1, 2014 at 6:01 PM, Mark Hamstra wrote: > >> What the "..." is kind of depends on what

Re: sbt-package-bin

...or at least you could do that if the Maven build wasn't broken right now. On Tue, Apr 1, 2014 at 6:01 PM, Mark Hamstra wrote: > What the "..." is kind of depends on what you're trying to accomplish. > You could be setting Hadoop version and other stuff in there, but if you > go too much beyo

Re: sbt-package-bin

What the "..." is kind of depends on what you're trying to accomplish. You could be setting Hadoop version and other stuff in there, but if you go too much beyond a pretty basic build, you're probably also going to have to modify the of the jdeb plugin in assembly/pom.xml to include/exclude just

Re: sbt-package-bin

Mark - sorry, would you mind expanding what the "" is? Something like mvn -Pdeb package ? I get: [ERROR] Plugin org.apache.maven.plugins:maven-compiler-plugin:3.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-compile

Re: sbt-package-bin

Ya there is already some fragmentation here. Maven has some "dist" targets and there is also ./make-distribution.sh. On Tue, Apr 1, 2014 at 11:31 AM, Mark Hamstra wrote: > A basic Debian package can already be created from the Maven build: mvn > -Pdeb ... > > > On Tue, Apr 1, 2014 at 11:24 AM, E

Re: sbt-package-bin

And there is a deb target as well - ah didn't see Mark's email. On Tue, Apr 1, 2014 at 11:36 AM, Patrick Wendell wrote: > Ya there is already some fragmentation here. Maven has some "dist" targets > and there is also ./make-distribution.sh. > > > On Tue, Apr 1, 2014 at 11:31 AM, Mark Hamstra wr

Re: [VOTE] Release Apache Spark 0.9.1 (RC3)

No one else has reported seeing the issue so I think documenting it is fine.   Tom On Tuesday, April 1, 2014 1:40 PM, Patrick Wendell wrote: Tom, Given this is a pretty straightforward workaround, what do yo think about the following course of action: (a) We can put the workaround in the do

Re: [VOTE] Release Apache Spark 0.9.1 (RC3)

Tom, Given this is a pretty straightforward workaround, what do yo think about the following course of action: (a) We can put the workaround in the docs for 0.9.1. We don't need to do a new RC/vote for this since we can update the published docs independently. (b) We try to get a fix in for this

Re: sbt-package-bin

A basic Debian package can already be created from the Maven build: mvn -Pdeb ... On Tue, Apr 1, 2014 at 11:24 AM, Evan Chan wrote: > Also, I understand this is the last week / merge window for 1.0, so if > folks are interested I'd like to get in a PR quickly. > > thanks, > Evan > > > > On Tue,

Re: sbt-package-bin

Also, I understand this is the last week / merge window for 1.0, so if folks are interested I'd like to get in a PR quickly. thanks, Evan On Tue, Apr 1, 2014 at 11:24 AM, Evan Chan wrote: > Hey folks, > > We are in the middle of creating a Chef recipe for Spark. As part of > that we want to

sbt-package-bin

Hey folks, We are in the middle of creating a Chef recipe for Spark. As part of that we want to create a Debian package for Spark. What do folks think of adding the sbt-package-bin plugin to allow easy creation of a Spark .deb file? I believe it adds all dependency jars into a single lib/ fold

Re: [VOTE] Release Apache Spark 0.9.1 (RC3)

Thanks for extending the voting. Unfortunately I've found an issue with the spark-shell in yarn-client mode.  It doesn't work with secure HDFS unless you  export SPARK_YARN_MODE=true before starting the shell, or if you happen to do something immediately with HDFS.  If you wait for the connectio