Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-26 Thread Sean Owen
I still don't see any artifacts in maven -- did it publish? http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.spark%22 On Wed, May 25, 2016 at 10:30 AM, Reynold Xin wrote: > Yup I have published it to maven. Will post the link in a bit. > > One thing is that for developers, it might b

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-26 Thread Gurvinder Singh
On 05/26/2016 02:38 AM, Matei Zaharia wrote: > Just wondering, what is the main use case for the Docker images -- to > develop apps locally or to deploy a cluster? I use docker images for both development and deploying on production cluster. As it makes sure I have the correct version of Java and

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Marcin Tustin
The use case of docker images in general is that you can deploy and develop with exactly the same binary environment - same java 8, same scala, same spark. This makes things repeatable. On Wed, May 25, 2016 at 8:38 PM, Matei Zaharia wrote: > Just wondering, what is the main use case for the Dock

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Matei Zaharia
Just wondering, what is the main use case for the Docker images -- to develop apps locally or to deploy a cluster? If the image is really just a script to download a certain package name from a mirror, it may be okay to create an official one, though it does seem tricky to make it properly use t

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Luciano Resende
On Wed, May 25, 2016 at 2:34 PM, Sean Owen wrote: > I don't think the project would bless anything but the standard > release artifacts since only those are voted on. People are free to > maintain whatever they like and even share it, as long as it's clear > it's not from the Apache project. > >

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Sean Owen
I don't think the project would bless anything but the standard release artifacts since only those are voted on. People are free to maintain whatever they like and even share it, as long as it's clear it's not from the Apache project. On Wed, May 25, 2016 at 3:41 PM, Marcin Tustin wrote: > Ah ver

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Marcin Tustin
Ah very nice. Would it be possible to have this blessed into an official image? On Wed, May 25, 2016 at 4:12 PM, Luciano Resende wrote: > > > On Wed, May 25, 2016 at 6:53 AM, Marcin Tustin > wrote: > >> Would it be useful to start baking docker images? Would anyone find that >> a boon to their

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Luciano Resende
On Wed, May 25, 2016 at 6:53 AM, Marcin Tustin wrote: > Would it be useful to start baking docker images? Would anyone find that a > boon to their testing? > > +1, I had done one (still based on 1.6) for some SystemML experiments, I could easily get it based on a nightly build. https://github.co

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Reynold Xin
The maven artifacts can be found at https://repository.apache.org/content/repositories/orgapachespark-1182/ But really for people on this list, it might be better to go straight to the nightly snapshots. https://repository.apache.org/content/groups/snapshots/org/apache/spark/ On Wed, May 25, 201

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Reynold Xin
Yup I have published it to maven. Will post the link in a bit. One thing is that for developers, it might be better to use the nightly snapshot because that one probably has fewer bugs than the preview one. On Wednesday, May 25, 2016, Daniel Darabos wrote: > Awesome, thanks! It's very helpful f

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Marcin Tustin
Would it be useful to start baking docker images? Would anyone find that a boon to their testing? On Wed, May 25, 2016 at 2:44 AM, Reynold Xin wrote: > In the past the Spark community have created preview packages (not > official releases) and used those as opportunities to ask community members

Re: [ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-25 Thread Daniel Darabos
Awesome, thanks! It's very helpful for preparing for the migration. Do you plan to push 2.0.0-preview to Maven too? (I for one would appreciate the convenience.) On Wed, May 25, 2016 at 8:44 AM, Reynold Xin wrote: > In the past the Spark community have created preview packages (not > official re

[ANNOUNCE] Apache Spark 2.0.0-preview release

2016-05-24 Thread Reynold Xin
In the past the Spark community have created preview packages (not official releases) and used those as opportunities to ask community members to test the upcoming versions of Apache Spark. Several people in the Apache community have suggested we conduct votes for these preview packages and turn th