Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-07 Thread Shivaram Venkataraman
As far as I know the process is just to copy docs/_site from the build to the appropriate location in the SVN repo (i.e. site/docs/2.0.0-preview). Thanks Shivaram On Tue, Jun 7, 2016 at 8:14 AM, Sean Owen wrote: > As a stop-gap, I can edit that page to have a small section about > preview releas

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-07 Thread Tom Graves
Thanks Sean, you were right, hard refresh made it show up. Seems like we should at least link to the preview docs from  http://spark.apache.org/documentation.html. Tom On Tuesday, June 7, 2016 10:04 AM, Sean Owen wrote: It's there (refresh maybe?). See the end of the downloads dropdown.

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-07 Thread Sean Owen
As a stop-gap, I can edit that page to have a small section about preview releases and point to the nightly docs. Not sure who has the power to push 2.0.0-preview to site/docs, but, if that's done then we can symlink "preview" in that dir to it and be done, and update this section about preview do

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-07 Thread Sean Owen
It's there (refresh maybe?). See the end of the downloads dropdown. For the moment you can see the docs in the nightly docs build: https://home.apache.org/~pwendell/spark-nightly/spark-branch-2.0-docs/latest/ I don't know, what's the best way to put this into the main site? under a /preview root?

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-07 Thread Tom Graves
I just checked and I don't see the 2.0 preview release at all anymore on .http://spark.apache.org/downloads.html, is it in transition?    The only place I can see it is at http://spark.apache.org/news/spark-2.0.0-preview.html I would like to see docs there too.  My opinion is it should be as eas

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Imran Rashid
I've been a bit on the fence on this, but I agree that Luciano makes a compelling reason for why we really should publish things to maven central. Sure we slightly increase the risk somebody refers to the preview release too late, but really that is their own fault. And I also I agree with commen

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Luciano Resende
On Mon, Jun 6, 2016 at 12:05 PM, Reynold Xin wrote: > The bahir one was a good argument actually. I just clicked the button to > push it into Maven central. > > Thank You !!!

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Reynold Xin
The bahir one was a good argument actually. I just clicked the button to push it into Maven central. On Mon, Jun 6, 2016 at 12:00 PM, Mark Hamstra wrote: > Fine. I don't feel strongly enough about it to continue to argue against > putting the artifacts on Maven Central. > > On Mon, Jun 6, 2016

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Mark Hamstra
Fine. I don't feel strongly enough about it to continue to argue against putting the artifacts on Maven Central. On Mon, Jun 6, 2016 at 11:48 AM, Sean Owen wrote: > Artifacts can't be removed from Maven in any normal circumstance, but, > it's no problem. > > The argument that people might keep

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Sean Owen
Artifacts can't be removed from Maven in any normal circumstance, but, it's no problem. The argument that people might keep using it goes for any older release. Why would anyone use 1.6.0 when 1.6.1 exists? yet we keep 1.6.0 just for the record and to not break builds. It may be that Foobar 3.0-be

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Ovidiu-Cristian MARCU
+1 for moving this discussion to a proactive new (alpha/beta) release of Apache Spark 2.0! > On 06 Jun 2016, at 20:25, Ovidiu Cristian Marcu wrote: > > Any chance to start preparing a new alpha/beta release for 2.0 this month or > the preview will be pushed to maven and considered an alpha? >

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Mark Hamstra
Precisely because the naming of the preview artifacts has to fall outside of the normal versioning, I can easily see incautious Maven users a few months from now mistaking the preview artifacts as spark-2.0-something-special instead of spark-2.0-something-stale. On Mon, Jun 6, 2016 at 10:49 AM, Lu

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Luciano Resende
On Mon, Jun 6, 2016 at 11:12 AM, Matei Zaharia wrote: > Is there any way to remove artifacts from Maven Central? Maybe that would > help clean these things up long-term, though it would create problems for > users who for some reason decide to rely on these previews. > > In any case, if people ar

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Matei Zaharia
BTW, same goes with docs -- Sean, if you want to add a /docs/2.0-preview on the website and link to it, go for it! Matei On Mon, Jun 6, 2016 at 11:12 AM, Matei Zaharia wrote: > Is there any way to remove artifacts from Maven Central? Maybe that would > help clean these things up long-term, thou

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Matei Zaharia
Is there any way to remove artifacts from Maven Central? Maybe that would help clean these things up long-term, though it would create problems for users who for some reason decide to rely on these previews. In any case, if people are *really* concerned about this, we should just put it there. My

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Luciano Resende
On Mon, Jun 6, 2016 at 10:08 AM, Mark Hamstra wrote: > I still don't know where this "severely compromised builds of limited >> usefulness" thing comes from? what's so bad? You didn't veto its >> release, after all. > > > I simply mean that it was released with the knowledge that there are still

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Luciano Resende
On Mon, Jun 6, 2016 at 9:51 AM, Sean Owen wrote: > I still don't know where this "severely compromised builds of limited > usefulness" thing comes from? what's so bad? You didn't veto its > release, after all. And rightly so: a release doesn't mean "definitely > works"; it means it was created th

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Mark Hamstra
> > I still don't know where this "severely compromised builds of limited > usefulness" thing comes from? what's so bad? You didn't veto its > release, after all. I simply mean that it was released with the knowledge that there are still significant bugs in the preview that definitely would warra

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Sean Owen
I still don't know where this "severely compromised builds of limited usefulness" thing comes from? what's so bad? You didn't veto its release, after all. And rightly so: a release doesn't mean "definitely works"; it means it was created the right way. It's OK to say it's buggy alpha software; this

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Nicholas Chammas
+1 to what Mark said. I've been following this discussion and I don't understand where the sudden "Databricks vs. everybody else" narrative came from. On Mon, Jun 6, 2016 at 11:00 AM Mark Hamstra wrote: > This is not a Databricks vs. The World situation, and the fact that some > persist in forci

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Mark Hamstra
This is not a Databricks vs. The World situation, and the fact that some persist in forcing every issue into that frame is getting annoying. There are good engineering and project-management reasons not to populate the long-term, canonical repository of Maven artifacts with what are known to be se

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-06 Thread Shane Curcuru
On 2016-06-04 18:42 (-0400), Sean Owen wrote: ... > The question is, can you just not fully release it? I don't think so, > even as a matter of process, and don't see a good reason not to. > > To Reynold's quote, I think that's suggesting that not all projects > will release to a repo at all (e

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-05 Thread Marcin Tustin
+1 agree that right the problem is theoretical esp if the preview label is in the version coordinates as it should be. On Saturday, June 4, 2016, Sean Owen wrote: > Artifacts that are not for public consumption shouldn't be in a public > release; this is instead what nightlies are for. However,

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-05 Thread Ovidiu-Cristian MARCU
Hi all IMHO the preview ‘release’ is good at is is now, so no further changes required. For me the preview was a trigger to what will be the next Spark 2.0, really appreciate the effort guys made to describe it and market it:) I’ll appreciate if the Apache Spark team will start a vote for a new

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-04 Thread Sean Owen
Artifacts that are not for public consumption shouldn't be in a public release; this is instead what nightlies are for. However, this was a normal public release. I am not even sure why it's viewed as particularly unsafe, but, unsafe alpha and beta releases are just releases, and their name and do

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-04 Thread Matei Zaharia
Personally I'd just put them on the staging repo and link to that on the downloads page. It will create less confusion for people browsing Maven Central later and wondering which releases are safe to use. Matei > On Jun 3, 2016, at 8:22 AM, Mark Hamstra wrote: > > It's not a question of wheth

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-03 Thread Mark Hamstra
It's not a question of whether the preview artifacts can be made available on Maven central, but rather whether they must be or should be. I've got no problems leaving these unstable, transitory artifacts out of the more permanent, canonical repository. On Fri, Jun 3, 2016 at 1:53 AM, Steve Lough

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-03 Thread Steve Loughran
It's been voted on by the project, so can go up on central There's already some JIRAs being filed against it, this is a metric of success as pre-beta of the artifacts. The risk of exercising the m2 central option is that people may get expectations that they can point their code at the 2.0.0-p

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-02 Thread Reynold Xin
One thing we can do is to do monthly milestone releases, similar to other projects (e.g. Scala). So we can have Apache Spark 2.1.0-M1, Apache Spark 2.1.0-M2. On Thu, Jun 2, 2016 at 12:42 PM, Tom Graves wrote: > The documentation for the preview release also seem to be missing? > > Also what

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-02 Thread Tom Graves
The documentation for the preview release also seem to be missing? Also what happens if we want to do a second preview release?  The naming doesn't seem to allow then unless we call it preview 2. Tom On Wednesday, June 1, 2016 6:27 PM, Sean Owen wrote: On Wed, Jun 1, 2016 at 5:58 PM, Re

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Sean Owen
On Wed, Jun 1, 2016 at 5:58 PM, Reynold Xin wrote: > The preview release is available here: > http://spark.apache.org/downloads.html (there is an entire section dedicated > to it and also there is a news link to it on the right). Oops, it is indeed down there at the bottom, before nightlies. I ho

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Reynold Xin
Hi Sean, (writing this email with my Apache hat on only and not Databricks hat) The preview release is available here: http://spark.apache.org/downloads.html (there is an entire section dedicated to it and also there is a news link to it on the right). Again, I think this is a good opportunity t

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Sean Owen
I'll be more specific about the issue that I think trumps all this, which I realize maybe not everyone was aware of. There was a long and contentious discussion on the PMC about, among other things, advertising a "Spark 2.0 preview" from Databricks, such as at https://databricks.com/blog/2016/05/

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Michael Armbrust
> > I'd think we want less effort, not more, to let people test it? for > example, right now I can't easily try my product build against > 2.0.0-preview. I don't feel super strongly one way or the other, so if we need to publish it permanently we can. However, either way you can still test again

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Marcelo Vanzin
On Wed, Jun 1, 2016 at 2:51 PM, Sean Owen wrote: > I'd think we want less effort, not more, to let people test it? for > example, right now I can't easily try my product build against > 2.0.0-preview. While I understand your point of view, I like the extra effort to get to these artifacts because

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Sean Owen
An RC is something that gets voted on, and the final one is turned into a blessed release. I agree that RCs don't get published to Maven Central, but releases do of course. This was certainly to be an official release, right? A beta or alpha can still be an official, published release. The proxima

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Jonathan Kelly
I think what Reynold probably means is that previews are releases for which a vote *passed*. ~ Jonathan On Wed, Jun 1, 2016 at 1:53 PM Marcelo Vanzin wrote: > So are RCs, aren't they? > > Personally I'm fine with not releasing to maven central. Any extra > effort needed by regular users to use

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Marcelo Vanzin
So are RCs, aren't they? Personally I'm fine with not releasing to maven central. Any extra effort needed by regular users to use a preview / RC is good with me. On Wed, Jun 1, 2016 at 1:50 PM, Reynold Xin wrote: > To play devil's advocate, previews are technically not RCs. They are > actually v

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Reynold Xin
To play devil's advocate, previews are technically not RCs. They are actually voted releases. On Wed, Jun 1, 2016 at 1:46 PM, Michael Armbrust wrote: > Yeah, we don't usually publish RCs to central, right? > > On Wed, Jun 1, 2016 at 1:06 PM, Reynold Xin wrote: > >> They are here ain't they? >>

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Michael Armbrust
Yeah, we don't usually publish RCs to central, right? On Wed, Jun 1, 2016 at 1:06 PM, Reynold Xin wrote: > They are here ain't they? > > https://repository.apache.org/content/repositories/orgapachespark-1182/ > > Did you mean publishing them to maven central? My understanding is that > publishin

Re: Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Reynold Xin
They are here ain't they? https://repository.apache.org/content/repositories/orgapachespark-1182/ Did you mean publishing them to maven central? My understanding is that publishing to maven central isn't a required step of doing theses. This might be a good opportunity to discuss that. My thought

Spark 2.0.0-preview artifacts still not available in Maven

2016-06-01 Thread Sean Owen
Just checked and they are still not published this week. Can these be published ASAP to complete the 2.0.0-preview release? - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark