On Saturday, March 26, 2016, Sean Owen <so...@cloudera.com> wrote: > This has been resolved; see the JIRA and related PRs but also > > http://apache-spark-developers-list.1001551.n3.nabble.com/SPARK-13843-Next-steps-td16783.html > > This change happened subsequent to current thread (thanks Marcelo) and could as well have gone unnoticed until release vote.
> This is not a scenario where a [VOTE] needs to take place, and code > changes don't proceed through PMC votes. From the project perspective, > code was deleted/retired for lack of interest, and this is controlled > by the normal lazy consensus protocol which wasn't vetoed. I have not seen Apache owned artifacts moved out of it's governance without discussion - this was not refactoring or cleanup (as was suggested disingenuously) but migration of submodules/functionality (though from Reynold's clarification, looks like for good enough reasons). A vote might or might not have required but a discussion must have happened - atleast going forward, it will help us not to miss things (artifact and project namespace, license, ownership, release cycle, version compatibility, etc of the sub project could be of interest to users and developers). Regards Mridul > The subsequent discussion was in part about whether other modules > should go, or whether one should come back, which it did. The latter > suggests that change could have been left open for some discussion > longer. Ideally, you would have commented before the initial change > happened, but it sounds like several people would have liked more > time. I don't think I'd call that "improper conduct" though, no. It > was reversed via the same normal code management process. > > The rest of the question concerned what becomes of the code that was > removed. It was revived outside the project for anyone who cares to > continue collaborating. There seemed to be no disagreement about that, > mostly because the code in question was of minimal interest. PMC > doesn't need to rule on anything. There may still be some loose ends > there like namespace changes. I'll add to the other thread about this. > > > > On Sat, Mar 26, 2016 at 1:17 PM, Jacek Laskowski <ja...@japila.pl > <javascript:;>> wrote: > > Hi, > > > > Although I'm not that much experienced member of ASF, I share your > > concerns. I haven't looked at the issue from this point of view, but > > after having read the thread I think PMC should've signed off the > > migration of ASF-owned code to a non-ASF repo. At least a vote is > > required (and this discussion is a sign that the process has not been > > conducted properly as people have concerns, me including). > > > > Thanks Mridul! > > > > Pozdrawiam, > > Jacek Laskowski > > ---- > > https://medium.com/@jaceklaskowski/ > > Mastering Apache Spark http://bit.ly/mastering-apache-spark > > Follow me at https://twitter.com/jaceklaskowski > > > > > > On Thu, Mar 17, 2016 at 9:13 PM, Mridul Muralidharan <mri...@gmail.com > <javascript:;>> wrote: > >> I am not referring to code edits - but to migrating submodules and > >> code currently in Apache Spark to 'outside' of it. > >> If I understand correctly, assets from Apache Spark are being moved > >> out of it into thirdparty external repositories - not owned by Apache. > >> > >> At a minimum, dev@ discussion (like this one) should be initiated. > >> As PMC is responsible for the project assets (including code), signoff > >> is required for it IMO. > >> > >> More experienced Apache members might be opine better in case I got it > wrong ! > >> > >> > >> Regards, > >> Mridul > >> > >> > >> On Thu, Mar 17, 2016 at 12:55 PM, Cody Koeninger <c...@koeninger.org > <javascript:;>> wrote: > >>> Why would a PMC vote be necessary on every code deletion? > >>> > >>> There was a Jira and pull request discussion about the submodules that > >>> have been removed so far. > >>> > >>> https://issues.apache.org/jira/browse/SPARK-13843 > >>> > >>> There's another ongoing one about Kafka specifically > >>> > >>> https://issues.apache.org/jira/browse/SPARK-13877 > >>> > >>> > >>> On Thu, Mar 17, 2016 at 2:49 PM, Mridul Muralidharan <mri...@gmail.com > <javascript:;>> wrote: > >>>> > >>>> I was not aware of a discussion in Dev list about this - agree with > most of > >>>> the observations. > >>>> In addition, I did not see PMC signoff on moving (sub-)modules out. > >>>> > >>>> Regards > >>>> Mridul > >>>> > >>>> > >>>> > >>>> On Thursday, March 17, 2016, Marcelo Vanzin <van...@cloudera.com > <javascript:;>> wrote: > >>>>> > >>>>> Hello all, > >>>>> > >>>>> Recently a lot of the streaming backends were moved to a separate > >>>>> project on github and removed from the main Spark repo. > >>>>> > >>>>> While I think the idea is great, I'm a little worried about the > >>>>> execution. Some concerns were already raised on the bug mentioned > >>>>> above, but I'd like to have a more explicit discussion about this so > >>>>> things don't fall through the cracks. > >>>>> > >>>>> Mainly I have three concerns. > >>>>> > >>>>> i. Ownership > >>>>> > >>>>> That code used to be run by the ASF, but now it's hosted in a github > >>>>> repo owned not by the ASF. That sounds a little sub-optimal, if not > >>>>> problematic. > >>>>> > >>>>> ii. Governance > >>>>> > >>>>> Similar to the above; who has commit access to the above repos? Will > >>>>> all the Spark committers, present and future, have commit access to > >>>>> all of those repos? Are they still going to be considered part of > >>>>> Spark and have release management done through the Spark community? > >>>>> > >>>>> > >>>>> For both of the questions above, why are they not turned into > >>>>> sub-projects of Spark and hosted on the ASF repos? I believe there is > >>>>> a mechanism to do that, without the need to keep the code in the main > >>>>> Spark repo, right? > >>>>> > >>>>> iii. Usability > >>>>> > >>>>> This is another thing I don't see discussed. For Scala-based code > >>>>> things don't change much, I guess, if the artifact names don't change > >>>>> (another reason to keep things in the ASF?), but what about python? > >>>>> How are pyspark users expected to get that code going forward, since > >>>>> it's not in Spark's pyspark.zip anymore? > >>>>> > >>>>> > >>>>> Is there an easy way of keeping these things within the ASF Spark > >>>>> project? I think that would be better for everybody. > >>>>> > >>>>> -- > >>>>> Marcelo > >>>>> > >>>>> --------------------------------------------------------------------- > >>>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > <javascript:;> > >>>>> For additional commands, e-mail: dev-h...@spark.apache.org > <javascript:;> > >>>>> > >>>> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org <javascript:;> > >> For additional commands, e-mail: dev-h...@spark.apache.org > <javascript:;> > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org <javascript:;> > > For additional commands, e-mail: dev-h...@spark.apache.org > <javascript:;> > > >
