Hey guys, I think the JIRA associated to this discussion needs some attention: https://issues.apache.org/jira/browse/FLINK-1452
On Mon, Jan 26, 2015 at 2:24 PM, Max Michels <m...@data-artisans.com> wrote: > +1 for having an optional flink-contrib maven dependency and an > extension repository in the long run. > > On Mon, Jan 26, 2015 at 12:00 PM, Robert Metzger <rmetz...@apache.org> > wrote: > > I've added a JIRA issue to create the module: > > https://issues.apache.org/jira/browse/FLINK-1452 > > > > > > On Mon, Jan 26, 2015 at 11:39 AM, Till Rohrmann <trohrm...@apache.org> > > wrote: > > > >> +1 for Robert's proposal. > >> > >> On Mon, Jan 26, 2015 at 5:43 AM, Henry Saputra <henry.sapu...@gmail.com > > > >> wrote: > >> > >> > +1 for the idea. > >> > > >> > We need to make sure PMC of Flink maintains knowledge of standard > >> > Flink distribution, hence the "flink-contrib" should not be part of > >> > the release. > >> > > >> > - Henry > >> > > >> > On Sun, Jan 25, 2015 at 10:33 AM, Robert Metzger <rmetz...@apache.org > > > >> > wrote: > >> > > I'm also in favor of option 1) with a "flink-contrib" maven module. > >> > > > >> > > I agree with Ted that we should certainly think about establishing a > >> > highly > >> > > visible, easy to contribute and easy to use infrastructure for all > >> kinds > >> > of > >> > > contributions around the project. > >> > > But I suspect that we need some time to come up with a good > >> architecture > >> > > and infrastructure for that. Maybe this also comes as an outside > >> > > contribution to Flink? > >> > > > >> > > To have something immediately, we should start with a > "flink-contrib" > >> > > module. > >> > > > >> > > One thing that I would like to discuss first is a clear set of rules > >> for > >> > > contributions into that module. > >> > > Code contributions to "flink-contrib" need: > >> > > - to be tested on a cluster (not only by single-jvm tests) > >> > > - to have test cases (because otherwise we can not guarantee that > they > >> > > build with our changes > >> > > - to be of use for others > >> > > - to have some documentation > >> > > > >> > > I would not deploy the flink-contrib package in the standard flink > >> > > distribution. Users will have to add them as a maven dependency. > >> > > > >> > > > >> > > > >> > > > >> > > On Sat, Jan 24, 2015 at 11:02 PM, Ted Dunning < > ted.dunn...@gmail.com> > >> > wrote: > >> > > > >> > >> As the community of flink add-ons grows, a CPAN or maven-like > >> mechanism > >> > >> might be a nice option. That would let people download and install > >> > >> extensions very fluidly. > >> > >> > >> > >> The argument for making Apache contributions is definitely valid, > but > >> > the > >> > >> argument for the agility of fostering independent projects is that > >> > projects > >> > >> can gain lots of popularity very quickly this way. CPAN, CRAN, > pip, > >> > maven > >> > >> and RubyGems can be argued to be critical components of the > popularity > >> > of > >> > >> Perl, R, Python, Java/Scala and Ruby respectively. > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> On Sat, Jan 24, 2015 at 4:26 PM, Fabian Hueske <fhue...@gmail.com> > >> > wrote: > >> > >> > >> > >> > I am also more in favor of option 1). > >> > >> > > >> > >> > 2015-01-24 20:27 GMT+01:00 Kostas Tzoumas <ktzou...@apache.org>: > >> > >> > > >> > >> > > Thanks Fabian for starting the discussion. > >> > >> > > > >> > >> > > I would be biased towards option (1) that Stephan highlighted > for > >> > the > >> > >> > > following reasons: > >> > >> > > > >> > >> > > - A separate github project is one more infrastructure to > manage, > >> > and > >> > >> it > >> > >> > > lives outside the ASF. I would like to bring as much code as > >> > possible > >> > >> to > >> > >> > > the Apache Software Foundation, and not divide the codebase > into > >> two > >> > >> > > separate repositories. > >> > >> > > > >> > >> > > - The personal gratification (and thus motivation) is higher > when > >> > >> > > contributing to a top-level Apache project than a github > >> repository > >> > >> > > slightly associated with an ASF project. And contributors to > the > >> > Flink > >> > >> > > project get karma that may lead to new committers, which is > >> crucial > >> > as > >> > >> > the > >> > >> > > project is growing. > >> > >> > > > >> > >> > > Of course, non Apache-licensed contributions cannot be > accepted. > >> If > >> > we > >> > >> > have > >> > >> > > a good amount of those, we can start an infrastructure for > Flink > >> > >> packages > >> > >> > > that lives outside the ASF, but I would wait for the need to > come > >> > >> before > >> > >> > > doing this. > >> > >> > > > >> > >> > > My proposal would be to funnel contributions to the main > >> repository > >> > >> (in a > >> > >> > > flink-contrib module) for now, including the recent > contributions. > >> > >> > > > >> > >> > > Kostas > >> > >> > > > >> > >> > > > >> > >> > > On Sat, Jan 24, 2015 at 11:15 AM, Stephan Ewen < > se...@apache.org> > >> > >> wrote: > >> > >> > > > >> > >> > > > Yes, a "flink-contrib" project would be great. > >> > >> > > > > >> > >> > > > We have two options: > >> > >> > > > > >> > >> > > > 1) Make it part of the flink apache project. > >> > >> > > > - PRO this makes it easy to get stuff for users > >> > >> > > > - CONTRA this means stronger requirements on the code, > blocker > >> > for > >> > >> > code > >> > >> > > > that uses dependencies under certain licenses, etc. > >> > >> > > > > >> > >> > > > 2) Make an independent github project. > >> > >> > > > - PRO contributions can depended on more licenses, such as > LGPL > >> > >> > > > - PRO we can have more people that commit to this repo, > >> > committers > >> > >> can > >> > >> > > be > >> > >> > > > different from flink committers > >> > >> > > > - CONTRA people need to grab the extensions from a different > >> > >> location > >> > >> > > > > >> > >> > > > > >> > >> > > > I am slightly biased towards (2), but open to both. > >> > >> > > > > >> > >> > > > Stephan > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > On Sat, Jan 24, 2015 at 5:29 AM, Chiwan Park < > >> > chiwanp...@icloud.com> > >> > >> > > > wrote: > >> > >> > > > > >> > >> > > > > I think top level maven module called "flink-contrib" is > >> > >> reasonable. > >> > >> > > > There > >> > >> > > > > are other projects having contrib package such as Akka, > >> Django. > >> > >> > > > > > >> > >> > > > > Regards, Chiwan Park (Sent with iPhone) > >> > >> > > > > > >> > >> > > > > 2015. 1. 24. 오후 7:15 Fabian Hueske <fhue...@gmail.com> 작성: > >> > >> > > > > > >> > >> > > > > > Hi all, > >> > >> > > > > > > >> > >> > > > > > we got a few contribution requests lately to add cool but > >> > >> > "non-core" > >> > >> > > > > > features to our API. > >> > >> > > > > > In previous discussions, concerns were raised to not > bloat > >> the > >> > >> APIs > >> > >> > > > with > >> > >> > > > > > too many "shortcut", "syntactic sugar", or special-case > >> > features. > >> > >> > > > > > > >> > >> > > > > > Instead we could setup a place to add > Input/OutputFormats, > >> > common > >> > >> > > > > > operations, etc. which does not need as much control as > the > >> > core > >> > >> > > APIs. > >> > >> > > > > Open > >> > >> > > > > > questions are: > >> > >> > > > > > - How do we organize it? (top-level maven module, > modules in > >> > >> > > > flink-java, > >> > >> > > > > > flink-scala, java packages in the API modules, ...) > >> > >> > > > > > - How do we name it? flink-utils, flink-packages, ... > >> > >> > > > > > > >> > >> > > > > > Any opinions on this? > >> > >> > > > > > > >> > >> > > > > > Cheers, Fabian > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > > >> >
