+1 for your pragmatic approach, Vasia. A simple collection of third party software using Flink should be enough for now; of course, outside the Apache realm.
On Thu, Oct 8, 2015 at 12:45 PM, Chiwan Park <chiwanp...@apache.org> wrote: > +1 for Vasia’s suggestion. From a long-term perspective, the site like Spark > Packages [1] would be helpful to manage external contribution. > > [1] http://spark-packages.org > >> On Oct 8, 2015, at 12:28 PM, Matthias J. Sax <mj...@apache.org> wrote: >> >> Thanks for the feedback. >> >> I think, the repository does not need to build on a single Flink >> release. From my point of view, there should be a single parent module >> that contains *independent modules* for each extension/library (there >> should be no "cross dependencies" between the modules and each module >> can specify the flink dependencies it needs by itself). This make is >> most flexible. And if a library works on an old release, it might just >> stay there as is. If a library changes (due to Flink changes), it might >> just be contained multiple times for different Flink releases. >> >> Each module should provide a short doc (README) that shows how to use an >> integrate it with Flink. Thus, the responsibility goes to the >> contributor to maintain the library. If it breaks and is not maintained >> any further, we can simple remove it. >> >> I agree, that the community might not be able to maintain those >> extension/libraries right now. I would put the responsibility (more or >> less completely) on the contributor and delete project that do not fix >> any more. >> >> @Vasia: a link to a library could be included in the README. If anybody >> only wants to share a library but not contribute code, the parent README >> could contain a list of additional links. >> >> >> -Matthias >> >> >> On 10/08/2015 12:15 PM, Vasiliki Kalavri wrote: >>> How about, for now, we simply create a page where we gather links/short >>> descriptions of all these contributions >>> and let the maintenance and dependency management to the tool/library >>> creators? >>> This way we will at least have these contributions in one place and link to >>> them somewhere from the website. >>> >>> -Vasia. >>> >>> On 8 October 2015 at 12:06, Maximilian Michels <m...@apache.org> wrote: >>> >>>> Hi Matthias, >>>> >>>> Thanks for bringing up this idea. Actually, it has been discussed a >>>> couple of times on the mailing list whether we should have a central >>>> place for third-party extensions/contributions/libraries. This could >>>> either be something package-based or, like you proposed, another >>>> repository. >>>> >>>> An external place for contributions raises a couple of questions >>>> >>>> - Which version should the external contributions be based on? >>>> - How do we make sure, the extensions are continuously updated? >>>> (dedicated maintainers or automatic compatibility checks) >>>> - How do we easily plug-in the external modules into Flink? >>>> >>>> In the long term, we really need a solution for these questions. The >>>> code base of Flink is growing and more and more packages go to >>>> flink-contrib/flink-staging. I would find something packaged-based >>>> better than a repository. Quite frankly, momentarily, I think >>>> developing such a plugin system is out of scope for most Flink >>>> developers. At the current pace of Flink development, collecting these >>>> contributions externally without properly maintaining them, doesn't >>>> make much sense to me. >>>> >>>> Cheers, >>>> Max >>>> >>>> >>>> >>>> On Wed, Oct 7, 2015 at 11:42 AM, Matthias J. Sax <mj...@apache.org> wrote: >>>>> >>>>> Hi, >>>>> >>>>> many people are building quite exiting stuff on top of Flink. It is hard >>>>> to keep an good overview on what stuff is available and what not. What >>>>> do you think about starting a second git repository "flink-external" >>>>> that collects all those code? >>>>> >>>>> The ideas would be to collect stuff in a central point, such that people >>>>> can access it easily and get an overview what is already available (this >>>>> might also avoid duplicate development). It might also be a good point >>>>> to show common patterns. In order to collect as much as possible, the >>>>> contributing requirement (with respect to testing etc) could be lower >>>>> than for Flink itself. >>>>> >>>>> For example, I recently started a small flink-clojure module with a >>>>> simple word-count example to answer a question on SO. Including this in >>>>> Flink would not be appropriate. However, for a flink-external repro it >>>>> might be nice to have. >>>>> >>>>> What do you think about it? >>>>> >>>>> >>>>> -Matthias >>>>> >>>> >>> >> > > > > Regards, > Chiwan Park > > >
