Should we add a new page at Flink project web page? On 10/08/2015 12:56 PM, Maximilian Michels wrote: > +1 for your pragmatic approach, Vasia. A simple collection of third > party software using Flink should be enough for now; of course, > outside the Apache realm. > > On Thu, Oct 8, 2015 at 12:45 PM, Chiwan Park <chiwanp...@apache.org> wrote: >> +1 for Vasia’s suggestion. From a long-term perspective, the site like Spark >> Packages [1] would be helpful to manage external contribution. >> >> [1] http://spark-packages.org >> >>> On Oct 8, 2015, at 12:28 PM, Matthias J. Sax <mj...@apache.org> wrote: >>> >>> Thanks for the feedback. >>> >>> I think, the repository does not need to build on a single Flink >>> release. From my point of view, there should be a single parent module >>> that contains *independent modules* for each extension/library (there >>> should be no "cross dependencies" between the modules and each module >>> can specify the flink dependencies it needs by itself). This make is >>> most flexible. And if a library works on an old release, it might just >>> stay there as is. If a library changes (due to Flink changes), it might >>> just be contained multiple times for different Flink releases. >>> >>> Each module should provide a short doc (README) that shows how to use an >>> integrate it with Flink. Thus, the responsibility goes to the >>> contributor to maintain the library. If it breaks and is not maintained >>> any further, we can simple remove it. >>> >>> I agree, that the community might not be able to maintain those >>> extension/libraries right now. I would put the responsibility (more or >>> less completely) on the contributor and delete project that do not fix >>> any more. >>> >>> @Vasia: a link to a library could be included in the README. If anybody >>> only wants to share a library but not contribute code, the parent README >>> could contain a list of additional links. >>> >>> >>> -Matthias >>> >>> >>> On 10/08/2015 12:15 PM, Vasiliki Kalavri wrote: >>>> How about, for now, we simply create a page where we gather links/short >>>> descriptions of all these contributions >>>> and let the maintenance and dependency management to the tool/library >>>> creators? >>>> This way we will at least have these contributions in one place and link to >>>> them somewhere from the website. >>>> >>>> -Vasia. >>>> >>>> On 8 October 2015 at 12:06, Maximilian Michels <m...@apache.org> wrote: >>>> >>>>> Hi Matthias, >>>>> >>>>> Thanks for bringing up this idea. Actually, it has been discussed a >>>>> couple of times on the mailing list whether we should have a central >>>>> place for third-party extensions/contributions/libraries. This could >>>>> either be something package-based or, like you proposed, another >>>>> repository. >>>>> >>>>> An external place for contributions raises a couple of questions >>>>> >>>>> - Which version should the external contributions be based on? >>>>> - How do we make sure, the extensions are continuously updated? >>>>> (dedicated maintainers or automatic compatibility checks) >>>>> - How do we easily plug-in the external modules into Flink? >>>>> >>>>> In the long term, we really need a solution for these questions. The >>>>> code base of Flink is growing and more and more packages go to >>>>> flink-contrib/flink-staging. I would find something packaged-based >>>>> better than a repository. Quite frankly, momentarily, I think >>>>> developing such a plugin system is out of scope for most Flink >>>>> developers. At the current pace of Flink development, collecting these >>>>> contributions externally without properly maintaining them, doesn't >>>>> make much sense to me. >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>> >>>>> >>>>> On Wed, Oct 7, 2015 at 11:42 AM, Matthias J. Sax <mj...@apache.org> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> many people are building quite exiting stuff on top of Flink. It is hard >>>>>> to keep an good overview on what stuff is available and what not. What >>>>>> do you think about starting a second git repository "flink-external" >>>>>> that collects all those code? >>>>>> >>>>>> The ideas would be to collect stuff in a central point, such that people >>>>>> can access it easily and get an overview what is already available (this >>>>>> might also avoid duplicate development). It might also be a good point >>>>>> to show common patterns. In order to collect as much as possible, the >>>>>> contributing requirement (with respect to testing etc) could be lower >>>>>> than for Flink itself. >>>>>> >>>>>> For example, I recently started a small flink-clojure module with a >>>>>> simple word-count example to answer a question on SO. Including this in >>>>>> Flink would not be appropriate. However, for a flink-external repro it >>>>>> might be nice to have. >>>>>> >>>>>> What do you think about it? >>>>>> >>>>>> >>>>>> -Matthias >>>>>> >>>>> >>>> >>> >> >> >> >> Regards, >> Chiwan Park >> >> >>
signature.asc
Description: OpenPGP digital signature
