I was not sure what we should add and was hoping for input from the community.
I am aware of the following projects we might want to add: - Zeppelin - SAMOA - Mahout - Cascading (dataartisan repo) - BigPetStore - Gradoop -Matthias On 10/09/2015 03:07 PM, Maximilian Michels wrote: > Cool. Right now the list is empty. Do you already have a list you > could include in the upcoming pull request? :) > > On Fri, Oct 9, 2015 at 2:29 PM, Matthias J. Sax <mj...@apache.org> wrote: >> Hi, >> >> I just started this. Please see >> https://github.com/mjsax/flink-web/tree/flink-external-page >> >> I think, it is the best way to extend the "Downloads" page. I would also >> add a link to this on the main page's "Getting Started" section. >> >> As a first try, I started like this: >>> Third party packages >>> >>> This is a list of third party packages (ie, libraries, system extensions, >>> or examples) build for Flink. The Flink community only collects links to >>> those packages but does not maintain them. Thus, they do not belong to the >>> Apache Flink project and the community cannot give any support for them. >>> Package Name >>> >>> Available for Flink 0.8.x and 0.9.x >>> >>> Short description >>> >>> Please let us know, if we missed to list your package. Be aware, that we >>> might remove listed packages without notice. >> >> Can you please give me some input, what projects I should add initially? >> >> >> -Matthias >> >> >> On 10/08/2015 04:03 PM, Maximilian Michels wrote: >>> IMHO we can do that. There should be a disclaimer that the third party >>> software is not officially supported. >>> >>> On Thu, Oct 8, 2015 at 2:25 PM, Matthias J. Sax <mj...@apache.org> wrote: >>>> Should we add a new page at Flink project web page? >>>> >>>> On 10/08/2015 12:56 PM, Maximilian Michels wrote: >>>>> +1 for your pragmatic approach, Vasia. A simple collection of third >>>>> party software using Flink should be enough for now; of course, >>>>> outside the Apache realm. >>>>> >>>>> On Thu, Oct 8, 2015 at 12:45 PM, Chiwan Park <chiwanp...@apache.org> >>>>> wrote: >>>>>> +1 for Vasia’s suggestion. From a long-term perspective, the site like >>>>>> Spark Packages [1] would be helpful to manage external contribution. >>>>>> >>>>>> [1] http://spark-packages.org >>>>>> >>>>>>> On Oct 8, 2015, at 12:28 PM, Matthias J. Sax <mj...@apache.org> wrote: >>>>>>> >>>>>>> Thanks for the feedback. >>>>>>> >>>>>>> I think, the repository does not need to build on a single Flink >>>>>>> release. From my point of view, there should be a single parent module >>>>>>> that contains *independent modules* for each extension/library (there >>>>>>> should be no "cross dependencies" between the modules and each module >>>>>>> can specify the flink dependencies it needs by itself). This make is >>>>>>> most flexible. And if a library works on an old release, it might just >>>>>>> stay there as is. If a library changes (due to Flink changes), it might >>>>>>> just be contained multiple times for different Flink releases. >>>>>>> >>>>>>> Each module should provide a short doc (README) that shows how to use an >>>>>>> integrate it with Flink. Thus, the responsibility goes to the >>>>>>> contributor to maintain the library. If it breaks and is not maintained >>>>>>> any further, we can simple remove it. >>>>>>> >>>>>>> I agree, that the community might not be able to maintain those >>>>>>> extension/libraries right now. I would put the responsibility (more or >>>>>>> less completely) on the contributor and delete project that do not fix >>>>>>> any more. >>>>>>> >>>>>>> @Vasia: a link to a library could be included in the README. If anybody >>>>>>> only wants to share a library but not contribute code, the parent README >>>>>>> could contain a list of additional links. >>>>>>> >>>>>>> >>>>>>> -Matthias >>>>>>> >>>>>>> >>>>>>> On 10/08/2015 12:15 PM, Vasiliki Kalavri wrote: >>>>>>>> How about, for now, we simply create a page where we gather links/short >>>>>>>> descriptions of all these contributions >>>>>>>> and let the maintenance and dependency management to the tool/library >>>>>>>> creators? >>>>>>>> This way we will at least have these contributions in one place and >>>>>>>> link to >>>>>>>> them somewhere from the website. >>>>>>>> >>>>>>>> -Vasia. >>>>>>>> >>>>>>>> On 8 October 2015 at 12:06, Maximilian Michels <m...@apache.org> wrote: >>>>>>>> >>>>>>>>> Hi Matthias, >>>>>>>>> >>>>>>>>> Thanks for bringing up this idea. Actually, it has been discussed a >>>>>>>>> couple of times on the mailing list whether we should have a central >>>>>>>>> place for third-party extensions/contributions/libraries. This could >>>>>>>>> either be something package-based or, like you proposed, another >>>>>>>>> repository. >>>>>>>>> >>>>>>>>> An external place for contributions raises a couple of questions >>>>>>>>> >>>>>>>>> - Which version should the external contributions be based on? >>>>>>>>> - How do we make sure, the extensions are continuously updated? >>>>>>>>> (dedicated maintainers or automatic compatibility checks) >>>>>>>>> - How do we easily plug-in the external modules into Flink? >>>>>>>>> >>>>>>>>> In the long term, we really need a solution for these questions. The >>>>>>>>> code base of Flink is growing and more and more packages go to >>>>>>>>> flink-contrib/flink-staging. I would find something packaged-based >>>>>>>>> better than a repository. Quite frankly, momentarily, I think >>>>>>>>> developing such a plugin system is out of scope for most Flink >>>>>>>>> developers. At the current pace of Flink development, collecting these >>>>>>>>> contributions externally without properly maintaining them, doesn't >>>>>>>>> make much sense to me. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Max >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Oct 7, 2015 at 11:42 AM, Matthias J. Sax <mj...@apache.org> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> many people are building quite exiting stuff on top of Flink. It is >>>>>>>>>> hard >>>>>>>>>> to keep an good overview on what stuff is available and what not. >>>>>>>>>> What >>>>>>>>>> do you think about starting a second git repository "flink-external" >>>>>>>>>> that collects all those code? >>>>>>>>>> >>>>>>>>>> The ideas would be to collect stuff in a central point, such that >>>>>>>>>> people >>>>>>>>>> can access it easily and get an overview what is already available >>>>>>>>>> (this >>>>>>>>>> might also avoid duplicate development). It might also be a good >>>>>>>>>> point >>>>>>>>>> to show common patterns. In order to collect as much as possible, the >>>>>>>>>> contributing requirement (with respect to testing etc) could be lower >>>>>>>>>> than for Flink itself. >>>>>>>>>> >>>>>>>>>> For example, I recently started a small flink-clojure module with a >>>>>>>>>> simple word-count example to answer a question on SO. Including this >>>>>>>>>> in >>>>>>>>>> Flink would not be appropriate. However, for a flink-external repro >>>>>>>>>> it >>>>>>>>>> might be nice to have. >>>>>>>>>> >>>>>>>>>> What do you think about it? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -Matthias >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Regards, >>>>>> Chiwan Park >>>>>> >>>>>> >>>>>> >>>> >>
signature.asc
Description: OpenPGP digital signature
