Hi Matthias, Thanks for bringing up this idea. Actually, it has been discussed a couple of times on the mailing list whether we should have a central place for third-party extensions/contributions/libraries. This could either be something package-based or, like you proposed, another repository.
An external place for contributions raises a couple of questions - Which version should the external contributions be based on? - How do we make sure, the extensions are continuously updated? (dedicated maintainers or automatic compatibility checks) - How do we easily plug-in the external modules into Flink? In the long term, we really need a solution for these questions. The code base of Flink is growing and more and more packages go to flink-contrib/flink-staging. I would find something packaged-based better than a repository. Quite frankly, momentarily, I think developing such a plugin system is out of scope for most Flink developers. At the current pace of Flink development, collecting these contributions externally without properly maintaining them, doesn't make much sense to me. Cheers, Max On Wed, Oct 7, 2015 at 11:42 AM, Matthias J. Sax <mj...@apache.org> wrote: > > Hi, > > many people are building quite exiting stuff on top of Flink. It is hard > to keep an good overview on what stuff is available and what not. What > do you think about starting a second git repository "flink-external" > that collects all those code? > > The ideas would be to collect stuff in a central point, such that people > can access it easily and get an overview what is already available (this > might also avoid duplicate development). It might also be a good point > to show common patterns. In order to collect as much as possible, the > contributing requirement (with respect to testing etc) could be lower > than for Flink itself. > > For example, I recently started a small flink-clojure module with a > simple word-count example to answer a question on SO. Including this in > Flink would not be appropriate. However, for a flink-external repro it > might be nice to have. > > What do you think about it? > > > -Matthias >
