Hi, Thank you for starting the discussion Marton!
I would really like to merge the storm compat to our source repo. I think that code which is not merged there will not get enough attention. I'm against splitting flink-contrib into small maven modules. I totally understand your reasoning (mixed dependencies), but "flink-staging" exists exactly for that purpose (one maven module per "beta"-module). For now, users depending on flink-contrib have to define exclusions to control the dependencies. So I'm +1 for merging it to "flink-contrib". I guess not all committers have time to look into the pull request, therefore, I want to remind you that the code contains a lot of TODOs and UnsupportedOperationExceptions. @Marton: Do you know anybody from the Budapest Flink Streaming crew with some Storm experience who could try out the code on a cluster and give some feedback? On Tue, May 12, 2015 at 9:52 AM, Márton Balassi <balassi.mar...@gmail.com> wrote: > The purpose of flink-contrib currently is to hold contributions to the > project that we do not consider part of the core flink functionality, but > provide useful tools around it. In general code placed here has to meet > less requirements in terms of covering all corner cases if it provides a > nice solution for a set of well defined problems. > > As of today it has two small utilities, the TweetInputFormat (by Mustafa > Elbehery) and the collect functionality for the DataStream (by Gabor > Gevay). > > The pull request for the Storm compatibility layer (by Matthias J. Sax) [1] > raises the issue as it is way more code to maintain and is more complex in > general that how the community would like to handle these in terms of > distribution. Do we want to have it in the Flink repository or maybe in a > separate one. > > I am personally really for having the Storm compatibility layer under > flink-contrib as Matthias is very active on the mailing list and has also > expressed his interest of further developing the functionality of the > compatibility layer. To top that a couple of users got excited about the > new feature, so I see no risk in having this code in the main repository. > > As for the structure of flink-contrib I would have the contents separated > to slim as possible maven projects, to make sure that the users only get > the dependencies that they really need. > > [1] https://github.com/apache/flink/pull/573 >
