@Robert: I have a little storm experience. I will try to run some examples on the cluster.
Peter 2015-05-12 11:40 GMT+02:00 Matthias J. Sax <mj...@informatik.hu-berlin.de>: > Hi, > > some UnsupportedOperationExceptions are required, because Storm > interfaces are implement but Flink cannot support those functionality. > Some other are "not yet implemented" once. > > A few other of them could be removed (in case an interface in not > implemented, but only mimicked), by removing the whole method. I prefer > to mimic interfaced completely and through > UnsupportedOperationExceptions, because if a user wants to execute Storm > code on Flink, less changes to the original Storm code are necessary, > making the transition easier. > > Of course, it is a valid argument to remove the methods completely to > raise incompatibilities directly are compile time. > > The TODOs are points that might or might not be Flink compatible. This > must be checked and maybe discussed. > > Any feedback is welcome. > > > -Matthias > > On 05/12/2015 10:46 AM, Robert Metzger wrote: > > Hi, > > > > Thank you for starting the discussion Marton! > > > > I would really like to merge the storm compat to our source repo. I think > > that code which is not merged there will not get enough attention. > > > > I'm against splitting flink-contrib into small maven modules. I totally > > understand your reasoning (mixed dependencies), but "flink-staging" > exists > > exactly for that purpose (one maven module per "beta"-module). > > For now, users depending on flink-contrib have to define exclusions to > > control the dependencies. > > > > So I'm +1 for merging it to "flink-contrib". > > I guess not all committers have time to look into the pull request, > > therefore, I want to remind you that the code contains a lot of TODOs and > > UnsupportedOperationExceptions. > > > > @Marton: Do you know anybody from the Budapest Flink Streaming crew with > > some Storm experience who could try out the code on a cluster and give > some > > feedback? > > > > > > On Tue, May 12, 2015 at 9:52 AM, Márton Balassi < > balassi.mar...@gmail.com> > > wrote: > > > >> The purpose of flink-contrib currently is to hold contributions to the > >> project that we do not consider part of the core flink functionality, > but > >> provide useful tools around it. In general code placed here has to meet > >> less requirements in terms of covering all corner cases if it provides a > >> nice solution for a set of well defined problems. > >> > >> As of today it has two small utilities, the TweetInputFormat (by Mustafa > >> Elbehery) and the collect functionality for the DataStream (by Gabor > >> Gevay). > >> > >> The pull request for the Storm compatibility layer (by Matthias J. Sax) > [1] > >> raises the issue as it is way more code to maintain and is more complex > in > >> general that how the community would like to handle these in terms of > >> distribution. Do we want to have it in the Flink repository or maybe in > a > >> separate one. > >> > >> I am personally really for having the Storm compatibility layer under > >> flink-contrib as Matthias is very active on the mailing list and has > also > >> expressed his interest of further developing the functionality of the > >> compatibility layer. To top that a couple of users got excited about the > >> new feature, so I see no risk in having this code in the main > repository. > >> > >> As for the structure of flink-contrib I would have the contents > separated > >> to slim as possible maven projects, to make sure that the users only get > >> the dependencies that they really need. > >> > >> [1] https://github.com/apache/flink/pull/573 > >> > > > >
