This sounds amazing :) thanks Matthias! Tomorrow I will spend some time to look through your work and give some comments.
Also I would love to help with this effort so once we merge an initial prototype let's open some Jiras and I will pick some up :) Gyula On Thursday, April 2, 2015, Márton Balassi <balassi.mar...@gmail.com> wrote: > Hey Mathias, > > Thanks, this is a really nice contribution. I just scrolled through the > code, but I really like it and big thanks for the the tests for the > examples. > > The rebase Fabian suggested would help a lot when merging. > > > > On Thu, Apr 2, 2015 at 9:19 PM, Fabian Hueske <fhue...@gmail.com > <javascript:;>> wrote: > > > Hi Matthias, > > > > this is really cool!I especially like that you can use Storm code within > a > > Flink streaming program :-) > > > > One thing that might be good to do rather soon is to collect all your > > commits and put them on top of a fresh forked Flink master branch. > > When merging we cannot change the history and try to put only > fast-forward > > commits on top of the master branch. > > As time goes on it becomes more likely that you run into merge issues > when > > cherry-picking the commits. > > > > 2015-04-02 21:09 GMT+02:00 Robert Metzger <rmetz...@apache.org > <javascript:;>>: > > > > > Hey Henry, > > > > > > you can check out the files here: > > > > > > > > > https://github.com/mjsax/flink/tree/flink-storm-compatibility/flink-staging/flink-streaming/flink-storm-compatibility > > > ... so yes, they are located in the flink-streaming directory .. which > > is a > > > good place for now. > > > Once we move flink-streaming out of staging, we might want to keep the > > > storm compat in staging. > > > > > > > > > On Thu, Apr 2, 2015 at 8:16 PM, Henry Saputra <henry.sapu...@gmail.com > <javascript:;>> > > > wrote: > > > > > > > HI Matthias, > > > > > > > > Where do you put the code for the Storm compatibility? Under streams > > > > module directory? > > > > > > > > - Henry > > > > > > > > On Thu, Apr 2, 2015 at 10:31 AM, Matthias J. Sax > > > > <mj...@informatik.hu-berlin.de <javascript:;>> wrote: > > > > > Hi @all, > > > > > > > > > > I started to work on an compatibility layer to run Storm Topologies > > on > > > > > Flink. I just pushed a first beta: > > > > > https://github.com/mjsax/flink/tree/flink-storm-compatibility > > > > > > > > > > Please check it out, and let me know how you like it. In this first > > > > > version, I tried to code without changing too many things in the > > > > > existing code. However, a deeper integration would be nice. > > > > > > > > > > There are many things that do not work yet. Currently, only shuffle > > and > > > > > fieldsGrouping is supported (and only Storm's default output > stream). > > > > > Furthermore, topologies must be "simple", ie, they cannot be > > configured > > > > > with a Config object and Storm meta information (ie, > TopologyContext) > > > is > > > > > also not there. > > > > > > > > > > The layer can be used to integrate existing Spouts and/or Bolts as > > > Flink > > > > > operators into a regular Flink program. Furthermore, a whole Storm > > > > > topology can be executed by switching from TopologyBuilder to > > > > > FlinkTopologyBuilder and from > > LocalCluster/NimbusClient/StormSubmitter > > > > > to FlinkLocalCluster/FlinkClient/FlinkStormSubmitter. > > > > > > > > > > Examples of both cases are given as ITCases. > > > > > > > > > > > > > > > Cheers, > > > > > Matthias > > > > > > > > > > > > > > > > > > > >
