That’s pretty nice Matthias, we could use a compositional API in streaming that many people are familiar with. I can also help in some parts, I see some issues we already encountered while creating the samoa adapter (eg. dealing with circles in the topology). Thanks again for initiating this!
Paris > On 02 Apr 2015, at 23:14, Gyula Fóra <gyf...@apache.org> wrote: > > This sounds amazing :) thanks Matthias! > > Tomorrow I will spend some time to look through your work and give some > comments. > > Also I would love to help with this effort so once we merge an initial > prototype let's open some Jiras and I will pick some up :) > > Gyula > > On Thursday, April 2, 2015, Márton Balassi <balassi.mar...@gmail.com> wrote: > >> Hey Mathias, >> >> Thanks, this is a really nice contribution. I just scrolled through the >> code, but I really like it and big thanks for the the tests for the >> examples. >> >> The rebase Fabian suggested would help a lot when merging. >> >> >> >> On Thu, Apr 2, 2015 at 9:19 PM, Fabian Hueske <fhue...@gmail.com >> <javascript:;>> wrote: >> >>> Hi Matthias, >>> >>> this is really cool!I especially like that you can use Storm code within >> a >>> Flink streaming program :-) >>> >>> One thing that might be good to do rather soon is to collect all your >>> commits and put them on top of a fresh forked Flink master branch. >>> When merging we cannot change the history and try to put only >> fast-forward >>> commits on top of the master branch. >>> As time goes on it becomes more likely that you run into merge issues >> when >>> cherry-picking the commits. >>> >>> 2015-04-02 21:09 GMT+02:00 Robert Metzger <rmetz...@apache.org >> <javascript:;>>: >>> >>>> Hey Henry, >>>> >>>> you can check out the files here: >>>> >>>> >>> >> https://github.com/mjsax/flink/tree/flink-storm-compatibility/flink-staging/flink-streaming/flink-storm-compatibility >>>> ... so yes, they are located in the flink-streaming directory .. which >>> is a >>>> good place for now. >>>> Once we move flink-streaming out of staging, we might want to keep the >>>> storm compat in staging. >>>> >>>> >>>> On Thu, Apr 2, 2015 at 8:16 PM, Henry Saputra <henry.sapu...@gmail.com >> <javascript:;>> >>>> wrote: >>>> >>>>> HI Matthias, >>>>> >>>>> Where do you put the code for the Storm compatibility? Under streams >>>>> module directory? >>>>> >>>>> - Henry >>>>> >>>>> On Thu, Apr 2, 2015 at 10:31 AM, Matthias J. Sax >>>>> <mj...@informatik.hu-berlin.de <javascript:;>> wrote: >>>>>> Hi @all, >>>>>> >>>>>> I started to work on an compatibility layer to run Storm Topologies >>> on >>>>>> Flink. I just pushed a first beta: >>>>>> https://github.com/mjsax/flink/tree/flink-storm-compatibility >>>>>> >>>>>> Please check it out, and let me know how you like it. In this first >>>>>> version, I tried to code without changing too many things in the >>>>>> existing code. However, a deeper integration would be nice. >>>>>> >>>>>> There are many things that do not work yet. Currently, only shuffle >>> and >>>>>> fieldsGrouping is supported (and only Storm's default output >> stream). >>>>>> Furthermore, topologies must be "simple", ie, they cannot be >>> configured >>>>>> with a Config object and Storm meta information (ie, >> TopologyContext) >>>> is >>>>>> also not there. >>>>>> >>>>>> The layer can be used to integrate existing Spouts and/or Bolts as >>>> Flink >>>>>> operators into a regular Flink program. Furthermore, a whole Storm >>>>>> topology can be executed by switching from TopologyBuilder to >>>>>> FlinkTopologyBuilder and from >>> LocalCluster/NimbusClient/StormSubmitter >>>>>> to FlinkLocalCluster/FlinkClient/FlinkStormSubmitter. >>>>>> >>>>>> Examples of both cases are given as ITCases. >>>>>> >>>>>> >>>>>> Cheers, >>>>>> Matthias >>>>>> >>>>>> >>>>> >>>> >>> >>
