Thanks, Matthias. Yes, please. :) On Mon, Apr 6, 2015 at 3:40 PM, Matthias J. Sax < mj...@informatik.hu-berlin.de> wrote:
> Done. Shall I open a pull request? > > -Matthias > > > On 04/03/2015 09:32 PM, Robert Metzger wrote: > > As far as I understood git rebase [1], cherry-picking all changes in > order > > to the current master is exactly equal to "git rebase flink/master". > > The problem is that you have to resolve all conflicts again. But in this > > case the changes to existing code are pretty small, so that might > actually > > work conflict-free. > > > > > > [1] http://git-scm.com/docs/git-rebase > > > > On Fri, Apr 3, 2015 at 9:13 PM, Fabian Hueske <fhue...@gmail.com> wrote: > > > >> Right now, your commits in your working branch are mixed with commits > which > >> are already in pushed to the master branch. > >> Merging this branch to the master branch in order to push it to our > master > >> might turn out into a complex merging process. > >> Merging becomes far easier for us, if all commits that you add are > placed > >> on top of the latest Flink master branch. > >> > >> To setup a working branch, I would start with the latest Flink master > >> branch and cherry-pick your commits from first to last to that branch. > If > >> you want to update the Flink master branch, pull the changes to the > master > >> branch and rebase your working branch onto the master branch. That way > your > >> commit will be placed on top again. > >> > >> 2015-04-03 0:03 GMT+02:00 Paris Carbone <par...@kth.se>: > >> > >>> That’s pretty nice Matthias, we could use a compositional API in > >> streaming > >>> that many people are familiar with. > >>> I can also help in some parts, I see some issues we already encountered > >>> while creating the samoa adapter (eg. dealing with circles in the > >>> topology). Thanks again for initiating this! > >>> > >>> Paris > >>> > >>>> On 02 Apr 2015, at 23:14, Gyula Fóra <gyf...@apache.org> wrote: > >>>> > >>>> This sounds amazing :) thanks Matthias! > >>>> > >>>> Tomorrow I will spend some time to look through your work and give > some > >>>> comments. > >>>> > >>>> Also I would love to help with this effort so once we merge an initial > >>>> prototype let's open some Jiras and I will pick some up :) > >>>> > >>>> Gyula > >>>> > >>>> On Thursday, April 2, 2015, Márton Balassi <balassi.mar...@gmail.com> > >>> wrote: > >>>> > >>>>> Hey Mathias, > >>>>> > >>>>> Thanks, this is a really nice contribution. I just scrolled through > >> the > >>>>> code, but I really like it and big thanks for the the tests for the > >>>>> examples. > >>>>> > >>>>> The rebase Fabian suggested would help a lot when merging. > >>>>> > >>>>> > >>>>> > >>>>> On Thu, Apr 2, 2015 at 9:19 PM, Fabian Hueske <fhue...@gmail.com > >>>>> <javascript:;>> wrote: > >>>>> > >>>>>> Hi Matthias, > >>>>>> > >>>>>> this is really cool!I especially like that you can use Storm code > >>> within > >>>>> a > >>>>>> Flink streaming program :-) > >>>>>> > >>>>>> One thing that might be good to do rather soon is to collect all > your > >>>>>> commits and put them on top of a fresh forked Flink master branch. > >>>>>> When merging we cannot change the history and try to put only > >>>>> fast-forward > >>>>>> commits on top of the master branch. > >>>>>> As time goes on it becomes more likely that you run into merge > issues > >>>>> when > >>>>>> cherry-picking the commits. > >>>>>> > >>>>>> 2015-04-02 21:09 GMT+02:00 Robert Metzger <rmetz...@apache.org > >>>>> <javascript:;>>: > >>>>>> > >>>>>>> Hey Henry, > >>>>>>> > >>>>>>> you can check out the files here: > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >> > https://github.com/mjsax/flink/tree/flink-storm-compatibility/flink-staging/flink-streaming/flink-storm-compatibility > >>>>>>> ... so yes, they are located in the flink-streaming directory .. > >> which > >>>>>> is a > >>>>>>> good place for now. > >>>>>>> Once we move flink-streaming out of staging, we might want to keep > >> the > >>>>>>> storm compat in staging. > >>>>>>> > >>>>>>> > >>>>>>> On Thu, Apr 2, 2015 at 8:16 PM, Henry Saputra < > >>> henry.sapu...@gmail.com > >>>>> <javascript:;>> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> HI Matthias, > >>>>>>>> > >>>>>>>> Where do you put the code for the Storm compatibility? Under > >> streams > >>>>>>>> module directory? > >>>>>>>> > >>>>>>>> - Henry > >>>>>>>> > >>>>>>>> On Thu, Apr 2, 2015 at 10:31 AM, Matthias J. Sax > >>>>>>>> <mj...@informatik.hu-berlin.de <javascript:;>> wrote: > >>>>>>>>> Hi @all, > >>>>>>>>> > >>>>>>>>> I started to work on an compatibility layer to run Storm > >> Topologies > >>>>>> on > >>>>>>>>> Flink. I just pushed a first beta: > >>>>>>>>> https://github.com/mjsax/flink/tree/flink-storm-compatibility > >>>>>>>>> > >>>>>>>>> Please check it out, and let me know how you like it. In this > >> first > >>>>>>>>> version, I tried to code without changing too many things in the > >>>>>>>>> existing code. However, a deeper integration would be nice. > >>>>>>>>> > >>>>>>>>> There are many things that do not work yet. Currently, only > >> shuffle > >>>>>> and > >>>>>>>>> fieldsGrouping is supported (and only Storm's default output > >>>>> stream). > >>>>>>>>> Furthermore, topologies must be "simple", ie, they cannot be > >>>>>> configured > >>>>>>>>> with a Config object and Storm meta information (ie, > >>>>> TopologyContext) > >>>>>>> is > >>>>>>>>> also not there. > >>>>>>>>> > >>>>>>>>> The layer can be used to integrate existing Spouts and/or Bolts > as > >>>>>>> Flink > >>>>>>>>> operators into a regular Flink program. Furthermore, a whole > Storm > >>>>>>>>> topology can be executed by switching from TopologyBuilder to > >>>>>>>>> FlinkTopologyBuilder and from > >>>>>> LocalCluster/NimbusClient/StormSubmitter > >>>>>>>>> to FlinkLocalCluster/FlinkClient/FlinkStormSubmitter. > >>>>>>>>> > >>>>>>>>> Examples of both cases are given as ITCases. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Cheers, > >>>>>>>>> Matthias > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>> > >>> > >> > > > >
