Hi Stephan, Big +1 for adding stateful functions to Flink. I believe a lot of user would be interested to try this out and I could imagine how this could contribute to reduce the TCO for business requiring both streaming processing and stateful functions.
And my 2 cents is to put it into flink core repository since I could see a tight connection between this library and flink state. Best Regards, Yu On Sat, 12 Oct 2019 at 17:31, jincheng sun <sunjincheng...@gmail.com> wrote: > Hi Stephan, > > bit +1 for adding this great features to Apache Flink. > > Regarding where we should place it, put it into Flink core repository or > create a separate repository? I prefer put it into main repository and > looking forward the more detail discussion for this decision. > > Best, > Jincheng > > > Jingsong Li <jingsongl...@gmail.com> 于2019年10月12日周六 上午11:32写道： > >> Hi Stephan, >> >> big +1 for this contribution. It provides another user interface that is >> easy to use and popular at this time. these functions, It's hard for users >> to write in SQL/TableApi, while using DataStream is too complex. (We've >> done some stateFun kind jobs using DataStream before). With statefun, it is >> very easy. >> >> I think it's also a good opportunity to exercise Flink's core >> capabilities. I looked at stateful-functions-flink briefly, it is very >> interesting. I think there are many other things Flink can improve. So I >> think it's a better thing to put it into Flink, and the improvement for it >> will be more natural in the future. >> >> Best, >> Jingsong Lee >> >> On Fri, Oct 11, 2019 at 7:33 PM Dawid Wysakowicz <dwysakow...@apache.org> >> wrote: >> >>> Hi Stephan, >>> >>> I think this is a nice library, but what I like more about it is that it >>> suggests exploring different use-cases. I think it definitely makes sense >>> for the Flink community to explore more lightweight applications that >>> reuses resources. Therefore I definitely think it is a good idea for Flink >>> community to accept this contribution and help maintaining it. >>> >>> Personally I'd prefer to have it in a separate repository. There were a >>> few discussions before where different people were suggesting to extract >>> connectors and other libraries to separate repositories. Moreover I think >>> it could serve as an example for the Flink ecosystem website[1]. This could >>> be the first project in there and give a good impression that the community >>> sees potential in the ecosystem website. >>> >>> Lastly, I'm wondering if this should go through PMC vote according to >>> our bylaws[2]. In the end the suggestion is to adopt an existing code base >>> as is. It also proposes a new programs concept that could result in a shift >>> of priorities for the community in a long run. >>> >>> Best, >>> >>> Dawid >>> >>> [1] >>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Create-a-Flink-ecosystem-website-td27519.html >>> >>> [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws >>> On 11/10/2019 13:12, Till Rohrmann wrote: >>> >>> Hi Stephan, >>> >>> +1 for adding stateful functions to Flink. I believe the new set of >>> applications this feature will unlock will be super interesting for new and >>> existing Flink users alike. >>> >>> One reason for not including it in the main repository would to not >>> being bound to Flink's release cadence. This would allow to release faster >>> and more often. However, I believe that having it eventually in Flink's >>> main repository would be beneficial in the long run. >>> >>> Cheers, >>> Till >>> >>> On Fri, Oct 11, 2019 at 12:56 PM Trevor Grant <trevor.d.gr...@gmail.com> >>> wrote: >>> >>>> +1 non-binding on contribution. >>>> >>>> Separate repo, or feature branch to start maybe? I just feel like in >>>> the beginning this thing is going to have lots of breaking changes that >>>> maybe aren't going to fit well with tests / other "v1+" release code. Just >>>> my .02. >>>> >>>> >>>> >>>> On Fri, Oct 11, 2019 at 4:38 AM Stephan Ewen <se...@apache.org> wrote: >>>> >>>>> Dear Flink Community! >>>>> >>>>> Some of you probably heard it already: On Tuesday, at Flink Forward >>>>> Berlin, we announced **Stateful Functions**. >>>>> >>>>> Stateful Functions is a library on Flink to implement general purpose >>>>> applications. It is built around stateful functions (who would have thunk) >>>>> that can communicate arbitrarily through messages, have consistent >>>>> state, and a small resource footprint. They are a bit like keyed >>>>> ProcessFunctions >>>>> that can send each other messages. >>>>> As simple as this sounds, this means you can now communicate in >>>>> non-DAG patterns, so it allows users to build programs they cannot build >>>>> with Flink. >>>>> It also has other neat properties, like multiplexing of functions, >>>>> modular composition, tooling both container-based deployments and >>>>> as-a-Flink-job deployments. >>>>> >>>>> You can find out more about it here >>>>> - Website: https://statefun.io/ >>>>> - Code: https://github.com/ververica/stateful-functions >>>>> - Talk with motivation: >>>>> https://speakerdeck.com/stephanewen/stateful-functions-building-general-purpose-applications-and-services-on-apache-flink?slide=12 >>>>> >>>>> >>>>> Now for the main issue: **We would like to contribute this project to >>>>> Apache Flink** >>>>> >>>>> I believe that this is a great fit for both sides. >>>>> For the Flink community, it would be a way to extend the capabilities >>>>> and use cases of Flink into a completely different type of applications >>>>> and >>>>> thus grow the community into this new field. >>>>> Many discussions recently about evolving the Flink runtime (both on >>>>> the mailing list and at conferences) show the interest in Flink users in >>>>> the space that Stateful Functions covers. >>>>> It seems natural that Stateful Functions should closely co-develop >>>>> with Apache Flink, ideally as part of the project. >>>>> >>>>> There are many details to be discusses, for example whether this >>>>> should be added to the Flink core repository, or whether we and to create >>>>> a >>>>> separate repository >>>>> for this. But I think we should start discussing this after we have >>>>> consensus on whether the community wants this contribution. >>>>> >>>>> Really looking forward to hear what you think! >>>>> >>>>> Best Regards, >>>>> Stephan >>>>> >>>>> >> >> -- >> Best, Jingsong Lee >> >
