Re: Regarding Stateful Functions

2021-05-13 Thread Jessy Ping
Hi Austin, Thanks for your insights. We are currently following a microservice architecture for accomplishing our data processing requirements. We are planning to use Flink as our unified platform for all data processing tasks. Although most of our use cases are a suitable fit for Flink, there

Re: Regarding Stateful Functions

2021-05-12 Thread Austin Cawley-Edwards
Hey Jessy, I'm not a Statefun expert but, hopefully, I can point you in the right direction for some of your questions. I'll also cc Gordan, who helps to maintain Statefun. *1. Is the stateful function a good candidate for a system(as above) that > should process incoming requests at the rate of

Regarding Stateful Functions

2021-05-12 Thread Jessy Ping
Hi all, I have gone through the stateful function's documentation and required some expert advice or clarification regarding the following points. *Note: My data processing flow is as follows,* *ingress(10k/s)--> First transformation based on certain static rules --> second transformation bas

Re: questions regarding stateful functions

2021-04-07 Thread Marco Villalobos
That is a more round-abou then I hoped for. I was hoping that a process function could call a stateful function and receive and act upon the response, but that's not the case. Thank you. On Wed, Apr 7, 2021 at 11:38 AM Igal Shilman wrote: > Yes it is possible. A process function upstream to a

Re: questions regarding stateful functions

2021-04-07 Thread Igal Shilman
Yes it is possible. A process function upstream to a stateful function can emit a message that in turn will be routed to that function using the data stream integration. On Wed, Apr 7, 2021 at 7:16 PM Marco Villalobos wrote: > Thank you for the clarification. > > BUTthere was one question n

Re: questions regarding stateful functions

2021-04-07 Thread Marco Villalobos
Thank you for the clarification. BUTthere was one question not addressed: Can a stateful function be called by a process function? On Wed, Apr 7, 2021 at 8:19 AM Igal Shilman wrote: > Hello Marco! > > Your understanding is correct, but in addition > You can also use StateFun within a DataS

Re: questions regarding stateful functions

2021-04-07 Thread Igal Shilman
Hello Marco! Your understanding is correct, but in addition You can also use StateFun within a DataStream application [1] [1] https://ci.apache.org/projects/flink/flink-statefun-docs-master/docs/sdk/flink-datastream/ On Wed, Apr 7, 2021 at 2:49 AM Marco Villalobos wrote: > Upon reading about s

questions regarding stateful functions

2021-04-06 Thread Marco Villalobos
Upon reading about stateful functions, it seems as though first, a data stream has to flow to an event ingress. Then, the stateful functions will perform computations via whatever functionality it provides. Finally, the results of said computations will flow to the event egress which will be yet a